From lp_benchmark_robot at intel.com Mon Aug 1 06:04:16 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Mon, 1 Aug 2016 11:04:16 +0100 Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-08-01 Message-ID: Results for project Python 2.7, build date 2016-08-01 02:48:19 +0000 commit: ba915d561667 previous commit: c8c1ea94379a revision date: 2016-07-30 12:49:53 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.13% -1.44% 3.69% 6.87% :-) pybench 0.18% -0.17% 5.70% 4.95% :-( regex_v8 0.65% -0.15% -2.56% 10.28% :-) nbody 0.27% -0.13% 8.51% 0.88% :-| json_dump_v2 0.33% 0.94% 1.95% 11.49% :-( normal_startup 1.91% -0.15% -6.14% 1.94% :-) ssbench 0.15% 0.84% 2.76% 1.17% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-08-01/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Mon Aug 1 08:57:37 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Mon, 1 Aug 2016 13:57:37 +0100 Subject: [Python-checkins] UGLY Benchmark Results for Python Default 2016-08-01 Message-ID: Results for project Python default, build date 2016-08-01 02:02:04 +0000 commit: 6c2e2de5ab8e previous commit: 9b135b8d3166 revision date: 2016-07-31 06:51:13 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.17% 0.34% 10.14% 17.15% :-| pybench 0.10% -0.22% 1.66% 8.28% :-( regex_v8 2.71% 0.46% -2.83% 4.11% :-| nbody 0.12% 1.31% -0.61% 11.03% :-| json_dump_v2 0.36% 0.31% -1.17% 12.73% :-( normal_startup 0.72% -2.42% -2.35% 6.32% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-default-2016-08-01/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Tue Aug 2 11:07:16 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 2 Aug 2016 16:07:16 +0100 Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-08-02 Message-ID: <1833fe3a-0ffa-43ff-b1f0-e42e0743a239@irsmsx106.ger.corp.intel.com> No new revisions. Here are the previous results: Results for project Python 2.7, build date 2016-08-02 02:03:39 +0000 commit: ba915d561667 previous commit: c8c1ea94379a revision date: 2016-07-30 12:49:53 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.13% -1.44% 3.69% 6.87% :-) pybench 0.18% -0.17% 5.70% 4.95% :-( regex_v8 0.65% -0.15% -2.56% 10.28% :-) nbody 0.27% -0.13% 8.51% 0.88% :-| json_dump_v2 0.33% 0.94% 1.95% 11.49% :-( normal_startup 1.91% -0.15% -6.14% 1.94% :-) ssbench 0.15% 0.84% 2.76% 1.17% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-08-02/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Tue Aug 2 11:10:34 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 2 Aug 2016 16:10:34 +0100 Subject: [Python-checkins] UGLY Benchmark Results for Python Default 2016-08-02 Message-ID: No new revisions. Here are the previous results: Results for project Python default, build date 2016-08-02 02:01:46 +0000 commit: 6c2e2de5ab8e previous commit: 9b135b8d3166 revision date: 2016-07-31 06:51:13 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.17% 0.34% 10.14% 17.15% :-| pybench 0.10% -0.22% 1.66% 8.28% :-( regex_v8 2.71% 0.46% -2.83% 4.11% :-| nbody 0.12% 1.31% -0.61% 11.03% :-| json_dump_v2 0.36% 0.31% -1.17% 12.73% :-( normal_startup 0.72% -2.42% -2.35% 6.32% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-default-2016-08-02/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Tue Aug 2 15:52:56 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 02 Aug 2016 19:52:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2322557=3A_Now_impo?= =?utf-8?q?rting_already_imported_modules_is_up_to_2=2E5_times_faster=2E?= Message-ID: <20160802195256.1412.61045.309C482D@psf.io> https://hg.python.org/cpython/rev/64f195790a3a changeset: 102506:64f195790a3a user: Serhiy Storchaka date: Tue Aug 02 22:51:21 2016 +0300 summary: Issue #22557: Now importing already imported modules is up to 2.5 times faster. files: Include/pystate.h | 1 + Misc/NEWS | 2 + Python/ceval.c | 79 ++++-- Python/import.c | 395 +++++++++++++----------------- Python/pylifecycle.c | 5 + Python/pystate.c | 2 + 6 files changed, 238 insertions(+), 246 deletions(-) diff --git a/Include/pystate.h b/Include/pystate.h --- a/Include/pystate.h +++ b/Include/pystate.h @@ -41,6 +41,7 @@ #endif PyObject *builtins_copy; + PyObject *import_func; } PyInterpreterState; #endif diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #22557: Now importing already imported modules is up to 2.5 times faster. + - Issue #17596: Include to help with Min GW building. - Issue #27507: Add integer overflow check in bytearray.extend(). Patch by diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -139,6 +139,7 @@ PyThreadState *, PyFrameObject *, int *, int *, int *); static PyObject * cmp_outcome(int, PyObject *, PyObject *); +static PyObject * import_name(PyFrameObject *, PyObject *, PyObject *, PyObject *); static PyObject * import_from(PyObject *, PyObject *); static int import_all_from(PyObject *, PyObject *); static void format_exc_check_arg(PyObject *, const char *, PyObject *); @@ -2808,37 +2809,15 @@ } TARGET(IMPORT_NAME) { - _Py_IDENTIFIER(__import__); PyObject *name = GETITEM(names, oparg); - PyObject *func = _PyDict_GetItemId(f->f_builtins, &PyId___import__); - PyObject *from, *level, *args, *res; - if (func == NULL) { - PyErr_SetString(PyExc_ImportError, - "__import__ not found"); - goto error; - } - Py_INCREF(func); - from = POP(); - level = TOP(); - args = PyTuple_Pack(5, - name, - f->f_globals, - f->f_locals == NULL ? - Py_None : f->f_locals, - from, - level); + PyObject *fromlist = POP(); + PyObject *level = TOP(); + PyObject *res; + READ_TIMESTAMP(intr0); + res = import_name(f, name, fromlist, level); Py_DECREF(level); - Py_DECREF(from); - if (args == NULL) { - Py_DECREF(func); - STACKADJ(-1); - goto error; - } - READ_TIMESTAMP(intr0); - res = PyEval_CallObject(func, args); + Py_DECREF(fromlist); READ_TIMESTAMP(intr1); - Py_DECREF(args); - Py_DECREF(func); SET_TOP(res); if (res == NULL) goto error; @@ -5159,6 +5138,50 @@ } static PyObject * +import_name(PyFrameObject *f, PyObject *name, PyObject *fromlist, PyObject *level) +{ + _Py_IDENTIFIER(__import__); + PyObject *import_func, *args, *res; + + import_func = _PyDict_GetItemId(f->f_builtins, &PyId___import__); + if (import_func == NULL) { + PyErr_SetString(PyExc_ImportError, "__import__ not found"); + return NULL; + } + + /* Fast path for not overloaded __import__. */ + if (import_func == PyThreadState_GET()->interp->import_func) { + int ilevel = _PyLong_AsInt(level); + if (ilevel == -1 && PyErr_Occurred()) { + return NULL; + } + res = PyImport_ImportModuleLevelObject( + name, + f->f_globals, + f->f_locals == NULL ? Py_None : f->f_locals, + fromlist, + ilevel); + return res; + } + + Py_INCREF(import_func); + args = PyTuple_Pack(5, + name, + f->f_globals, + f->f_locals == NULL ? Py_None : f->f_locals, + fromlist, + level); + if (args == NULL) { + Py_DECREF(import_func); + return NULL; + } + res = PyEval_CallObject(import_func, args); + Py_DECREF(args); + Py_DECREF(import_func); + return res; +} + +static PyObject * import_from(PyObject *v, PyObject *name) { PyObject *x; diff --git a/Python/import.c b/Python/import.c --- a/Python/import.c +++ b/Python/import.c @@ -1336,61 +1336,170 @@ } -PyObject * -PyImport_ImportModuleLevelObject(PyObject *name, PyObject *given_globals, - PyObject *locals, PyObject *given_fromlist, - int level) +static PyObject * +resolve_name(PyObject *name, PyObject *globals, int level) { - _Py_IDENTIFIER(__import__); _Py_IDENTIFIER(__spec__); - _Py_IDENTIFIER(_initializing); _Py_IDENTIFIER(__package__); _Py_IDENTIFIER(__path__); _Py_IDENTIFIER(__name__); - _Py_IDENTIFIER(_find_and_load); - _Py_IDENTIFIER(_handle_fromlist); - _Py_IDENTIFIER(_lock_unlock_module); - PyObject *abs_name = NULL; - PyObject *builtins_import = NULL; - PyObject *final_mod = NULL; - PyObject *mod = NULL; + _Py_IDENTIFIER(parent); + PyObject *abs_name; PyObject *package = NULL; - PyObject *spec = NULL; - PyObject *globals = NULL; - PyObject *fromlist = NULL; + PyObject *spec; PyInterpreterState *interp = PyThreadState_GET()->interp; - int has_from; + Py_ssize_t last_dot; + PyObject *base; + int level_up; - /* Make sure to use default values so as to not have - PyObject_CallMethodObjArgs() truncate the parameter list because of a - NULL argument. */ - if (given_globals == NULL) { - globals = PyDict_New(); - if (globals == NULL) { + if (globals == NULL) { + PyErr_SetString(PyExc_KeyError, "'__name__' not in globals"); + goto error; + } + if (!PyDict_Check(globals)) { + PyErr_SetString(PyExc_TypeError, "globals must be a dict"); + goto error; + } + package = _PyDict_GetItemId(globals, &PyId___package__); + if (package == Py_None) { + package = NULL; + } + spec = _PyDict_GetItemId(globals, &PyId___spec__); + + if (package != NULL) { + Py_INCREF(package); + if (!PyUnicode_Check(package)) { + PyErr_SetString(PyExc_TypeError, "package must be a string"); + goto error; + } + else if (spec != NULL && spec != Py_None) { + int equal; + PyObject *parent = _PyObject_GetAttrId(spec, &PyId_parent); + if (parent == NULL) { + goto error; + } + + equal = PyObject_RichCompareBool(package, parent, Py_EQ); + Py_DECREF(parent); + if (equal < 0) { + goto error; + } + else if (equal == 0) { + if (PyErr_WarnEx(PyExc_ImportWarning, + "__package__ != __spec__.parent", 1) < 0) { + goto error; + } + } + } + } + else if (spec != NULL && spec != Py_None) { + package = _PyObject_GetAttrId(spec, &PyId_parent); + if (package == NULL) { + goto error; + } + else if (!PyUnicode_Check(package)) { + PyErr_SetString(PyExc_TypeError, + "__spec__.parent must be a string"); goto error; } } else { - /* Only have to care what given_globals is if it will be used - for something. */ - if (level > 0 && !PyDict_Check(given_globals)) { - PyErr_SetString(PyExc_TypeError, "globals must be a dict"); + if (PyErr_WarnEx(PyExc_ImportWarning, + "can't resolve package from __spec__ or __package__, " + "falling back on __name__ and __path__", 1) < 0) { goto error; } - globals = given_globals; - Py_INCREF(globals); + + package = _PyDict_GetItemId(globals, &PyId___name__); + if (package == NULL) { + PyErr_SetString(PyExc_KeyError, "'__name__' not in globals"); + goto error; + } + + Py_INCREF(package); + if (!PyUnicode_Check(package)) { + PyErr_SetString(PyExc_TypeError, "__name__ must be a string"); + goto error; + } + + if (_PyDict_GetItemId(globals, &PyId___path__) == NULL) { + Py_ssize_t dot; + + if (PyUnicode_READY(package) < 0) { + goto error; + } + + dot = PyUnicode_FindChar(package, '.', + 0, PyUnicode_GET_LENGTH(package), -1); + if (dot == -2) { + goto error; + } + + if (dot >= 0) { + PyObject *substr = PyUnicode_Substring(package, 0, dot); + if (substr == NULL) { + goto error; + } + Py_SETREF(package, substr); + } + } } - if (given_fromlist == NULL) { - fromlist = PyList_New(0); - if (fromlist == NULL) { + last_dot = PyUnicode_GET_LENGTH(package); + if (last_dot == 0) { + PyErr_SetString(PyExc_ImportError, + "attempted relative import with no known parent package"); + goto error; + } + else if (PyDict_GetItem(interp->modules, package) == NULL) { + PyErr_Format(PyExc_SystemError, + "Parent module %R not loaded, cannot perform relative " + "import", package); + goto error; + } + + for (level_up = 1; level_up < level; level_up += 1) { + last_dot = PyUnicode_FindChar(package, '.', 0, last_dot, -1); + if (last_dot == -2) { + goto error; + } + else if (last_dot == -1) { + PyErr_SetString(PyExc_ValueError, + "attempted relative import beyond top-level " + "package"); goto error; } } - else { - fromlist = given_fromlist; - Py_INCREF(fromlist); + + base = PyUnicode_Substring(package, 0, last_dot); + Py_DECREF(package); + if (base == NULL || PyUnicode_GET_LENGTH(name) == 0) { + return base; } + + abs_name = PyUnicode_FromFormat("%U.%U", base, name); + Py_DECREF(base); + return abs_name; + + error: + Py_XDECREF(package); + return NULL; +} + +PyObject * +PyImport_ImportModuleLevelObject(PyObject *name, PyObject *globals, + PyObject *locals, PyObject *fromlist, + int level) +{ + _Py_IDENTIFIER(_find_and_load); + _Py_IDENTIFIER(_handle_fromlist); + PyObject *abs_name = NULL; + PyObject *final_mod = NULL; + PyObject *mod = NULL; + PyObject *package = NULL; + PyInterpreterState *interp = PyThreadState_GET()->interp; + int has_from; + if (name == NULL) { PyErr_SetString(PyExc_ValueError, "Empty module name"); goto error; @@ -1403,170 +1512,28 @@ PyErr_SetString(PyExc_TypeError, "module name must be a string"); goto error; } - else if (PyUnicode_READY(name) < 0) { + if (PyUnicode_READY(name) < 0) { goto error; } if (level < 0) { PyErr_SetString(PyExc_ValueError, "level must be >= 0"); goto error; } - else if (level > 0) { - package = _PyDict_GetItemId(globals, &PyId___package__); - spec = _PyDict_GetItemId(globals, &PyId___spec__); - if (package != NULL && package != Py_None) { - Py_INCREF(package); - if (!PyUnicode_Check(package)) { - PyErr_SetString(PyExc_TypeError, "package must be a string"); - goto error; - } - else if (spec != NULL && spec != Py_None) { - int equal; - PyObject *parent = PyObject_GetAttrString(spec, "parent"); - if (parent == NULL) { - goto error; - } - - equal = PyObject_RichCompareBool(package, parent, Py_EQ); - Py_DECREF(parent); - if (equal < 0) { - goto error; - } - else if (equal == 0) { - if (PyErr_WarnEx(PyExc_ImportWarning, - "__package__ != __spec__.parent", 1) < 0) { - goto error; - } - } - } - } - else if (spec != NULL && spec != Py_None) { - package = PyObject_GetAttrString(spec, "parent"); - if (package == NULL) { - goto error; - } - else if (!PyUnicode_Check(package)) { - PyErr_SetString(PyExc_TypeError, - "__spec__.parent must be a string"); - goto error; - } - } - else { - package = NULL; - if (PyErr_WarnEx(PyExc_ImportWarning, - "can't resolve package from __spec__ or __package__, " - "falling back on __name__ and __path__", 1) < 0) { - goto error; - } - - package = _PyDict_GetItemId(globals, &PyId___name__); - if (package == NULL) { - PyErr_SetString(PyExc_KeyError, "'__name__' not in globals"); - goto error; - } - - Py_INCREF(package); - if (!PyUnicode_Check(package)) { - PyErr_SetString(PyExc_TypeError, "__name__ must be a string"); - goto error; - } - - if (_PyDict_GetItemId(globals, &PyId___path__) == NULL) { - Py_ssize_t dot; - - if (PyUnicode_READY(package) < 0) { - goto error; - } - - dot = PyUnicode_FindChar(package, '.', - 0, PyUnicode_GET_LENGTH(package), -1); - if (dot == -2) { - goto error; - } - - if (dot >= 0) { - PyObject *substr = PyUnicode_Substring(package, 0, dot); - if (substr == NULL) { - goto error; - } - Py_SETREF(package, substr); - } - } - } - - if (PyUnicode_CompareWithASCIIString(package, "") == 0) { - PyErr_SetString(PyExc_ImportError, - "attempted relative import with no known parent package"); + if (level > 0) { + abs_name = resolve_name(name, globals, level); + if (abs_name == NULL) goto error; - } - else if (PyDict_GetItem(interp->modules, package) == NULL) { - PyErr_Format(PyExc_SystemError, - "Parent module %R not loaded, cannot perform relative " - "import", package); - goto error; - } } else { /* level == 0 */ if (PyUnicode_GET_LENGTH(name) == 0) { PyErr_SetString(PyExc_ValueError, "Empty module name"); goto error; } - package = Py_None; - Py_INCREF(package); - } - - if (level > 0) { - Py_ssize_t last_dot = PyUnicode_GET_LENGTH(package); - PyObject *base = NULL; - int level_up = 1; - - for (level_up = 1; level_up < level; level_up += 1) { - last_dot = PyUnicode_FindChar(package, '.', 0, last_dot, -1); - if (last_dot == -2) { - goto error; - } - else if (last_dot == -1) { - PyErr_SetString(PyExc_ValueError, - "attempted relative import beyond top-level " - "package"); - goto error; - } - } - - base = PyUnicode_Substring(package, 0, last_dot); - if (base == NULL) - goto error; - - if (PyUnicode_GET_LENGTH(name) > 0) { - abs_name = PyUnicode_FromFormat("%U.%U", base, name); - Py_DECREF(base); - if (abs_name == NULL) { - goto error; - } - } - else { - abs_name = base; - } - } - else { abs_name = name; Py_INCREF(abs_name); } -#ifdef WITH_THREAD - _PyImport_AcquireLock(); -#endif - /* From this point forward, goto error_with_unlock! */ - /* XXX interp->builtins_copy is NULL in subinterpreter! */ - builtins_import = _PyDict_GetItemId(interp->builtins_copy ? - interp->builtins_copy : - interp->builtins, &PyId___import__); - if (builtins_import == NULL) { - PyErr_SetString(PyExc_ImportError, "__import__ not found"); - goto error_with_unlock; - } - Py_INCREF(builtins_import); - mod = PyDict_GetItem(interp->modules, abs_name); if (mod == Py_None) { PyObject *msg = PyUnicode_FromFormat("import of %R halted; " @@ -1576,9 +1543,12 @@ Py_DECREF(msg); } mod = NULL; - goto error_with_unlock; + goto error; } else if (mod != NULL) { + _Py_IDENTIFIER(__spec__); + _Py_IDENTIFIER(_initializing); + _Py_IDENTIFIER(_lock_unlock_module); PyObject *value = NULL; PyObject *spec; int initializing = 0; @@ -1601,39 +1571,39 @@ Py_DECREF(value); if (initializing == -1) PyErr_Clear(); - } - if (initializing > 0) { - /* _bootstrap._lock_unlock_module() releases the import lock */ - value = _PyObject_CallMethodIdObjArgs(interp->importlib, - &PyId__lock_unlock_module, abs_name, - NULL); - if (value == NULL) - goto error; - Py_DECREF(value); - } - else { + if (initializing > 0) { #ifdef WITH_THREAD - if (_PyImport_ReleaseLock() < 0) { - PyErr_SetString(PyExc_RuntimeError, "not holding the import lock"); - goto error; + _PyImport_AcquireLock(); +#endif + /* _bootstrap._lock_unlock_module() releases the import lock */ + value = _PyObject_CallMethodIdObjArgs(interp->importlib, + &PyId__lock_unlock_module, abs_name, + NULL); + if (value == NULL) + goto error; + Py_DECREF(value); } -#endif } } else { +#ifdef WITH_THREAD + _PyImport_AcquireLock(); +#endif /* _bootstrap._find_and_load() releases the import lock */ mod = _PyObject_CallMethodIdObjArgs(interp->importlib, &PyId__find_and_load, abs_name, - builtins_import, NULL); + interp->import_func, NULL); if (mod == NULL) { goto error; } } - /* From now on we don't hold the import lock anymore. */ - has_from = PyObject_IsTrue(fromlist); - if (has_from < 0) - goto error; + has_from = 0; + if (fromlist != NULL && fromlist != Py_None) { + has_from = PyObject_IsTrue(fromlist); + if (has_from < 0) + goto error; + } if (!has_from) { Py_ssize_t len = PyUnicode_GET_LENGTH(name); if (level == 0 || len > 0) { @@ -1657,7 +1627,7 @@ goto error; } - final_mod = PyObject_CallFunctionObjArgs(builtins_import, front, NULL); + final_mod = PyImport_ImportModuleLevelObject(front, NULL, NULL, NULL, 0); Py_DECREF(front); } else { @@ -1670,15 +1640,14 @@ } final_mod = PyDict_GetItem(interp->modules, to_return); + Py_DECREF(to_return); if (final_mod == NULL) { PyErr_Format(PyExc_KeyError, "%R not in sys.modules as expected", to_return); + goto error; } - else { - Py_INCREF(final_mod); - } - Py_DECREF(to_return); + Py_INCREF(final_mod); } } else { @@ -1689,24 +1658,14 @@ else { final_mod = _PyObject_CallMethodIdObjArgs(interp->importlib, &PyId__handle_fromlist, mod, - fromlist, builtins_import, + fromlist, interp->import_func, NULL); } - goto error; - error_with_unlock: -#ifdef WITH_THREAD - if (_PyImport_ReleaseLock() < 0) { - PyErr_SetString(PyExc_RuntimeError, "not holding the import lock"); - } -#endif error: Py_XDECREF(abs_name); - Py_XDECREF(builtins_import); Py_XDECREF(mod); Py_XDECREF(package); - Py_XDECREF(globals); - Py_XDECREF(fromlist); if (final_mod == NULL) remove_importlib_frames(); return final_mod; diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -254,6 +254,11 @@ interp->importlib = importlib; Py_INCREF(interp->importlib); + interp->import_func = PyDict_GetItemString(interp->builtins, "__import__"); + if (interp->import_func == NULL) + Py_FatalError("Py_Initialize: __import__ not found"); + Py_INCREF(interp->import_func); + /* Import the _imp module */ impmod = PyInit_imp(); if (impmod == NULL) { diff --git a/Python/pystate.c b/Python/pystate.c --- a/Python/pystate.c +++ b/Python/pystate.c @@ -90,6 +90,7 @@ interp->codecs_initialized = 0; interp->fscodec_initialized = 0; interp->importlib = NULL; + interp->import_func = NULL; #ifdef HAVE_DLOPEN #if HAVE_DECL_RTLD_NOW interp->dlopenflags = RTLD_NOW; @@ -128,6 +129,7 @@ Py_CLEAR(interp->builtins); Py_CLEAR(interp->builtins_copy); Py_CLEAR(interp->importlib); + Py_CLEAR(interp->import_func); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 2 16:31:29 2016 From: python-checkins at python.org (stefan.krah) Date: Tue, 02 Aug 2016 20:31:29 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3MjQx?= =?utf-8?q?=3A_Catch_exception_when_running_pstats_as_main=2E?= Message-ID: <20160802203129.23559.83793.8AF8DBE3@psf.io> https://hg.python.org/cpython/rev/d9e25fe35f46 changeset: 102507:d9e25fe35f46 branch: 3.5 parent: 102504:7062eaee3adf user: Stefan Krah date: Tue Aug 02 22:30:24 2016 +0200 summary: Issue #27241: Catch exception when running pstats as main. files: Lib/pstats.py | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/pstats.py b/Lib/pstats.py --- a/Lib/pstats.py +++ b/Lib/pstats.py @@ -574,7 +574,10 @@ def do_add(self, line): if self.stats: - self.stats.add(line) + try: + self.stats.add(line) + except IOError as e: + print("Failed to load statistics for %s: %s" % (line, e), file=self.stream) else: print("No statistics object is loaded.", file=self.stream) return 0 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 2 16:31:29 2016 From: python-checkins at python.org (stefan.krah) Date: Tue, 02 Aug 2016 20:31:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41Lg==?= Message-ID: <20160802203129.71649.32139.9A324433@psf.io> https://hg.python.org/cpython/rev/e6f74b435f19 changeset: 102508:e6f74b435f19 parent: 102506:64f195790a3a parent: 102507:d9e25fe35f46 user: Stefan Krah date: Tue Aug 02 22:30:57 2016 +0200 summary: Merge 3.5. files: Lib/pstats.py | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/pstats.py b/Lib/pstats.py --- a/Lib/pstats.py +++ b/Lib/pstats.py @@ -574,7 +574,10 @@ def do_add(self, line): if self.stats: - self.stats.add(line) + try: + self.stats.add(line) + except IOError as e: + print("Failed to load statistics for %s: %s" % (line, e), file=self.stream) else: print("No statistics object is loaded.", file=self.stream) return 0 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 2 17:49:33 2016 From: python-checkins at python.org (alexander.belopolsky) Date: Tue, 02 Aug 2016 21:49:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2327661=3A_Added_t?= =?utf-8?q?zinfo_keyword_argument_to_datetime=2Ecombine=2E?= Message-ID: <20160802214933.43562.7013.4029BD56@psf.io> https://hg.python.org/cpython/rev/adce94a718e3 changeset: 102509:adce94a718e3 user: Alexander Belopolsky date: Tue Aug 02 17:49:30 2016 -0400 summary: Closes #27661: Added tzinfo keyword argument to datetime.combine. files: Doc/library/datetime.rst | 17 ++++++++--- Lib/datetime.py | 6 ++- Lib/test/datetimetester.py | 13 ++++++++- Misc/NEWS | 2 + Modules/_datetimemodule.c | 36 ++++++++++++++----------- 5 files changed, 50 insertions(+), 24 deletions(-) diff --git a/Doc/library/datetime.rst b/Doc/library/datetime.rst --- a/Doc/library/datetime.rst +++ b/Doc/library/datetime.rst @@ -794,16 +794,23 @@ microsecond of the result are all 0, and :attr:`.tzinfo` is ``None``. -.. classmethod:: datetime.combine(date, time) +.. classmethod:: datetime.combine(date, time[, tzinfo]) Return a new :class:`.datetime` object whose date components are equal to the - given :class:`date` object's, and whose time components and :attr:`.tzinfo` - attributes are equal to the given :class:`.time` object's. For any - :class:`.datetime` object *d*, - ``d == datetime.combine(d.date(), d.timetz())``. If date is a + given :class:`date` object's, and whose time components + are equal to the given :class:`.time` object's. If the *tzinfo* + argument is provided, its value is used to set the :attr:`.tzinfo` attribute + of the result, otherwise the :attr:`~.time.tzinfo` attribute of the *time* argument + is used. + + For any :class:`.datetime` object *d*, + ``d == datetime.combine(d.date(), d.time(), d.tzinfo)``. If date is a :class:`.datetime` object, its time components and :attr:`.tzinfo` attributes are ignored. + .. versionchanged:: 3.6 + Added the *tzinfo* argument. + .. classmethod:: datetime.strptime(date_string, format) diff --git a/Lib/datetime.py b/Lib/datetime.py --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -1479,15 +1479,17 @@ return cls.utcfromtimestamp(t) @classmethod - def combine(cls, date, time): + def combine(cls, date, time, tzinfo=True): "Construct a datetime from a given date and a given time." if not isinstance(date, _date_class): raise TypeError("date argument must be a date instance") if not isinstance(time, _time_class): raise TypeError("time argument must be a time instance") + if tzinfo is True: + tzinfo = time.tzinfo return cls(date.year, date.month, date.day, time.hour, time.minute, time.second, time.microsecond, - time.tzinfo, fold=time.fold) + tzinfo, fold=time.fold) def timetuple(self): "Return local time tuple compatible with time.localtime()." diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -2117,11 +2117,22 @@ self.assertRaises(TypeError, combine) # need an arg self.assertRaises(TypeError, combine, d) # need two args self.assertRaises(TypeError, combine, t, d) # args reversed - self.assertRaises(TypeError, combine, d, t, 1) # too many args + self.assertRaises(TypeError, combine, d, t, 1) # wrong tzinfo type + self.assertRaises(TypeError, combine, d, t, 1, 2) # too many args self.assertRaises(TypeError, combine, "date", "time") # wrong types self.assertRaises(TypeError, combine, d, "time") # wrong type self.assertRaises(TypeError, combine, "date", t) # wrong type + # tzinfo= argument + dt = combine(d, t, timezone.utc) + self.assertIs(dt.tzinfo, timezone.utc) + dt = combine(d, t, tzinfo=timezone.utc) + self.assertIs(dt.tzinfo, timezone.utc) + t = time() + dt = combine(dt, t) + self.assertEqual(dt.date(), d) + self.assertEqual(dt.time(), t) + def test_replace(self): cls = self.theclass args = [1, 2, 3, 4, 5, 6, 7] diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -40,6 +40,8 @@ Library ------- +- Issue #27661: Added tzinfo keyword argument to datetime.combine. + - Issue #27568: Prevent HTTPoxy attack (CVE-2016-1000110). Ignore the HTTP_PROXY variable when REQUEST_METHOD environment is set, which indicates that the script is in CGI mode. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -4430,28 +4430,32 @@ static PyObject * datetime_combine(PyObject *cls, PyObject *args, PyObject *kw) { - static char *keywords[] = {"date", "time", NULL}; + static char *keywords[] = {"date", "time", "tzinfo", NULL}; PyObject *date; PyObject *time; + PyObject *tzinfo = NULL; PyObject *result = NULL; - if (PyArg_ParseTupleAndKeywords(args, kw, "O!O!:combine", keywords, + if (PyArg_ParseTupleAndKeywords(args, kw, "O!O!|O:combine", keywords, &PyDateTime_DateType, &date, - &PyDateTime_TimeType, &time)) { - PyObject *tzinfo = Py_None; - - if (HASTZINFO(time)) - tzinfo = ((PyDateTime_Time *)time)->tzinfo; + &PyDateTime_TimeType, &time, &tzinfo)) { + if (tzinfo == NULL) { + if (HASTZINFO(time)) + tzinfo = ((PyDateTime_Time *)time)->tzinfo; + else + tzinfo = Py_None; + } result = PyObject_CallFunction(cls, "iiiiiiiO", - GET_YEAR(date), - GET_MONTH(date), - GET_DAY(date), - TIME_GET_HOUR(time), - TIME_GET_MINUTE(time), - TIME_GET_SECOND(time), - TIME_GET_MICROSECOND(time), - tzinfo); - DATE_SET_FOLD(result, TIME_GET_FOLD(time)); + GET_YEAR(date), + GET_MONTH(date), + GET_DAY(date), + TIME_GET_HOUR(time), + TIME_GET_MINUTE(time), + TIME_GET_SECOND(time), + TIME_GET_MICROSECOND(time), + tzinfo); + if (result) + DATE_SET_FOLD(result, TIME_GET_FOLD(time)); } return result; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 03:16:54 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 03 Aug 2016 07:16:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326576=3A_Merge_from_3=2E5?= Message-ID: <20160803071653.26341.20170.37C17989@psf.io> https://hg.python.org/cpython/rev/08359651815e changeset: 102511:08359651815e parent: 102509:adce94a718e3 parent: 102510:e0f9f8be7963 user: Berker Peksag date: Wed Aug 03 10:17:51 2016 +0300 summary: Issue #26576: Merge from 3.5 files: Doc/reference/compound_stmts.rst | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -503,11 +503,13 @@ @f2 def func(): pass -is equivalent to :: +is roughly equivalent to :: def func(): pass func = f1(arg)(f2(func)) +except that the original function is not temporarily bound to the name ``func``. + .. index:: triple: default; parameter; value single: argument; function definition @@ -638,14 +640,13 @@ @f2 class Foo: pass -is equivalent to :: +is roughly equivalent to :: class Foo: pass Foo = f1(arg)(f2(Foo)) The evaluation rules for the decorator expressions are the same as for function -decorators. The result must be a class object, which is then bound to the class -name. +decorators. The result is then bound to the class name. **Programmer's note:** Variables defined in the class definition are class attributes; they are shared by instances. Instance attributes can be set in a -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 03:20:14 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 03 Aug 2016 07:20:14 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2NTc2?= =?utf-8?q?=3A_Clarify_that_the_=40deco_syntax_is_not_always_an_equivalent?= =?utf-8?q?_of_f_=3D?= Message-ID: <20160803071653.23447.47861.FDA12586@psf.io> https://hg.python.org/cpython/rev/e0f9f8be7963 changeset: 102510:e0f9f8be7963 branch: 3.5 parent: 102507:d9e25fe35f46 user: Berker Peksag date: Wed Aug 03 10:17:21 2016 +0300 summary: Issue #26576: Clarify that the @deco syntax is not always an equivalent of f = deco(f) Patch by Chris Angelico. files: Doc/reference/compound_stmts.rst | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -503,11 +503,13 @@ @f2 def func(): pass -is equivalent to :: +is roughly equivalent to :: def func(): pass func = f1(arg)(f2(func)) +except that the original function is not temporarily bound to the name ``func``. + .. index:: triple: default; parameter; value single: argument; function definition @@ -638,14 +640,13 @@ @f2 class Foo: pass -is equivalent to :: +is roughly equivalent to :: class Foo: pass Foo = f1(arg)(f2(Foo)) The evaluation rules for the decorator expressions are the same as for function -decorators. The result must be a class object, which is then bound to the class -name. +decorators. The result is then bound to the class name. **Programmer's note:** Variables defined in the class definition are class attributes; they are shared by instances. Instance attributes can be set in a -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 05:19:29 2016 From: python-checkins at python.org (stefan.krah) Date: Wed, 03 Aug 2016 09:19:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41Lg==?= Message-ID: <20160803091920.26447.36898.A83A3F3C@psf.io> https://hg.python.org/cpython/rev/167974512561 changeset: 102513:167974512561 parent: 102511:08359651815e parent: 102512:441bbf4cc914 user: Stefan Krah date: Wed Aug 03 11:18:54 2016 +0200 summary: Merge 3.5. files: Lib/distutils/unixccompiler.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Lib/distutils/unixccompiler.py b/Lib/distutils/unixccompiler.py --- a/Lib/distutils/unixccompiler.py +++ b/Lib/distutils/unixccompiler.py @@ -227,6 +227,8 @@ if sys.platform[:6] == "darwin": # MacOSX's linker doesn't understand the -R flag at all return "-L" + dir + elif sys.platform[:7] == "freebsd": + return "-Wl,-rpath=" + dir elif sys.platform[:5] == "hp-ux": if self._is_gcc(compiler): return ["-Wl,+s", "-L" + dir] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 05:19:35 2016 From: python-checkins at python.org (stefan.krah) Date: Wed, 03 Aug 2016 09:19:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIwNzY3?= =?utf-8?q?=3A_Fix_-R_option_for_FreeBSD/clang=2E?= Message-ID: <20160803091920.458.9797.B7D1EBEF@psf.io> https://hg.python.org/cpython/rev/441bbf4cc914 changeset: 102512:441bbf4cc914 branch: 3.5 parent: 102510:e0f9f8be7963 user: Stefan Krah date: Wed Aug 03 11:18:26 2016 +0200 summary: Issue #20767: Fix -R option for FreeBSD/clang. files: Lib/distutils/unixccompiler.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Lib/distutils/unixccompiler.py b/Lib/distutils/unixccompiler.py --- a/Lib/distutils/unixccompiler.py +++ b/Lib/distutils/unixccompiler.py @@ -227,6 +227,8 @@ if sys.platform[:6] == "darwin": # MacOSX's linker doesn't understand the -R flag at all return "-L" + dir + elif sys.platform[:7] == "freebsd": + return "-Wl,-rpath=" + dir elif sys.platform[:5] == "hp-ux": if self._is_gcc(compiler): return ["-Wl,+s", "-L" + dir] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 05:23:53 2016 From: python-checkins at python.org (stefan.krah) Date: Wed, 03 Aug 2016 09:23:53 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIwNzY3?= =?utf-8?q?=3A_Fix_-R_option_for_FreeBSD/clang=2E?= Message-ID: <20160803092348.24944.57632.9D60272E@psf.io> https://hg.python.org/cpython/rev/65eb8d0ede75 changeset: 102514:65eb8d0ede75 branch: 2.7 parent: 102496:ba915d561667 user: Stefan Krah date: Wed Aug 03 11:23:31 2016 +0200 summary: Issue #20767: Fix -R option for FreeBSD/clang. files: Lib/distutils/unixccompiler.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Lib/distutils/unixccompiler.py b/Lib/distutils/unixccompiler.py --- a/Lib/distutils/unixccompiler.py +++ b/Lib/distutils/unixccompiler.py @@ -230,6 +230,8 @@ if sys.platform[:6] == "darwin": # MacOSX's linker doesn't understand the -R flag at all return "-L" + dir + elif sys.platform[:7] == "freebsd": + return "-Wl,-rpath=" + dir elif sys.platform[:5] == "hp-ux": if self._is_gcc(compiler): return ["-Wl,+s", "-L" + dir] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 05:58:20 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 03 Aug 2016 09:58:20 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIzNzEw?= =?utf-8?q?=3A_Update_PyObject=5FHEAD_documentation?= Message-ID: <20160803095817.1710.27391.3003CD1B@psf.io> https://hg.python.org/cpython/rev/92b468020e07 changeset: 102515:92b468020e07 branch: 3.5 parent: 102512:441bbf4cc914 user: Berker Peksag date: Wed Aug 03 12:58:49 2016 +0300 summary: Issue #23710: Update PyObject_HEAD documentation Since PEP 3123, PyObject_HEAD only has one field named ob_base. Users now need to use the Py_TYPE macro instead of self->ob_type. Initial patch by Ammar Askar. files: Doc/extending/newtypes.rst | 14 +++++++++----- 1 files changed, 9 insertions(+), 5 deletions(-) diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst --- a/Doc/extending/newtypes.rst +++ b/Doc/extending/newtypes.rst @@ -52,11 +52,15 @@ } noddy_NoddyObject; This is what a Noddy object will contain---in this case, nothing more than what -every Python object contains---a refcount and a pointer to a type object. -These are the fields the ``PyObject_HEAD`` macro brings in. The reason for the -macro is to standardize the layout and to enable special debugging fields in -debug builds. Note that there is no semicolon after the ``PyObject_HEAD`` -macro; one is included in the macro definition. Be wary of adding one by +every Python object contains---a field called ``ob_base`` of type +:c:type:`PyObject`. :c:type:`PyObject` in turn, contains an ``ob_refcnt`` +field and a pointer to a type object. These can be accessed using the macros +:c:macro:`Py_REFCNT` and :c:macro:`Py_TYPE` respectively. These are the fields +the :c:macro:`PyObject_HEAD` macro brings in. The reason for the macro is to +standardize the layout and to enable special debugging fields in debug builds. + +Note that there is no semicolon after the :c:macro:`PyObject_HEAD` macro; +one is included in the macro definition. Be wary of adding one by accident; it's easy to do from habit, and your compiler might not complain, but someone else's probably will! (On Windows, MSVC is known to call this an error and refuse to compile the code.) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 05:58:20 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 03 Aug 2016 09:58:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2323710=3A_Merge_from_3=2E5?= Message-ID: <20160803095817.32663.56045.9057749C@psf.io> https://hg.python.org/cpython/rev/26dd6e7f1733 changeset: 102516:26dd6e7f1733 parent: 102513:167974512561 parent: 102515:92b468020e07 user: Berker Peksag date: Wed Aug 03 12:59:16 2016 +0300 summary: Issue #23710: Merge from 3.5 files: Doc/extending/newtypes.rst | 14 +++++++++----- 1 files changed, 9 insertions(+), 5 deletions(-) diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst --- a/Doc/extending/newtypes.rst +++ b/Doc/extending/newtypes.rst @@ -52,11 +52,15 @@ } noddy_NoddyObject; This is what a Noddy object will contain---in this case, nothing more than what -every Python object contains---a refcount and a pointer to a type object. -These are the fields the ``PyObject_HEAD`` macro brings in. The reason for the -macro is to standardize the layout and to enable special debugging fields in -debug builds. Note that there is no semicolon after the ``PyObject_HEAD`` -macro; one is included in the macro definition. Be wary of adding one by +every Python object contains---a field called ``ob_base`` of type +:c:type:`PyObject`. :c:type:`PyObject` in turn, contains an ``ob_refcnt`` +field and a pointer to a type object. These can be accessed using the macros +:c:macro:`Py_REFCNT` and :c:macro:`Py_TYPE` respectively. These are the fields +the :c:macro:`PyObject_HEAD` macro brings in. The reason for the macro is to +standardize the layout and to enable special debugging fields in debug builds. + +Note that there is no semicolon after the :c:macro:`PyObject_HEAD` macro; +one is included in the macro definition. Be wary of adding one by accident; it's easy to do from habit, and your compiler might not complain, but someone else's probably will! (On Windows, MSVC is known to call this an error and refuse to compile the code.) -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Wed Aug 3 09:18:14 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 3 Aug 2016 14:18:14 +0100 Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-08-03 Message-ID: No new revisions. Here are the previous results: Results for project Python 2.7, build date 2016-08-03 02:47:32 +0000 commit: ba915d561667 previous commit: c8c1ea94379a revision date: 2016-07-30 12:49:53 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.13% -1.44% 3.69% 6.87% :-) pybench 0.18% -0.17% 5.70% 4.95% :-( regex_v8 0.65% -0.15% -2.56% 10.28% :-) nbody 0.27% -0.13% 8.51% 0.88% :-| json_dump_v2 0.33% 0.94% 1.95% 11.49% :-( normal_startup 1.91% -0.15% -6.14% 1.94% :-) ssbench 0.15% 0.84% 2.76% 1.17% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-08-03/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Wed Aug 3 09:19:43 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 3 Aug 2016 14:19:43 +0100 Subject: [Python-checkins] UGLY Benchmark Results for Python Default 2016-08-03 Message-ID: Results for project Python default, build date 2016-08-03 02:01:48 +0000 commit: adce94a718e3 previous commit: 6c2e2de5ab8e revision date: 2016-08-02 21:49:30 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.18% -1.46% 8.83% 17.77% :-) pybench 0.14% 3.73% 5.33% 6.73% :-( regex_v8 2.77% -0.69% -3.54% 4.44% :-) nbody 0.27% 2.40% 1.80% 7.61% :-| json_dump_v2 0.24% 0.15% -1.02% 11.23% :-| normal_startup 0.84% 0.32% -1.86% 5.70% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-default-2016-08-03/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Wed Aug 3 18:54:57 2016 From: python-checkins at python.org (donald.stufft) Date: Wed, 03 Aug 2016 22:54:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_Merge_3=2E3?= Message-ID: <20160803225457.24790.73673.7EF0E273@psf.io> https://hg.python.org/cpython/rev/f1aefe5947c5 changeset: 102518:f1aefe5947c5 branch: 3.4 parent: 102502:3c19023c9fec parent: 102517:54bfff5503d3 user: Donald Stufft date: Wed Aug 03 18:48:17 2016 -0400 summary: Merge 3.3 files: Lib/distutils/config.py | 2 +- Lib/distutils/tests/test_config.py | 4 ++-- Lib/distutils/tests/test_upload.py | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Lib/distutils/config.py b/Lib/distutils/config.py --- a/Lib/distutils/config.py +++ b/Lib/distutils/config.py @@ -21,7 +21,7 @@ class PyPIRCCommand(Command): """Base command that knows how to handle the .pypirc file """ - DEFAULT_REPOSITORY = 'https://upload.pypi.io/legacy/' + DEFAULT_REPOSITORY = 'https://upload.pypi.org/legacy/' DEFAULT_REALM = 'pypi' repository = None realm = None diff --git a/Lib/distutils/tests/test_config.py b/Lib/distutils/tests/test_config.py --- a/Lib/distutils/tests/test_config.py +++ b/Lib/distutils/tests/test_config.py @@ -87,7 +87,7 @@ config = list(sorted(config.items())) waited = [('password', 'secret'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/'), + ('repository', 'https://upload.pypi.org/legacy/'), ('server', 'server1'), ('username', 'me')] self.assertEqual(config, waited) @@ -96,7 +96,7 @@ config = cmd._read_pypirc() config = list(sorted(config.items())) waited = [('password', 'secret'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/'), + ('repository', 'https://upload.pypi.org/legacy/'), ('server', 'server-login'), ('username', 'tarek')] self.assertEqual(config, waited) diff --git a/Lib/distutils/tests/test_upload.py b/Lib/distutils/tests/test_upload.py --- a/Lib/distutils/tests/test_upload.py +++ b/Lib/distutils/tests/test_upload.py @@ -90,7 +90,7 @@ cmd.finalize_options() for attr, waited in (('username', 'me'), ('password', 'secret'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/')): + ('repository', 'https://upload.pypi.org/legacy/')): self.assertEqual(getattr(cmd, attr), waited) def test_saved_password(self): @@ -131,7 +131,7 @@ content_type = headers['Content-type'] self.assertTrue(content_type.startswith('multipart/form-data')) self.assertEqual(self.last_open.req.get_method(), 'POST') - expected_url = 'https://upload.pypi.io/legacy/' + expected_url = 'https://upload.pypi.org/legacy/' self.assertEqual(self.last_open.req.get_full_url(), expected_url) self.assertTrue(b'xxx' in self.last_open.req.data) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 18:54:57 2016 From: python-checkins at python.org (donald.stufft) Date: Wed, 03 Aug 2016 22:54:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Switch_upload?= =?utf-8?q?=2Epypi=2Eio_to_upload=2Epypi=2Eorg?= Message-ID: <20160803225457.23731.47769.CC66F8F3@psf.io> https://hg.python.org/cpython/rev/54bfff5503d3 changeset: 102517:54bfff5503d3 branch: 3.3 parent: 102501:95b09ccc8a3e user: Donald Stufft date: Wed Aug 03 18:43:38 2016 -0400 summary: Switch upload.pypi.io to upload.pypi.org files: Lib/distutils/config.py | 2 +- Lib/distutils/tests/test_config.py | 4 ++-- Lib/distutils/tests/test_upload.py | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Lib/distutils/config.py b/Lib/distutils/config.py --- a/Lib/distutils/config.py +++ b/Lib/distutils/config.py @@ -21,7 +21,7 @@ class PyPIRCCommand(Command): """Base command that knows how to handle the .pypirc file """ - DEFAULT_REPOSITORY = 'https://upload.pypi.io/legacy/' + DEFAULT_REPOSITORY = 'https://upload.pypi.org/legacy/' DEFAULT_REALM = 'pypi' repository = None realm = None diff --git a/Lib/distutils/tests/test_config.py b/Lib/distutils/tests/test_config.py --- a/Lib/distutils/tests/test_config.py +++ b/Lib/distutils/tests/test_config.py @@ -87,7 +87,7 @@ config = list(sorted(config.items())) waited = [('password', 'secret'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/'), + ('repository', 'https://upload.pypi.org/legacy/'), ('server', 'server1'), ('username', 'me')] self.assertEqual(config, waited) @@ -96,7 +96,7 @@ config = cmd._read_pypirc() config = list(sorted(config.items())) waited = [('password', 'secret'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/'), + ('repository', 'https://upload.pypi.org/legacy/'), ('server', 'server-login'), ('username', 'tarek')] self.assertEqual(config, waited) diff --git a/Lib/distutils/tests/test_upload.py b/Lib/distutils/tests/test_upload.py --- a/Lib/distutils/tests/test_upload.py +++ b/Lib/distutils/tests/test_upload.py @@ -127,7 +127,7 @@ self.assertTrue(headers['Content-type'].startswith('multipart/form-data')) self.assertEqual(self.last_open.req.get_method(), 'POST') self.assertEqual(self.last_open.req.get_full_url(), - 'https://upload.pypi.io/legacy/') + 'https://upload.pypi.org/legacy/') self.assertIn(b'xxx', self.last_open.req.data) # The PyPI response body was echoed -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 18:54:57 2016 From: python-checkins at python.org (donald.stufft) Date: Wed, 03 Aug 2016 22:54:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4?= Message-ID: <20160803225457.1517.34587.7BF79799@psf.io> https://hg.python.org/cpython/rev/2b1a84a8758b changeset: 102519:2b1a84a8758b branch: 3.5 parent: 102515:92b468020e07 parent: 102518:f1aefe5947c5 user: Donald Stufft date: Wed Aug 03 18:49:19 2016 -0400 summary: Merge 3.4 files: Lib/distutils/config.py | 2 +- Lib/distutils/tests/test_config.py | 6 +++--- Lib/distutils/tests/test_upload.py | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Lib/distutils/config.py b/Lib/distutils/config.py --- a/Lib/distutils/config.py +++ b/Lib/distutils/config.py @@ -21,7 +21,7 @@ class PyPIRCCommand(Command): """Base command that knows how to handle the .pypirc file """ - DEFAULT_REPOSITORY = 'https://upload.pypi.io/legacy/' + DEFAULT_REPOSITORY = 'https://upload.pypi.org/legacy/' DEFAULT_REALM = 'pypi' repository = None realm = None diff --git a/Lib/distutils/tests/test_config.py b/Lib/distutils/tests/test_config.py --- a/Lib/distutils/tests/test_config.py +++ b/Lib/distutils/tests/test_config.py @@ -95,7 +95,7 @@ config = list(sorted(config.items())) waited = [('password', 'secret'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/'), + ('repository', 'https://upload.pypi.org/legacy/'), ('server', 'server1'), ('username', 'me')] self.assertEqual(config, waited) @@ -104,7 +104,7 @@ config = cmd._read_pypirc() config = list(sorted(config.items())) waited = [('password', 'secret'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/'), + ('repository', 'https://upload.pypi.org/legacy/'), ('server', 'server-login'), ('username', 'tarek')] self.assertEqual(config, waited) @@ -130,7 +130,7 @@ config = list(sorted(config.items())) waited = [('password', 'yh^%#rest-of-my-password'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/'), + ('repository', 'https://upload.pypi.org/legacy/'), ('server', 'server3'), ('username', 'cbiggles')] self.assertEqual(config, waited) diff --git a/Lib/distutils/tests/test_upload.py b/Lib/distutils/tests/test_upload.py --- a/Lib/distutils/tests/test_upload.py +++ b/Lib/distutils/tests/test_upload.py @@ -93,7 +93,7 @@ cmd.finalize_options() for attr, waited in (('username', 'me'), ('password', 'secret'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/')): + ('repository', 'https://upload.pypi.org/legacy/')): self.assertEqual(getattr(cmd, attr), waited) def test_saved_password(self): @@ -134,7 +134,7 @@ content_type = headers['Content-type'] self.assertTrue(content_type.startswith('multipart/form-data')) self.assertEqual(self.last_open.req.get_method(), 'POST') - expected_url = 'https://upload.pypi.io/legacy/' + expected_url = 'https://upload.pypi.org/legacy/' self.assertEqual(self.last_open.req.get_full_url(), expected_url) self.assertTrue(b'xxx' in self.last_open.req.data) self.assertIn(b'protocol_version', self.last_open.req.data) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 18:54:57 2016 From: python-checkins at python.org (donald.stufft) Date: Wed, 03 Aug 2016 22:54:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20160803225457.26341.90380.C12A9D89@psf.io> https://hg.python.org/cpython/rev/a9bc4c2b3381 changeset: 102520:a9bc4c2b3381 parent: 102516:26dd6e7f1733 parent: 102519:2b1a84a8758b user: Donald Stufft date: Wed Aug 03 18:49:46 2016 -0400 summary: Merge 3.5 files: Lib/distutils/config.py | 2 +- Lib/distutils/tests/test_config.py | 6 +++--- Lib/distutils/tests/test_upload.py | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Lib/distutils/config.py b/Lib/distutils/config.py --- a/Lib/distutils/config.py +++ b/Lib/distutils/config.py @@ -21,7 +21,7 @@ class PyPIRCCommand(Command): """Base command that knows how to handle the .pypirc file """ - DEFAULT_REPOSITORY = 'https://upload.pypi.io/legacy/' + DEFAULT_REPOSITORY = 'https://upload.pypi.org/legacy/' DEFAULT_REALM = 'pypi' repository = None realm = None diff --git a/Lib/distutils/tests/test_config.py b/Lib/distutils/tests/test_config.py --- a/Lib/distutils/tests/test_config.py +++ b/Lib/distutils/tests/test_config.py @@ -93,7 +93,7 @@ config = list(sorted(config.items())) waited = [('password', 'secret'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/'), + ('repository', 'https://upload.pypi.org/legacy/'), ('server', 'server1'), ('username', 'me')] self.assertEqual(config, waited) @@ -102,7 +102,7 @@ config = cmd._read_pypirc() config = list(sorted(config.items())) waited = [('password', 'secret'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/'), + ('repository', 'https://upload.pypi.org/legacy/'), ('server', 'server-login'), ('username', 'tarek')] self.assertEqual(config, waited) @@ -128,7 +128,7 @@ config = list(sorted(config.items())) waited = [('password', 'yh^%#rest-of-my-password'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/'), + ('repository', 'https://upload.pypi.org/legacy/'), ('server', 'server3'), ('username', 'cbiggles')] self.assertEqual(config, waited) diff --git a/Lib/distutils/tests/test_upload.py b/Lib/distutils/tests/test_upload.py --- a/Lib/distutils/tests/test_upload.py +++ b/Lib/distutils/tests/test_upload.py @@ -93,7 +93,7 @@ cmd.finalize_options() for attr, waited in (('username', 'me'), ('password', 'secret'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/')): + ('repository', 'https://upload.pypi.org/legacy/')): self.assertEqual(getattr(cmd, attr), waited) def test_saved_password(self): @@ -134,7 +134,7 @@ content_type = headers['Content-type'] self.assertTrue(content_type.startswith('multipart/form-data')) self.assertEqual(self.last_open.req.get_method(), 'POST') - expected_url = 'https://upload.pypi.io/legacy/' + expected_url = 'https://upload.pypi.org/legacy/' self.assertEqual(self.last_open.req.get_full_url(), expected_url) self.assertTrue(b'xxx' in self.last_open.req.data) self.assertIn(b'protocol_version', self.last_open.req.data) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 19:00:52 2016 From: python-checkins at python.org (donald.stufft) Date: Wed, 03 Aug 2016 23:00:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Switch_upload?= =?utf-8?q?=2Epypi=2Eio_to_upload=2Epypi=2Eorg?= Message-ID: <20160803230050.401.59107.3A0EEA5D@psf.io> https://hg.python.org/cpython/rev/af2a7161fa7f changeset: 102521:af2a7161fa7f branch: 2.7 parent: 102514:65eb8d0ede75 user: Donald Stufft date: Wed Aug 03 18:58:12 2016 -0400 summary: Switch upload.pypi.io to upload.pypi.org files: Lib/distutils/config.py | 2 +- Lib/distutils/tests/test_config.py | 4 ++-- Lib/distutils/tests/test_upload.py | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Lib/distutils/config.py b/Lib/distutils/config.py --- a/Lib/distutils/config.py +++ b/Lib/distutils/config.py @@ -21,7 +21,7 @@ class PyPIRCCommand(Command): """Base command that knows how to handle the .pypirc file """ - DEFAULT_REPOSITORY = 'https://upload.pypi.io/legacy/' + DEFAULT_REPOSITORY = 'https://upload.pypi.org/legacy/' DEFAULT_REALM = 'pypi' repository = None realm = None diff --git a/Lib/distutils/tests/test_config.py b/Lib/distutils/tests/test_config.py --- a/Lib/distutils/tests/test_config.py +++ b/Lib/distutils/tests/test_config.py @@ -89,7 +89,7 @@ config = config.items() config.sort() waited = [('password', 'secret'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/'), + ('repository', 'https://upload.pypi.org/legacy/'), ('server', 'server1'), ('username', 'me')] self.assertEqual(config, waited) @@ -99,7 +99,7 @@ config = config.items() config.sort() waited = [('password', 'secret'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/'), + ('repository', 'https://upload.pypi.org/legacy/'), ('server', 'server-login'), ('username', 'tarek')] self.assertEqual(config, waited) diff --git a/Lib/distutils/tests/test_upload.py b/Lib/distutils/tests/test_upload.py --- a/Lib/distutils/tests/test_upload.py +++ b/Lib/distutils/tests/test_upload.py @@ -82,7 +82,7 @@ cmd.finalize_options() for attr, waited in (('username', 'me'), ('password', 'secret'), ('realm', 'pypi'), - ('repository', 'https://upload.pypi.io/legacy/')): + ('repository', 'https://upload.pypi.org/legacy/')): self.assertEqual(getattr(cmd, attr), waited) def test_saved_password(self): @@ -123,7 +123,7 @@ self.assertTrue(headers['Content-type'].startswith('multipart/form-data')) self.assertEqual(self.last_open.req.get_method(), 'POST') self.assertEqual(self.last_open.req.get_full_url(), - 'https://upload.pypi.io/legacy/') + 'https://upload.pypi.org/legacy/') self.assertIn('xxx', self.last_open.req.data) auth = self.last_open.req.headers['Authorization'] self.assertNotIn('\n', auth) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 21:56:04 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 04 Aug 2016 01:56:04 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0Mjc4?= =?utf-8?q?=3A_Explain_how_argument_parsing_output_buffers_are_managed?= Message-ID: <20160804015604.20234.18013.322CBB7B@psf.io> https://hg.python.org/cpython/rev/ad7da726bea6 changeset: 102522:ad7da726bea6 branch: 3.5 parent: 102519:2b1a84a8758b user: Martin Panter date: Wed Aug 03 14:35:05 2016 +0000 summary: Issue #24278: Explain how argument parsing output buffers are managed files: Doc/c-api/arg.rst | 33 +++++++++++++++++++-------------- 1 files changed, 19 insertions(+), 14 deletions(-) diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -32,8 +32,12 @@ These formats allow accessing an object as a contiguous chunk of memory. You don't have to provide raw storage for the returned unicode or bytes -area. Also, you won't have to release any memory yourself, except with the -``es``, ``es#``, ``et`` and ``et#`` formats. +area. + +In general, when a format sets a pointer to a buffer, the buffer is +managed by the corresponding Python object, and the buffer shares +the lifetime of this object. You won't have to release any memory yourself. +The only exceptions are ``es``, ``es#``, ``et`` and ``et#``. However, when a :c:type:`Py_buffer` structure gets filled, the underlying buffer is locked so that the caller can subsequently use the buffer even @@ -44,6 +48,11 @@ Unless otherwise stated, buffers are not NUL-terminated. +Some formats require a read-only :term:`bytes-like object`, and set a +pointer instead of a buffer structure. They work by checking that +the object's :c:member:`PyBufferProcs.bf_releasebuffer` field is *NULL*, +which disallows mutable objects such as :class:`bytearray`. + .. note:: For all ``#`` variants of formats (``s#``, ``y#``, etc.), the type of @@ -59,7 +68,7 @@ Convert a Unicode object to a C pointer to a character string. A pointer to an existing string is stored in the character pointer variable whose address you pass. The C string is NUL-terminated. - The Python string must not contain embedded null characters; if it does, + The Python string must not contain embedded null code points; if it does, a :exc:`ValueError` exception is raised. Unicode objects are converted to C strings using ``'utf-8'`` encoding. If this conversion fails, a :exc:`UnicodeError` is raised. @@ -72,7 +81,7 @@ as *converter*. .. versionchanged:: 3.5 - Previously, :exc:`TypeError` was raised when embedded null characters + Previously, :exc:`TypeError` was raised when embedded null code points were encountered in the Python string. ``s*`` (:class:`str` or :term:`bytes-like object`) [Py_buffer] @@ -82,8 +91,8 @@ Unicode objects are converted to C strings using ``'utf-8'`` encoding. ``s#`` (:class:`str`, read-only :term:`bytes-like object`) [const char \*, int or :c:type:`Py_ssize_t`] - Like ``s*``, except that it doesn't accept mutable bytes-like objects - such as :class:`bytearray`. The result is stored into two C variables, + Like ``s*``, except that it doesn't accept mutable objects. + The result is stored into two C variables, the first one a pointer to a C string, the second one its length. The string may contain embedded null bytes. Unicode objects are converted to C strings using ``'utf-8'`` encoding. @@ -135,21 +144,17 @@ pointer variable, which will be filled with the pointer to an existing Unicode buffer. Please note that the width of a :c:type:`Py_UNICODE` character depends on compilation options (it is either 16 or 32 bits). - The Python string must not contain embedded null characters; if it does, + The Python string must not contain embedded null code points; if it does, a :exc:`ValueError` exception is raised. - .. note:: - Since ``u`` doesn't give you back the length of the string, and it - may contain embedded NUL characters, it is recommended to use ``u#`` - or ``U`` instead. - .. versionchanged:: 3.5 - Previously, :exc:`TypeError` was raised when embedded null characters + Previously, :exc:`TypeError` was raised when embedded null code points were encountered in the Python string. ``u#`` (:class:`str`) [Py_UNICODE \*, int] This variant on ``u`` stores into two C variables, the first one a pointer to a - Unicode data buffer, the second one its length. + Unicode data buffer, the second one its length. This variant allows + null code points. ``Z`` (:class:`str` or ``None``) [Py_UNICODE \*] Like ``u``, but the Python object may also be ``None``, in which case the -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 21:56:05 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 04 Aug 2016 01:56:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2324278=3A_Merge_argument_parsing_docs_from_3=2E5?= Message-ID: <20160804015604.21340.50218.AFD3F089@psf.io> https://hg.python.org/cpython/rev/49c318c6294e changeset: 102524:49c318c6294e parent: 102520:a9bc4c2b3381 parent: 102522:ad7da726bea6 user: Martin Panter date: Thu Aug 04 01:51:32 2016 +0000 summary: Issue #24278: Merge argument parsing docs from 3.5 files: Doc/c-api/arg.rst | 33 +++++++++++++++++++-------------- 1 files changed, 19 insertions(+), 14 deletions(-) diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -32,8 +32,12 @@ These formats allow accessing an object as a contiguous chunk of memory. You don't have to provide raw storage for the returned unicode or bytes -area. Also, you won't have to release any memory yourself, except with the -``es``, ``es#``, ``et`` and ``et#`` formats. +area. + +In general, when a format sets a pointer to a buffer, the buffer is +managed by the corresponding Python object, and the buffer shares +the lifetime of this object. You won't have to release any memory yourself. +The only exceptions are ``es``, ``es#``, ``et`` and ``et#``. However, when a :c:type:`Py_buffer` structure gets filled, the underlying buffer is locked so that the caller can subsequently use the buffer even @@ -44,6 +48,11 @@ Unless otherwise stated, buffers are not NUL-terminated. +Some formats require a read-only :term:`bytes-like object`, and set a +pointer instead of a buffer structure. They work by checking that +the object's :c:member:`PyBufferProcs.bf_releasebuffer` field is *NULL*, +which disallows mutable objects such as :class:`bytearray`. + .. note:: For all ``#`` variants of formats (``s#``, ``y#``, etc.), the type of @@ -59,7 +68,7 @@ Convert a Unicode object to a C pointer to a character string. A pointer to an existing string is stored in the character pointer variable whose address you pass. The C string is NUL-terminated. - The Python string must not contain embedded null characters; if it does, + The Python string must not contain embedded null code points; if it does, a :exc:`ValueError` exception is raised. Unicode objects are converted to C strings using ``'utf-8'`` encoding. If this conversion fails, a :exc:`UnicodeError` is raised. @@ -72,7 +81,7 @@ as *converter*. .. versionchanged:: 3.5 - Previously, :exc:`TypeError` was raised when embedded null characters + Previously, :exc:`TypeError` was raised when embedded null code points were encountered in the Python string. ``s*`` (:class:`str` or :term:`bytes-like object`) [Py_buffer] @@ -82,8 +91,8 @@ Unicode objects are converted to C strings using ``'utf-8'`` encoding. ``s#`` (:class:`str`, read-only :term:`bytes-like object`) [const char \*, int or :c:type:`Py_ssize_t`] - Like ``s*``, except that it doesn't accept mutable bytes-like objects - such as :class:`bytearray`. The result is stored into two C variables, + Like ``s*``, except that it doesn't accept mutable objects. + The result is stored into two C variables, the first one a pointer to a C string, the second one its length. The string may contain embedded null bytes. Unicode objects are converted to C strings using ``'utf-8'`` encoding. @@ -135,21 +144,17 @@ pointer variable, which will be filled with the pointer to an existing Unicode buffer. Please note that the width of a :c:type:`Py_UNICODE` character depends on compilation options (it is either 16 or 32 bits). - The Python string must not contain embedded null characters; if it does, + The Python string must not contain embedded null code points; if it does, a :exc:`ValueError` exception is raised. - .. note:: - Since ``u`` doesn't give you back the length of the string, and it - may contain embedded NUL characters, it is recommended to use ``u#`` - or ``U`` instead. - .. versionchanged:: 3.5 - Previously, :exc:`TypeError` was raised when embedded null characters + Previously, :exc:`TypeError` was raised when embedded null code points were encountered in the Python string. ``u#`` (:class:`str`) [Py_UNICODE \*, int] This variant on ``u`` stores into two C variables, the first one a pointer to a - Unicode data buffer, the second one its length. + Unicode data buffer, the second one its length. This variant allows + null code points. ``Z`` (:class:`str` or ``None``) [Py_UNICODE \*] Like ``u``, but the Python object may also be ``None``, in which case the -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 21:56:05 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 04 Aug 2016 01:56:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327641=3A_Merge_cross-compiling_improvement_from?= =?utf-8?q?_3=2E5?= Message-ID: <20160804015605.10490.32159.47643439@psf.io> https://hg.python.org/cpython/rev/fc034d3607a8 changeset: 102525:fc034d3607a8 parent: 102524:49c318c6294e parent: 102523:bc677cb34889 user: Martin Panter date: Thu Aug 04 01:53:40 2016 +0000 summary: Issue #27641: Merge cross-compiling improvement from 3.5 files: Makefile.pre.in | 36 ++++++++++-------------------------- Misc/NEWS | 5 +++-- configure | 9 ++++----- configure.ac | 8 ++++---- 4 files changed, 21 insertions(+), 37 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -222,7 +222,6 @@ PYTHON= python$(EXE) BUILDPYTHON= python$(BUILDEXE) -cross_compiling=@cross_compiling@ PYTHON_FOR_GEN=@PYTHON_FOR_GEN@ PYTHON_FOR_BUILD=@PYTHON_FOR_BUILD@ _PYTHON_HOST_PLATFORM=@_PYTHON_HOST_PLATFORM@ @@ -720,17 +719,13 @@ Programs/_freeze_importlib: Programs/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN) $(LINKCC) $(PY_LDFLAGS) -o $@ Programs/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST) -Python/importlib_external.h: $(srcdir)/Lib/importlib/_bootstrap_external.py Programs/_freeze_importlib - if test "$(cross_compiling)" != "yes"; then \ - ./Programs/_freeze_importlib \ - $(srcdir)/Lib/importlib/_bootstrap_external.py Python/importlib_external.h; \ - fi +Python/importlib_external.h: @GENERATED_COMMENT@ $(srcdir)/Lib/importlib/_bootstrap_external.py Programs/_freeze_importlib + ./Programs/_freeze_importlib \ + $(srcdir)/Lib/importlib/_bootstrap_external.py Python/importlib_external.h -Python/importlib.h: $(srcdir)/Lib/importlib/_bootstrap.py Programs/_freeze_importlib - if test "$(cross_compiling)" != "yes"; then \ - ./Programs/_freeze_importlib \ - $(srcdir)/Lib/importlib/_bootstrap.py Python/importlib.h; \ - fi +Python/importlib.h: @GENERATED_COMMENT@ $(srcdir)/Lib/importlib/_bootstrap.py Programs/_freeze_importlib + ./Programs/_freeze_importlib \ + $(srcdir)/Lib/importlib/_bootstrap.py Python/importlib.h ############################################################################ @@ -791,22 +786,11 @@ $(IO_OBJS): $(IO_H) -$(GRAMMAR_H): $(GRAMMAR_INPUT) @PGEN_DEPENDENCY@ +$(GRAMMAR_H): @GENERATED_COMMENT@ $(GRAMMAR_INPUT) $(PGEN) @$(MKDIR_P) Include - # Avoid copying the file onto itself for an in-tree build - if test "$(cross_compiling)" != "yes"; then \ - $(PGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C); \ - else \ - cp $(srcdir)/Include/graminit.h $(GRAMMAR_H).tmp; \ - mv $(GRAMMAR_H).tmp $(GRAMMAR_H); \ - fi -$(GRAMMAR_C): $(GRAMMAR_H) - if test "$(cross_compiling)" != "yes"; then \ - touch $(GRAMMAR_C); \ - else \ - cp $(srcdir)/Python/graminit.c $(GRAMMAR_C).tmp; \ - mv $(GRAMMAR_C).tmp $(GRAMMAR_C); \ - fi + $(PGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C) +$(GRAMMAR_C): @GENERATED_COMMENT@ $(GRAMMAR_H) + touch $(GRAMMAR_C) $(PGEN): $(PGENOBJS) $(CC) $(OPT) $(PY_LDFLAGS) $(PGENOBJS) $(LIBS) -o $(PGEN) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -101,8 +101,9 @@ - Issue #27453: CPP invocation in configure must use CPPFLAGS. Patch by Chi Hsuan Yen. -- Issue #27490: Do not build pgen when cross-compiling. Patch by Thomas - Perl. +- Issue #27641: The configure script now inserts comments into the makefile + to prevent the pgen and _freeze_importlib executables from being cross- + compiled. - Issue #26662: Set PYTHON_FOR_GEN in configure as the Python program to be used for file generation during the build. diff --git a/configure b/configure --- a/configure +++ b/configure @@ -740,7 +740,7 @@ CONFIG_ARGS SOVERSION VERSION -PGEN_DEPENDENCY +GENERATED_COMMENT PYTHON_FOR_BUILD PYTHON_FOR_GEN host_os @@ -751,7 +751,6 @@ build_vendor build_cpu build -cross_compiling HAS_HG HGBRANCH HGTAG @@ -2876,7 +2875,6 @@ ac_config_headers="$ac_config_headers pyconfig.h" - ac_aux_dir= for ac_dir in "$srcdir" "$srcdir/.." "$srcdir/../.."; do if test -f "$ac_dir/install-sh"; then @@ -3051,12 +3049,13 @@ $as_echo "$interp" >&6; } PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib:$(srcdir)/Lib/$(PLATDIR) '$interp fi - PGEN_DEPENDENCY='' + # Used to comment out stuff for rebuilding generated files + GENERATED_COMMENT='#' elif test "$cross_compiling" = maybe; then as_fn_error $? "Cross compiling required --host=HOST-TUPLE and --build=ARCH" "$LINENO" 5 else PYTHON_FOR_BUILD='./$(BUILDPYTHON) -E' - PGEN_DEPENDENCY='$(PGEN)' + GENERATED_COMMENT='' fi diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -49,7 +49,6 @@ AC_CONFIG_SRCDIR([Include/object.h]) AC_CONFIG_HEADER(pyconfig.h) -AC_SUBST(cross_compiling) AC_CANONICAL_HOST AC_SUBST(build) AC_SUBST(host) @@ -81,15 +80,16 @@ AC_MSG_RESULT($interp) PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib:$(srcdir)/Lib/$(PLATDIR) '$interp fi - PGEN_DEPENDENCY='' + # Used to comment out stuff for rebuilding generated files + GENERATED_COMMENT='#' elif test "$cross_compiling" = maybe; then AC_MSG_ERROR([Cross compiling required --host=HOST-TUPLE and --build=ARCH]) else PYTHON_FOR_BUILD='./$(BUILDPYTHON) -E' - PGEN_DEPENDENCY='$(PGEN)' + GENERATED_COMMENT='' fi AC_SUBST(PYTHON_FOR_BUILD) -AC_SUBST(PGEN_DEPENDENCY) +AC_SUBST(GENERATED_COMMENT) dnl Ensure that if prefix is specified, it does not end in a slash. If dnl it does, we get path names containing '//' which is both ugly and -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 21:56:04 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 04 Aug 2016 01:56:04 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NjQx?= =?utf-8?q?=3A_Comment_out_regeneration_rules_when_cross_compiling?= Message-ID: <20160804015604.79537.6191.311AAA76@psf.io> https://hg.python.org/cpython/rev/bc677cb34889 changeset: 102523:bc677cb34889 branch: 3.5 user: Martin Panter date: Fri Jul 29 05:52:32 2016 +0000 summary: Issue #27641: Comment out regeneration rules when cross compiling files: Makefile.pre.in | 36 ++++++++++-------------------------- Misc/NEWS | 5 +++-- configure | 9 ++++----- configure.ac | 8 ++++---- 4 files changed, 21 insertions(+), 37 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -221,7 +221,6 @@ PYTHON= python$(EXE) BUILDPYTHON= python$(BUILDEXE) -cross_compiling=@cross_compiling@ PYTHON_FOR_GEN=@PYTHON_FOR_GEN@ PYTHON_FOR_BUILD=@PYTHON_FOR_BUILD@ _PYTHON_HOST_PLATFORM=@_PYTHON_HOST_PLATFORM@ @@ -719,17 +718,13 @@ Programs/_freeze_importlib: Programs/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN) $(LINKCC) $(PY_LDFLAGS) -o $@ Programs/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST) -Python/importlib_external.h: $(srcdir)/Lib/importlib/_bootstrap_external.py Programs/_freeze_importlib - if test "$(cross_compiling)" != "yes"; then \ - ./Programs/_freeze_importlib \ - $(srcdir)/Lib/importlib/_bootstrap_external.py Python/importlib_external.h; \ - fi +Python/importlib_external.h: @GENERATED_COMMENT@ $(srcdir)/Lib/importlib/_bootstrap_external.py Programs/_freeze_importlib + ./Programs/_freeze_importlib \ + $(srcdir)/Lib/importlib/_bootstrap_external.py Python/importlib_external.h -Python/importlib.h: $(srcdir)/Lib/importlib/_bootstrap.py Programs/_freeze_importlib - if test "$(cross_compiling)" != "yes"; then \ - ./Programs/_freeze_importlib \ - $(srcdir)/Lib/importlib/_bootstrap.py Python/importlib.h; \ - fi +Python/importlib.h: @GENERATED_COMMENT@ $(srcdir)/Lib/importlib/_bootstrap.py Programs/_freeze_importlib + ./Programs/_freeze_importlib \ + $(srcdir)/Lib/importlib/_bootstrap.py Python/importlib.h ############################################################################ @@ -789,22 +784,11 @@ $(IO_OBJS): $(IO_H) -$(GRAMMAR_H): $(GRAMMAR_INPUT) @PGEN_DEPENDENCY@ +$(GRAMMAR_H): @GENERATED_COMMENT@ $(GRAMMAR_INPUT) $(PGEN) @$(MKDIR_P) Include - # Avoid copying the file onto itself for an in-tree build - if test "$(cross_compiling)" != "yes"; then \ - $(PGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C); \ - else \ - cp $(srcdir)/Include/graminit.h $(GRAMMAR_H).tmp; \ - mv $(GRAMMAR_H).tmp $(GRAMMAR_H); \ - fi -$(GRAMMAR_C): $(GRAMMAR_H) - if test "$(cross_compiling)" != "yes"; then \ - touch $(GRAMMAR_C); \ - else \ - cp $(srcdir)/Python/graminit.c $(GRAMMAR_C).tmp; \ - mv $(GRAMMAR_C).tmp $(GRAMMAR_C); \ - fi + $(PGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C) +$(GRAMMAR_C): @GENERATED_COMMENT@ $(GRAMMAR_H) + touch $(GRAMMAR_C) $(PGEN): $(PGENOBJS) $(CC) $(OPT) $(PY_LDFLAGS) $(PGENOBJS) $(LIBS) -o $(PGEN) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -154,8 +154,9 @@ - Issue #27453: CPP invocation in configure must use CPPFLAGS. Patch by Chi Hsuan Yen. -- Issue #27490: Do not build pgen when cross-compiling. Patch by Thomas - Perl. +- Issue #27641: The configure script now inserts comments into the makefile + to prevent the pgen and _freeze_importlib executables from being cross- + compiled. - Issue #26662: Set PYTHON_FOR_GEN in configure as the Python program to be used for file generation during the build. diff --git a/configure b/configure --- a/configure +++ b/configure @@ -739,7 +739,7 @@ CONFIG_ARGS SOVERSION VERSION -PGEN_DEPENDENCY +GENERATED_COMMENT PYTHON_FOR_BUILD PYTHON_FOR_GEN host_os @@ -750,7 +750,6 @@ build_vendor build_cpu build -cross_compiling HAS_HG HGBRANCH HGTAG @@ -2875,7 +2874,6 @@ ac_config_headers="$ac_config_headers pyconfig.h" - ac_aux_dir= for ac_dir in "$srcdir" "$srcdir/.." "$srcdir/../.."; do if test -f "$ac_dir/install-sh"; then @@ -3050,12 +3048,13 @@ $as_echo "$interp" >&6; } PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib:$(srcdir)/Lib/$(PLATDIR) '$interp fi - PGEN_DEPENDENCY='' + # Used to comment out stuff for rebuilding generated files + GENERATED_COMMENT='#' elif test "$cross_compiling" = maybe; then as_fn_error $? "Cross compiling required --host=HOST-TUPLE and --build=ARCH" "$LINENO" 5 else PYTHON_FOR_BUILD='./$(BUILDPYTHON) -E' - PGEN_DEPENDENCY='$(PGEN)' + GENERATED_COMMENT='' fi diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -49,7 +49,6 @@ AC_CONFIG_SRCDIR([Include/object.h]) AC_CONFIG_HEADER(pyconfig.h) -AC_SUBST(cross_compiling) AC_CANONICAL_HOST AC_SUBST(build) AC_SUBST(host) @@ -81,15 +80,16 @@ AC_MSG_RESULT($interp) PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib:$(srcdir)/Lib/$(PLATDIR) '$interp fi - PGEN_DEPENDENCY='' + # Used to comment out stuff for rebuilding generated files + GENERATED_COMMENT='#' elif test "$cross_compiling" = maybe; then AC_MSG_ERROR([Cross compiling required --host=HOST-TUPLE and --build=ARCH]) else PYTHON_FOR_BUILD='./$(BUILDPYTHON) -E' - PGEN_DEPENDENCY='$(PGEN)' + GENERATED_COMMENT='' fi AC_SUBST(PYTHON_FOR_BUILD) -AC_SUBST(PGEN_DEPENDENCY) +AC_SUBST(GENERATED_COMMENT) dnl Ensure that if prefix is specified, it does not end in a slash. If dnl it does, we get path names containing '//' which is both ugly and -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 23:03:44 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 04 Aug 2016 03:03:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317599=3A_Use_uniq?= =?utf-8?q?ue_=5FPy=5FREPARSE=5FDATA=5FBUFFER_etc_names_to_avoid_conflict?= Message-ID: <20160804030344.30739.66224.7ED7DA7D@psf.io> https://hg.python.org/cpython/rev/e4ddfa61199c changeset: 102527:e4ddfa61199c parent: 102525:fc034d3607a8 user: Martin Panter date: Thu Aug 04 02:38:59 2016 +0000 summary: Issue #17599: Use unique _Py_REPARSE_DATA_BUFFER etc names to avoid conflict The conflict occurs with Min GW, which already defines REPARSE_DATA_BUFFER. Also, Min GW uses a lowercase filename. files: Misc/NEWS | 3 +++ Modules/_winapi.c | 8 ++++---- Modules/posixmodule.c | 8 ++++---- Modules/winreparse.h | 17 +++++++++-------- 4 files changed, 20 insertions(+), 16 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,9 @@ - Issue #17596: Include to help with Min GW building. +- Issue #17599: On Windows, rename the privately defined REPARSE_DATA_BUFFER + structure to avoid conflicting with the definition from Min GW. + - Issue #27507: Add integer overflow check in bytearray.extend(). Patch by Xiang Zhang. diff --git a/Modules/_winapi.c b/Modules/_winapi.c --- a/Modules/_winapi.c +++ b/Modules/_winapi.c @@ -486,7 +486,7 @@ const USHORT prefix_len = 4; USHORT print_len = 0; USHORT rdb_size = 0; - PREPARSE_DATA_BUFFER rdb = NULL; + _Py_PREPARSE_DATA_BUFFER rdb = NULL; /* Junction point creation */ HANDLE junction = NULL; @@ -542,18 +542,18 @@ - the size of the print name in bytes - the size of the substitute name in bytes - the size of two NUL terminators in bytes */ - rdb_size = REPARSE_DATA_BUFFER_HEADER_SIZE + + rdb_size = _Py_REPARSE_DATA_BUFFER_HEADER_SIZE + sizeof(rdb->MountPointReparseBuffer) - sizeof(rdb->MountPointReparseBuffer.PathBuffer) + /* Two +1's for NUL terminators. */ (prefix_len + print_len + 1 + print_len + 1) * sizeof(WCHAR); - rdb = (PREPARSE_DATA_BUFFER)PyMem_RawMalloc(rdb_size); + rdb = (_Py_PREPARSE_DATA_BUFFER)PyMem_RawMalloc(rdb_size); if (rdb == NULL) goto cleanup; memset(rdb, 0, rdb_size); rdb->ReparseTag = IO_REPARSE_TAG_MOUNT_POINT; - rdb->ReparseDataLength = rdb_size - REPARSE_DATA_BUFFER_HEADER_SIZE; + rdb->ReparseDataLength = rdb_size - _Py_REPARSE_DATA_BUFFER_HEADER_SIZE; rdb->MountPointReparseBuffer.SubstituteNameOffset = 0; rdb->MountPointReparseBuffer.SubstituteNameLength = (prefix_len + print_len) * sizeof(WCHAR); diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -1106,8 +1106,8 @@ static int win32_get_reparse_tag(HANDLE reparse_point_handle, ULONG *reparse_tag) { - char target_buffer[MAXIMUM_REPARSE_DATA_BUFFER_SIZE]; - REPARSE_DATA_BUFFER *rdb = (REPARSE_DATA_BUFFER *)target_buffer; + char target_buffer[_Py_MAXIMUM_REPARSE_DATA_BUFFER_SIZE]; + _Py_REPARSE_DATA_BUFFER *rdb = (_Py_REPARSE_DATA_BUFFER *)target_buffer; DWORD n_bytes_returned; if (0 == DeviceIoControl( @@ -7149,8 +7149,8 @@ int dir_fd; HANDLE reparse_point_handle; - char target_buffer[MAXIMUM_REPARSE_DATA_BUFFER_SIZE]; - REPARSE_DATA_BUFFER *rdb = (REPARSE_DATA_BUFFER *)target_buffer; + char target_buffer[_Py_MAXIMUM_REPARSE_DATA_BUFFER_SIZE]; + _Py_REPARSE_DATA_BUFFER *rdb = (_Py_REPARSE_DATA_BUFFER *)target_buffer; const wchar_t *print_name; static char *keywords[] = {"path", "dir_fd", NULL}; diff --git a/Modules/winreparse.h b/Modules/winreparse.h --- a/Modules/winreparse.h +++ b/Modules/winreparse.h @@ -2,7 +2,7 @@ #define Py_WINREPARSE_H #ifdef MS_WINDOWS -#include +#include #ifdef __cplusplus extern "C" { @@ -10,9 +10,10 @@ /* The following structure was copied from http://msdn.microsoft.com/en-us/library/ff552012.aspx as the required - include doesn't seem to be present in the Windows SDK (at least as included - with Visual Studio Express). */ -typedef struct _REPARSE_DATA_BUFFER { + include km\ntifs.h isn't present in the Windows SDK (at least as included + with Visual Studio Express). Use unique names to avoid conflicting with + the structure as defined by Min GW. */ +typedef struct { ULONG ReparseTag; USHORT ReparseDataLength; USHORT Reserved; @@ -38,11 +39,11 @@ UCHAR DataBuffer[1]; } GenericReparseBuffer; }; -} REPARSE_DATA_BUFFER, *PREPARSE_DATA_BUFFER; +} _Py_REPARSE_DATA_BUFFER, *_Py_PREPARSE_DATA_BUFFER; -#define REPARSE_DATA_BUFFER_HEADER_SIZE FIELD_OFFSET(REPARSE_DATA_BUFFER,\ - GenericReparseBuffer) -#define MAXIMUM_REPARSE_DATA_BUFFER_SIZE ( 16 * 1024 ) +#define _Py_REPARSE_DATA_BUFFER_HEADER_SIZE \ + FIELD_OFFSET(_Py_REPARSE_DATA_BUFFER, GenericReparseBuffer) +#define _Py_MAXIMUM_REPARSE_DATA_BUFFER_SIZE ( 16 * 1024 ) #ifdef __cplusplus } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 3 23:03:43 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 04 Aug 2016 03:03:43 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3NjQx?= =?utf-8?q?=3A_Comment_out_regeneration_rules_when_cross_compiling?= Message-ID: <20160804030343.108297.62151.6848A4BC@psf.io> https://hg.python.org/cpython/rev/e3757f3e1848 changeset: 102526:e3757f3e1848 branch: 2.7 parent: 102521:af2a7161fa7f user: Martin Panter date: Fri Jul 29 05:52:32 2016 +0000 summary: Issue #27641: Comment out regeneration rules when cross compiling files: Makefile.pre.in | 20 ++++---------------- Misc/NEWS | 4 ++-- configure | 9 ++++----- configure.ac | 8 ++++---- 4 files changed, 14 insertions(+), 27 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -200,7 +200,6 @@ PYTHON= python$(EXE) BUILDPYTHON= python$(BUILDEXE) -cross_compiling=@cross_compiling@ PYTHON_FOR_BUILD=@PYTHON_FOR_BUILD@ _PYTHON_HOST_PLATFORM=@_PYTHON_HOST_PLATFORM@ HOST_GNU_TYPE= @host@ @@ -680,22 +679,11 @@ Modules/pwdmodule.o: $(srcdir)/Modules/pwdmodule.c $(srcdir)/Modules/posixmodule.h -$(GRAMMAR_H): $(GRAMMAR_INPUT) @PGEN_DEPENDENCY@ +$(GRAMMAR_H): @GENERATED_COMMENT@ $(GRAMMAR_INPUT) $(PGEN) @$(MKDIR_P) Include - # Avoid copying the file onto itself for an in-tree build - if test "$(cross_compiling)" != "yes"; then \ - $(PGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C); \ - else \ - cp $(srcdir)/Include/graminit.h $(GRAMMAR_H).tmp; \ - mv $(GRAMMAR_H).tmp $(GRAMMAR_H); \ - fi -$(GRAMMAR_C): $(GRAMMAR_H) - if test "$(cross_compiling)" != "yes"; then \ - touch $(GRAMMAR_C); \ - else \ - cp $(srcdir)/Python/graminit.c $(GRAMMAR_C).tmp; \ - mv $(GRAMMAR_C).tmp $(GRAMMAR_C); \ - fi + $(PGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C) +$(GRAMMAR_C): @GENERATED_COMMENT@ $(GRAMMAR_H) + touch $(GRAMMAR_C) $(PGEN): $(PGENOBJS) $(CC) $(OPT) $(LDFLAGS) $(PGENOBJS) $(LIBS) -o $(PGEN) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -94,8 +94,8 @@ Build ----- -- Issue #27490: Do not build pgen when cross-compiling. Patch by Thomas - Perl. +- Issue #27641: The configure script now inserts comments into the makefile + to prevent the pgen executable from being cross-compiled. - Issue #26930: Update Windows builds to use OpenSSL 1.0.2h. diff --git a/configure b/configure --- a/configure +++ b/configure @@ -735,7 +735,7 @@ CONFIG_ARGS SOVERSION VERSION -PGEN_DEPENDENCY +GENERATED_COMMENT PYTHON_FOR_BUILD host_os host_vendor @@ -745,7 +745,6 @@ build_vendor build_cpu build -cross_compiling target_alias host_alias build_alias @@ -2800,7 +2799,6 @@ ac_config_headers="$ac_config_headers pyconfig.h" - ac_aux_dir= for ac_dir in "$srcdir" "$srcdir/.." "$srcdir/../.."; do if test -f "$ac_dir/install-sh"; then @@ -2925,12 +2923,13 @@ $as_echo "$interp" >&6; } PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib:$(srcdir)/Lib/$(PLATDIR) '$interp fi - PGEN_DEPENDENCY='' + # Used to comment out stuff for rebuilding generated files + GENERATED_COMMENT='#' elif test "$cross_compiling" = maybe; then as_fn_error $? "Cross compiling required --host=HOST-TUPLE and --build=ARCH" "$LINENO" 5 else PYTHON_FOR_BUILD='./$(BUILDPYTHON) -E' - PGEN_DEPENDENCY='$(PGEN)' + GENERATED_COMMENT='' fi diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -12,7 +12,6 @@ AC_CONFIG_SRCDIR([Include/object.h]) AC_CONFIG_HEADER(pyconfig.h) -AC_SUBST(cross_compiling) AC_CANONICAL_HOST AC_SUBST(build) AC_SUBST(host) @@ -36,15 +35,16 @@ AC_MSG_RESULT($interp) PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib:$(srcdir)/Lib/$(PLATDIR) '$interp fi - PGEN_DEPENDENCY='' + # Used to comment out stuff for rebuilding generated files + GENERATED_COMMENT='#' elif test "$cross_compiling" = maybe; then AC_MSG_ERROR([Cross compiling required --host=HOST-TUPLE and --build=ARCH]) else PYTHON_FOR_BUILD='./$(BUILDPYTHON) -E' - PGEN_DEPENDENCY='$(PGEN)' + GENERATED_COMMENT='' fi AC_SUBST(PYTHON_FOR_BUILD) -AC_SUBST(PGEN_DEPENDENCY) +AC_SUBST(GENERATED_COMMENT) dnl Ensure that if prefix is specified, it does not end in a slash. If dnl it does, we get path names containing '//' which is both ugly and -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 4 01:02:54 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 04 Aug 2016 05:02:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_https_link_to_?= =?utf-8?q?the_bug_tracker_=28closes_=2327670=29?= Message-ID: <20160804050253.79885.31342.0376065D@psf.io> https://hg.python.org/cpython/rev/e9f6af33cf79 changeset: 102528:e9f6af33cf79 branch: 3.5 parent: 102523:bc677cb34889 user: Benjamin Peterson date: Wed Aug 03 22:01:32 2016 -0700 summary: https link to the bug tracker (closes #27670) files: configure | 30 +++++++++++++++++++++--------- configure.ac | 2 +- 2 files changed, 22 insertions(+), 10 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -2,7 +2,7 @@ # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.69 for python 3.5. # -# Report bugs to . +# Report bugs to . # # # Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc. @@ -267,7 +267,7 @@ $as_echo "$0: be upgraded to zsh 4.3.4 or later." else $as_echo "$0: Please tell bug-autoconf at gnu.org and -$0: http://bugs.python.org/ about your system, including +$0: https://bugs.python.org/ about your system, including $0: any error possibly output before this message. Then $0: install a modern shell, or manually run the script $0: under such a shell if you do have one." @@ -582,7 +582,7 @@ PACKAGE_TARNAME='python' PACKAGE_VERSION='3.5' PACKAGE_STRING='python 3.5' -PACKAGE_BUGREPORT='http://bugs.python.org/' +PACKAGE_BUGREPORT='https://bugs.python.org/' PACKAGE_URL='' ac_unique_file="Include/object.h" @@ -774,6 +774,7 @@ docdir oldincludedir includedir +runstatedir localstatedir sharedstatedir sysconfdir @@ -884,6 +885,7 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' +runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1136,6 +1138,15 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; + -runstatedir | --runstatedir | --runstatedi | --runstated \ + | --runstate | --runstat | --runsta | --runst | --runs \ + | --run | --ru | --r) + ac_prev=runstatedir ;; + -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ + | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ + | --run=* | --ru=* | --r=*) + runstatedir=$ac_optarg ;; + -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1273,7 +1284,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir + libdir localedir mandir runstatedir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1426,6 +1437,7 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] + --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -1545,7 +1557,7 @@ Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. -Report bugs to . +Report bugs to . _ACEOF ac_status=$? fi @@ -1813,9 +1825,9 @@ $as_echo "$as_me: WARNING: $2: section \"Present But Cannot Be Compiled\"" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: proceeding with the compiler's result" >&5 $as_echo "$as_me: WARNING: $2: proceeding with the compiler's result" >&2;} -( $as_echo "## -------------------------------------- ## -## Report this to http://bugs.python.org/ ## -## -------------------------------------- ##" +( $as_echo "## --------------------------------------- ## +## Report this to https://bugs.python.org/ ## +## --------------------------------------- ##" ) | sed "s/^/$as_me: WARNING: /" >&2 ;; esac @@ -16868,7 +16880,7 @@ Configuration headers: $config_headers -Report bugs to ." +Report bugs to ." _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -7,7 +7,7 @@ AC_PREREQ(2.65) -AC_INIT(python, PYTHON_VERSION, http://bugs.python.org/) +AC_INIT(python, PYTHON_VERSION, https://bugs.python.org/) AC_SUBST(BASECPPFLAGS) if test "$abs_srcdir" != "$abs_builddir"; then -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 4 01:02:54 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 04 Aug 2016 05:02:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjc2NzAp?= Message-ID: <20160804050254.108680.21075.CE9BB844@psf.io> https://hg.python.org/cpython/rev/b04560c3ce69 changeset: 102530:b04560c3ce69 parent: 102527:e4ddfa61199c parent: 102528:e9f6af33cf79 user: Benjamin Peterson date: Wed Aug 03 22:02:42 2016 -0700 summary: merge 3.5 (#27670) files: configure | 30 +++++++++++++++++++++--------- configure.ac | 2 +- 2 files changed, 22 insertions(+), 10 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -2,7 +2,7 @@ # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.69 for python 3.6. # -# Report bugs to . +# Report bugs to . # # # Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc. @@ -267,7 +267,7 @@ $as_echo "$0: be upgraded to zsh 4.3.4 or later." else $as_echo "$0: Please tell bug-autoconf at gnu.org and -$0: http://bugs.python.org/ about your system, including +$0: https://bugs.python.org/ about your system, including $0: any error possibly output before this message. Then $0: install a modern shell, or manually run the script $0: under such a shell if you do have one." @@ -582,7 +582,7 @@ PACKAGE_TARNAME='python' PACKAGE_VERSION='3.6' PACKAGE_STRING='python 3.6' -PACKAGE_BUGREPORT='http://bugs.python.org/' +PACKAGE_BUGREPORT='https://bugs.python.org/' PACKAGE_URL='' ac_unique_file="Include/object.h" @@ -775,6 +775,7 @@ docdir oldincludedir includedir +runstatedir localstatedir sharedstatedir sysconfdir @@ -885,6 +886,7 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' +runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1137,6 +1139,15 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; + -runstatedir | --runstatedir | --runstatedi | --runstated \ + | --runstate | --runstat | --runsta | --runst | --runs \ + | --run | --ru | --r) + ac_prev=runstatedir ;; + -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ + | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ + | --run=* | --ru=* | --r=*) + runstatedir=$ac_optarg ;; + -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1274,7 +1285,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir + libdir localedir mandir runstatedir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1427,6 +1438,7 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] + --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -1546,7 +1558,7 @@ Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. -Report bugs to . +Report bugs to . _ACEOF ac_status=$? fi @@ -1814,9 +1826,9 @@ $as_echo "$as_me: WARNING: $2: section \"Present But Cannot Be Compiled\"" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: proceeding with the compiler's result" >&5 $as_echo "$as_me: WARNING: $2: proceeding with the compiler's result" >&2;} -( $as_echo "## -------------------------------------- ## -## Report this to http://bugs.python.org/ ## -## -------------------------------------- ##" +( $as_echo "## --------------------------------------- ## +## Report this to https://bugs.python.org/ ## +## --------------------------------------- ##" ) | sed "s/^/$as_me: WARNING: /" >&2 ;; esac @@ -17017,7 +17029,7 @@ Configuration headers: $config_headers -Report bugs to ." +Report bugs to ." _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -7,7 +7,7 @@ AC_PREREQ(2.65) -AC_INIT(python, PYTHON_VERSION, http://bugs.python.org/) +AC_INIT(python, PYTHON_VERSION, https://bugs.python.org/) AC_SUBST(BASECPPFLAGS) if test "$abs_srcdir" != "$abs_builddir"; then -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 4 01:03:51 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 04 Aug 2016 05:03:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_https_link_to_?= =?utf-8?q?the_bug_tracker_=28closes_=2327670=29?= Message-ID: <20160804050253.20147.89782.57574BE0@psf.io> https://hg.python.org/cpython/rev/930450f4ea5f changeset: 102529:930450f4ea5f branch: 2.7 parent: 102526:e3757f3e1848 user: Benjamin Peterson date: Wed Aug 03 22:01:32 2016 -0700 summary: https link to the bug tracker (closes #27670) files: configure | 16 ++++++++-------- configure.ac | 2 +- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -3,7 +3,7 @@ # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.69 for python 2.7. # -# Report bugs to . +# Report bugs to . # # # Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc. @@ -268,7 +268,7 @@ $as_echo "$0: be upgraded to zsh 4.3.4 or later." else $as_echo "$0: Please tell bug-autoconf at gnu.org and -$0: http://bugs.python.org/ about your system, including +$0: https://bugs.python.org/ about your system, including $0: any error possibly output before this message. Then $0: install a modern shell, or manually run the script $0: under such a shell if you do have one." @@ -583,7 +583,7 @@ PACKAGE_TARNAME='python' PACKAGE_VERSION='2.7' PACKAGE_STRING='python 2.7' -PACKAGE_BUGREPORT='http://bugs.python.org/' +PACKAGE_BUGREPORT='https://bugs.python.org/' PACKAGE_URL='' ac_unique_file="Include/object.h" @@ -1545,7 +1545,7 @@ Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. -Report bugs to . +Report bugs to . _ACEOF ac_status=$? fi @@ -1767,9 +1767,9 @@ $as_echo "$as_me: WARNING: $2: section \"Present But Cannot Be Compiled\"" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: proceeding with the compiler's result" >&5 $as_echo "$as_me: WARNING: $2: proceeding with the compiler's result" >&2;} -( $as_echo "## -------------------------------------- ## -## Report this to http://bugs.python.org/ ## -## -------------------------------------- ##" +( $as_echo "## --------------------------------------- ## +## Report this to https://bugs.python.org/ ## +## --------------------------------------- ##" ) | sed "s/^/$as_me: WARNING: /" >&2 ;; esac @@ -15547,7 +15547,7 @@ Configuration headers: $config_headers -Report bugs to ." +Report bugs to ." _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -8,7 +8,7 @@ AC_PREREQ(2.65) AC_REVISION($Revision$) -AC_INIT(python, PYTHON_VERSION, http://bugs.python.org/) +AC_INIT(python, PYTHON_VERSION, https://bugs.python.org/) AC_CONFIG_SRCDIR([Include/object.h]) AC_CONFIG_HEADER(pyconfig.h) -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Thu Aug 4 05:15:50 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 4 Aug 2016 10:15:50 +0100 Subject: [Python-checkins] GOOD Benchmark Results for Python 2.7 2016-08-04 Message-ID: <05f6ad73-d4b6-4188-9dd1-73db2cc2b89c@irsmsx101.ger.corp.intel.com> Results for project Python 2.7, build date 2016-08-04 02:47:38 +0000 commit: af2a7161fa7f previous commit: ba915d561667 revision date: 2016-08-03 22:58:12 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.10% 1.63% 5.26% 3.33% :-) pybench 0.18% 0.08% 5.77% 4.82% :-( regex_v8 0.63% 0.16% -2.40% 10.87% :-) nbody 0.27% 0.07% 8.57% 0.83% :-| json_dump_v2 0.31% -0.27% 1.68% 11.48% :-( normal_startup 1.92% 0.18% -5.96% 1.64% :-| ssbench 0.14% -0.88% 1.86% 2.01% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-2-7-2016-08-04/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Thu Aug 4 05:19:58 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 4 Aug 2016 10:19:58 +0100 Subject: [Python-checkins] UGLY Benchmark Results for Python Default 2016-08-04 Message-ID: Results for project Python default, build date 2016-08-04 02:01:42 +0000 commit: fc034d3607a8 previous commit: adce94a718e3 revision date: 2016-08-04 01:53:40 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.20% 1.27% 9.98% 17.49% :-) pybench 0.09% -0.00% 5.33% 7.09% :-( regex_v8 2.75% 0.16% -3.37% 4.72% :-| nbody 0.28% -0.16% 1.64% 7.47% :-( json_dump_v2 0.33% -1.91% -2.95% 11.60% :-| normal_startup 0.80% -0.08% -1.79% 5.88% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-default-2016-08-04/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Thu Aug 4 10:21:02 2016 From: python-checkins at python.org (berker.peksag) Date: Thu, 04 Aug 2016 14:21:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Indent_ipaddre?= =?utf-8?q?ss=2Eip=5Faddress=28=29_example_correctly?= Message-ID: <20160804142059.26668.23661.DEE7968C@psf.io> https://hg.python.org/cpython/rev/acf61d1cac51 changeset: 102531:acf61d1cac51 branch: 3.5 parent: 102528:e9f6af33cf79 user: Berker Peksag date: Thu Aug 04 17:21:46 2016 +0300 summary: Indent ipaddress.ip_address() example correctly files: Doc/library/ipaddress.rst | 11 ++++------- 1 files changed, 4 insertions(+), 7 deletions(-) diff --git a/Doc/library/ipaddress.rst b/Doc/library/ipaddress.rst --- a/Doc/library/ipaddress.rst +++ b/Doc/library/ipaddress.rst @@ -24,6 +24,10 @@ .. versionadded:: 3.3 +.. testsetup:: + >>> import ipaddress + >>> from ipaddress import (ip_network, IPv4Address, IPv4Interface, + ... IPv4Network) Convenience factory functions ----------------------------- @@ -39,13 +43,6 @@ A :exc:`ValueError` is raised if *address* does not represent a valid IPv4 or IPv6 address. -.. testsetup:: - >>> import ipaddress - >>> from ipaddress import (ip_network, IPv4Address, IPv4Interface, - ... IPv4Network) - -:: - >>> ipaddress.ip_address('192.168.0.1') IPv4Address('192.168.0.1') >>> ipaddress.ip_address('2001:db8::') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 4 10:21:04 2016 From: python-checkins at python.org (berker.peksag) Date: Thu, 04 Aug 2016 14:21:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E5?= Message-ID: <20160804142103.72118.81866.4F1BAF08@psf.io> https://hg.python.org/cpython/rev/f8d502ef21d9 changeset: 102532:f8d502ef21d9 parent: 102530:b04560c3ce69 parent: 102531:acf61d1cac51 user: Berker Peksag date: Thu Aug 04 17:22:05 2016 +0300 summary: Merge from 3.5 files: Doc/library/ipaddress.rst | 11 ++++------- 1 files changed, 4 insertions(+), 7 deletions(-) diff --git a/Doc/library/ipaddress.rst b/Doc/library/ipaddress.rst --- a/Doc/library/ipaddress.rst +++ b/Doc/library/ipaddress.rst @@ -24,6 +24,10 @@ .. versionadded:: 3.3 +.. testsetup:: + >>> import ipaddress + >>> from ipaddress import (ip_network, IPv4Address, IPv4Interface, + ... IPv4Network) Convenience factory functions ----------------------------- @@ -39,13 +43,6 @@ A :exc:`ValueError` is raised if *address* does not represent a valid IPv4 or IPv6 address. -.. testsetup:: - >>> import ipaddress - >>> from ipaddress import (ip_network, IPv4Address, IPv4Interface, - ... IPv4Network) - -:: - >>> ipaddress.ip_address('192.168.0.1') IPv4Address('192.168.0.1') >>> ipaddress.ip_address('2001:db8::') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 4 10:24:55 2016 From: python-checkins at python.org (berker.peksag) Date: Thu, 04 Aug 2016 14:24:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_indentatio?= =?utf-8?q?n_of_versionadded_directive_in_IPv4Address=2Ereverse=5Fpointer_?= =?utf-8?q?docs?= Message-ID: <20160804142454.20153.50831.977786CD@psf.io> https://hg.python.org/cpython/rev/999065975bda changeset: 102533:999065975bda branch: 3.5 parent: 102531:acf61d1cac51 user: Berker Peksag date: Thu Aug 04 17:25:40 2016 +0300 summary: Fix indentation of versionadded directive in IPv4Address.reverse_pointer docs files: Doc/library/ipaddress.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/ipaddress.rst b/Doc/library/ipaddress.rst --- a/Doc/library/ipaddress.rst +++ b/Doc/library/ipaddress.rst @@ -156,7 +156,7 @@ This is the name that could be used for performing a PTR lookup, not the resolved hostname itself. - .. versionadded:: 3.5 + .. versionadded:: 3.5 .. attribute:: is_multicast -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 4 10:25:07 2016 From: python-checkins at python.org (berker.peksag) Date: Thu, 04 Aug 2016 14:25:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E5?= Message-ID: <20160804142454.79993.7851.E1877A00@psf.io> https://hg.python.org/cpython/rev/bf9f39e5d672 changeset: 102534:bf9f39e5d672 parent: 102532:f8d502ef21d9 parent: 102533:999065975bda user: Berker Peksag date: Thu Aug 04 17:26:00 2016 +0300 summary: Merge from 3.5 files: Doc/library/ipaddress.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/ipaddress.rst b/Doc/library/ipaddress.rst --- a/Doc/library/ipaddress.rst +++ b/Doc/library/ipaddress.rst @@ -156,7 +156,7 @@ This is the name that could be used for performing a PTR lookup, not the resolved hostname itself. - .. versionadded:: 3.5 + .. versionadded:: 3.5 .. attribute:: is_multicast -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 4 13:15:24 2016 From: python-checkins at python.org (zach.ware) Date: Thu, 04 Aug 2016 17:15:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20160804171523.80096.24017.F87E6B02@psf.io> https://hg.python.org/cpython/rev/777256817c1b changeset: 102536:777256817c1b parent: 102534:bf9f39e5d672 parent: 102535:9c735c7bc6db user: Zachary Ware date: Thu Aug 04 12:14:51 2016 -0500 summary: Merge with 3.5 files: Doc/using/windows.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -382,7 +382,7 @@ PATH" can be selected to have the installer add the install location into the :envvar:`PATH`. The location of the :file:`Scripts\\` folder is also added. This allows you to type :command:`python` to run the interpreter, and -:command:`pip` or . Thus, you can also execute your +:command:`pip` for the package installer. Thus, you can also execute your scripts with command line options, see :ref:`using-on-cmdline` documentation. If you don't enable this option at install time, you can always re-run the -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 4 13:15:36 2016 From: python-checkins at python.org (zach.ware) Date: Thu, 04 Aug 2016 17:15:36 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_half-finis?= =?utf-8?q?hed_sentence=2E?= Message-ID: <20160804171523.21132.34366.2E32D733@psf.io> https://hg.python.org/cpython/rev/9c735c7bc6db changeset: 102535:9c735c7bc6db branch: 3.5 parent: 102533:999065975bda user: Zachary Ware date: Thu Aug 04 12:13:47 2016 -0500 summary: Fix half-finished sentence. Reported by Linda Pescatore on docs at . files: Doc/using/windows.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -382,7 +382,7 @@ PATH" can be selected to have the installer add the install location into the :envvar:`PATH`. The location of the :file:`Scripts\\` folder is also added. This allows you to type :command:`python` to run the interpreter, and -:command:`pip` or . Thus, you can also execute your +:command:`pip` for the package installer. Thus, you can also execute your scripts with command line options, see :ref:`using-on-cmdline` documentation. If you don't enable this option at install time, you can always re-run the -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 4 22:16:05 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 05 Aug 2016 02:16:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Spelling_and_g?= =?utf-8?q?rammar_fixes_in_code_comments_and_documentation?= Message-ID: <20160805021605.21340.71272.0EB06A32@psf.io> https://hg.python.org/cpython/rev/5ab52b5ea4a7 changeset: 102538:5ab52b5ea4a7 branch: 2.7 user: Martin Panter date: Thu Jul 28 02:37:04 2016 +0000 summary: Spelling and grammar fixes in code comments and documentation files: Include/unicodeobject.h | 4 ++-- Lib/lib-tk/ttk.py | 2 +- Lib/test/test_docxmlrpc.py | 2 +- Lib/test/test_socket.py | 2 +- Misc/HISTORY | 2 +- Misc/NEWS | 2 +- Objects/stringobject.c | 2 +- Objects/typeobject.c | 2 +- Parser/asdl_c.py | 2 +- Python/ast.c | 2 +- Tools/i18n/pygettext.py | 2 +- 11 files changed, 12 insertions(+), 12 deletions(-) diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h --- a/Include/unicodeobject.h +++ b/Include/unicodeobject.h @@ -1138,7 +1138,7 @@ These are capable of handling Unicode objects and strings on input (we refer to them as strings in the descriptions) and return - Unicode objects or integers as apporpriate. */ + Unicode objects or integers as appropriate. */ /* Concat two strings giving a new Unicode string. */ @@ -1288,7 +1288,7 @@ /* Rich compare two strings and return one of the following: - NULL in case an exception was raised - - Py_True or Py_False for successfuly comparisons + - Py_True or Py_False for successful comparisons - Py_NotImplemented in case the type combination is unknown Note that Py_EQ and Py_NE comparisons can cause a UnicodeWarning in diff --git a/Lib/lib-tk/ttk.py b/Lib/lib-tk/ttk.py --- a/Lib/lib-tk/ttk.py +++ b/Lib/lib-tk/ttk.py @@ -153,7 +153,7 @@ def _format_layoutlist(layout, indent=0, indent_size=2): """Formats a layout list so we can pass the result to ttk::style - layout and ttk::style settings. Note that the layout doesn't has to + layout and ttk::style settings. Note that the layout doesn't have to be a list necessarily. E.g.: diff --git a/Lib/test/test_docxmlrpc.py b/Lib/test/test_docxmlrpc.py --- a/Lib/test/test_docxmlrpc.py +++ b/Lib/test/test_docxmlrpc.py @@ -153,7 +153,7 @@ @make_request_and_skipIf(sys.flags.optimize >= 2, "Docstrings are omitted with -O2 and above") def test_system_methods(self): - """Test the precense of three consecutive system.* methods. + """Test the presence of three consecutive system.* methods. This also tests their use of parameter type recognition and the systems related to that process. diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -299,7 +299,7 @@ "Error raising socket exception.") def testSendtoErrors(self): - # Testing that sendto doesn't masks failures. See #10169. + # Testing that sendto doesn't mask failures. See #10169. s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) self.addCleanup(s.close) s.bind(('', 0)) diff --git a/Misc/HISTORY b/Misc/HISTORY --- a/Misc/HISTORY +++ b/Misc/HISTORY @@ -13058,7 +13058,7 @@ - The traceback.py module has a new function tb_lineno() by Marc-Andre Lemburg which extracts the line number from the linenumber table in -the code object. Apparently the traceback object doesn't contains the +the code object. Apparently the traceback object doesn't contain the right linenumber when -O is used. Rather than guessing whether -O is on or off, the module itself uses tb_lineno() unconditionally. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -3506,7 +3506,7 @@ - Issue #14157: Fix time.strptime failing without a year on February 29th. Patch by Hynek Schlawack. -- Issue #14768: os.path.expanduser('~/a') doesn't works correctly when HOME is '/'. +- Issue #14768: os.path.expanduser('~/a') doesn't work correctly when HOME is '/'. - Issue #13183: Fix pdb skipping frames after hitting a breakpoint and running step. Patch by Xavier de Gaye. diff --git a/Objects/stringobject.c b/Objects/stringobject.c --- a/Objects/stringobject.c +++ b/Objects/stringobject.c @@ -901,7 +901,7 @@ fputc(quote, fp); for (i = 0; i < str_len; i++) { /* Since strings are immutable and the caller should have a - reference, accessing the interal buffer should not be an issue + reference, accessing the internal buffer should not be an issue with the GIL released. */ c = op->ob_sval[i]; if (c == quote || c == '\\') diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -1379,7 +1379,7 @@ The next three properties are the 3 constraints in "C3". - Local precendece order. + Local precedence order. If A precedes B in C's MRO, then A will precede B in the MRO of all subclasses of C. diff --git a/Parser/asdl_c.py b/Parser/asdl_c.py --- a/Parser/asdl_c.py +++ b/Parser/asdl_c.py @@ -136,7 +136,7 @@ class StructVisitor(EmitVisitor): - """Visitor to generate typdefs for AST.""" + """Visitor to generate typedefs for AST.""" def visitModule(self, mod): for dfn in mod.dfns: diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -2179,7 +2179,7 @@ testlist: test (',' test)* [','] augassign: '+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=' | '**=' | '//=' - test: ... here starts the operator precendence dance + test: ... here starts the operator precedence dance */ if (NCH(n) == 1) { diff --git a/Tools/i18n/pygettext.py b/Tools/i18n/pygettext.py --- a/Tools/i18n/pygettext.py +++ b/Tools/i18n/pygettext.py @@ -482,7 +482,7 @@ '# File: %(filename)s, line: %(lineno)d') % d elif options.locationstyle == options.GNU: # fit as many locations on one line, as long as the - # resulting line length doesn't exceeds 'options.width' + # resulting line length doesn't exceed 'options.width' locline = '#:' for filename, lineno in v: d = {'filename': filename, 'lineno': lineno} -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 4 22:16:06 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 05 Aug 2016 02:16:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_spelling_a?= =?utf-8?q?nd_grammar_in_documentation_and_code_comments?= Message-ID: <20160805021605.80145.60696.7EDE1534@psf.io> https://hg.python.org/cpython/rev/42d84513c3f0 changeset: 102539:42d84513c3f0 branch: 3.5 parent: 102535:9c735c7bc6db user: Martin Panter date: Thu Aug 04 13:07:31 2016 +0000 summary: Fix spelling and grammar in documentation and code comments files: Doc/library/pdb.rst | 2 +- Include/unicodeobject.h | 2 +- Lib/test/test_cmd_line.py | 2 +- Lib/test/test_docxmlrpc.py | 2 +- Lib/test/test_socket.py | 2 +- Lib/threading.py | 2 +- Lib/tkinter/ttk.py | 2 +- Misc/HISTORY | 4 ++-- Misc/NEWS | 2 +- Misc/coverity_model.c | 2 +- Objects/odictobject.c | 2 +- Objects/typeobject.c | 2 +- Python/ast.c | 2 +- Tools/i18n/pygettext.py | 2 +- 14 files changed, 15 insertions(+), 15 deletions(-) diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -449,7 +449,7 @@ .. pdbcommand:: interact - Start an interative interpreter (using the :mod:`code` module) whose global + Start an interactive interpreter (using the :mod:`code` module) whose global namespace contains all the (global and local) names found in the current scope. diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h --- a/Include/unicodeobject.h +++ b/Include/unicodeobject.h @@ -2014,7 +2014,7 @@ /* Rich compare two strings and return one of the following: - NULL in case an exception was raised - - Py_True or Py_False for successfully comparisons + - Py_True or Py_False for successful comparisons - Py_NotImplemented in case the type combination is unknown Note that Py_EQ and Py_NE comparisons can cause a UnicodeWarning in diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py --- a/Lib/test/test_cmd_line.py +++ b/Lib/test/test_cmd_line.py @@ -465,7 +465,7 @@ rc, out, err = assert_python_ok('-I', '-c', 'from sys import flags as f; ' 'print(f.no_user_site, f.ignore_environment, f.isolated)', - # dummyvar to prevent extranous -E + # dummyvar to prevent extraneous -E dummyvar="") self.assertEqual(out.strip(), b'1 1 1') with test.support.temp_cwd() as tmpdir: diff --git a/Lib/test/test_docxmlrpc.py b/Lib/test/test_docxmlrpc.py --- a/Lib/test/test_docxmlrpc.py +++ b/Lib/test/test_docxmlrpc.py @@ -164,7 +164,7 @@ @make_request_and_skipIf(sys.flags.optimize >= 2, "Docstrings are omitted with -O2 and above") def test_system_methods(self): - """Test the precense of three consecutive system.* methods. + """Test the presence of three consecutive system.* methods. This also tests their use of parameter type recognition and the systems related to that process. diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -709,7 +709,7 @@ raise socket.gaierror def testSendtoErrors(self): - # Testing that sendto doesn't masks failures. See #10169. + # Testing that sendto doesn't mask failures. See #10169. s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) self.addCleanup(s.close) s.bind(('', 0)) diff --git a/Lib/threading.py b/Lib/threading.py --- a/Lib/threading.py +++ b/Lib/threading.py @@ -639,7 +639,7 @@ self._break() raise - # Wait in the barrier until we are relased. Raise an exception + # Wait in the barrier until we are released. Raise an exception # if the barrier is reset or broken. def _wait(self, timeout): if not self._cond.wait_for(lambda : self._state != 0, timeout): diff --git a/Lib/tkinter/ttk.py b/Lib/tkinter/ttk.py --- a/Lib/tkinter/ttk.py +++ b/Lib/tkinter/ttk.py @@ -151,7 +151,7 @@ def _format_layoutlist(layout, indent=0, indent_size=2): """Formats a layout list so we can pass the result to ttk::style - layout and ttk::style settings. Note that the layout doesn't has to + layout and ttk::style settings. Note that the layout doesn't have to be a list necessarily. E.g.: diff --git a/Misc/HISTORY b/Misc/HISTORY --- a/Misc/HISTORY +++ b/Misc/HISTORY @@ -1339,7 +1339,7 @@ Library ------- -- Issue #14768: os.path.expanduser('~/a') doesn't works correctly when HOME is '/'. +- Issue #14768: os.path.expanduser('~/a') doesn't work correctly when HOME is '/'. - Issue #14371: Support bzip2 in zipfile module. Patch by Serhiy Storchaka. @@ -23970,7 +23970,7 @@ - The traceback.py module has a new function tb_lineno() by Marc-Andre Lemburg which extracts the line number from the linenumber table in -the code object. Apparently the traceback object doesn't contains the +the code object. Apparently the traceback object doesn't contain the right linenumber when -O is used. Rather than guessing whether -O is on or off, the module itself uses tb_lineno() unconditionally. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -7127,7 +7127,7 @@ on Windows and adds no value over and above python -m pydoc ... - Issue #18155: The csv module now correctly handles csv files that use - a delimter character that has a special meaning in regexes, instead of + a delimiter character that has a special meaning in regexes, instead of throwing an exception. - Issue #14360: encode_quopri can now be successfully used as an encoder diff --git a/Misc/coverity_model.c b/Misc/coverity_model.c --- a/Misc/coverity_model.c +++ b/Misc/coverity_model.c @@ -30,7 +30,7 @@ typedef struct {} RFILE; /* Python/pythonrun.c - * resourece leak false positive */ + * resource leak false positive */ void Py_FatalError(const char *msg) { __coverity_panic__(); diff --git a/Objects/odictobject.c b/Objects/odictobject.c --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -194,7 +194,7 @@ Here are some ways to address this challenge: 1. Change the relevant usage of the concrete API in CPython and add - PyDict_CheckExact() calls to each of the concrete API funcions. + PyDict_CheckExact() calls to each of the concrete API functions. 2. Adjust the relevant concrete API functions to explicitly accommodate OrderedDict. 3. As with #1, add the checks, but improve the abstract API with smart fast diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -1494,7 +1494,7 @@ The next three properties are the 3 constraints in "C3". - Local precendece order. + Local precedence order. If A precedes B in C's MRO, then A will precede B in the MRO of all subclasses of C. diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -2810,7 +2810,7 @@ testlist_star_expr: (test|star_expr) (',' test|star_expr)* [','] augassign: '+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=' | '**=' | '//=' - test: ... here starts the operator precendence dance + test: ... here starts the operator precedence dance */ if (NCH(n) == 1) { diff --git a/Tools/i18n/pygettext.py b/Tools/i18n/pygettext.py --- a/Tools/i18n/pygettext.py +++ b/Tools/i18n/pygettext.py @@ -475,7 +475,7 @@ '# File: %(filename)s, line: %(lineno)d') % d, file=fp) elif options.locationstyle == options.GNU: # fit as many locations on one line, as long as the - # resulting line length doesn't exceeds 'options.width' + # resulting line length doesn't exceed 'options.width' locline = '#:' for filename, lineno in v: d = {'filename': filename, 'lineno': lineno} -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 4 22:16:05 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 05 Aug 2016 02:16:05 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI2NDYy?= =?utf-8?q?=3A_Doc=3A_avoid_literal=5Fblock_warnings=2C_fix_syntax_highlig?= =?utf-8?q?hting=2E?= Message-ID: <20160805021605.21107.13730.13BCF814@psf.io> https://hg.python.org/cpython/rev/6e9a040a993f changeset: 102537:6e9a040a993f branch: 2.7 parent: 102529:930450f4ea5f user: Martin Panter date: Tue Jul 26 11:18:21 2016 +0200 summary: Issue #26462: Doc: avoid literal_block warnings, fix syntax highlighting. Patch by Julien Palard. files: Doc/distutils/packageindex.rst | 4 +- Doc/extending/building.rst | 9 ++- Doc/extending/embedding.rst | 20 ++++++--- Doc/extending/extending.rst | 4 +- Doc/extending/newtypes.rst | 5 +- Doc/faq/extending.rst | 12 ++++- Doc/howto/logging-cookbook.rst | 16 +++++- Doc/howto/logging.rst | 17 +++++-- Doc/howto/regex.rst | 4 +- Doc/howto/unicode.rst | 4 +- Doc/library/2to3.rst | 28 ++++++++++--- Doc/library/argparse.rst | 20 +++++++-- Doc/library/ast.rst | 1 + Doc/library/cgi.rst | 5 +- Doc/library/doctest.rst | 16 +++++- Doc/library/htmlparser.rst | 4 +- Doc/library/idle.rst | 2 +- Doc/library/logging.config.rst | 37 ++++++++++++----- Doc/library/optparse.rst | 12 ++++- Doc/library/pyexpat.rst | 4 +- Doc/library/shutil.rst | 4 +- Doc/library/socketserver.rst | 12 ++++- Doc/library/subprocess.rst | 21 ++++++--- Doc/library/zipimport.rst | 4 +- Doc/reference/lexical_analysis.rst | 17 ++++++- Doc/tutorial/controlflow.rst | 4 +- Doc/tutorial/interpreter.rst | 4 +- Doc/tutorial/modules.rst | 4 +- Doc/whatsnew/2.3.rst | 13 ++++- Doc/whatsnew/2.4.rst | 4 +- Doc/whatsnew/2.7.rst | 5 +- 31 files changed, 228 insertions(+), 88 deletions(-) diff --git a/Doc/distutils/packageindex.rst b/Doc/distutils/packageindex.rst --- a/Doc/distutils/packageindex.rst +++ b/Doc/distutils/packageindex.rst @@ -235,7 +235,9 @@ To prevent registering broken reStructuredText content, you can use the :program:`rst2html` program that is provided by the :mod:`docutils` package and -check the ``long_description`` from the command line:: +check the ``long_description`` from the command line: + +.. code-block:: shell-session $ python setup.py --long-description | rst2html.py > output.html diff --git a/Doc/extending/building.rst b/Doc/extending/building.rst --- a/Doc/extending/building.rst +++ b/Doc/extending/building.rst @@ -24,7 +24,9 @@ extension. A distutils package contains a driver script, :file:`setup.py`. This is a plain -Python file, which, in the most simple case, could look like this:: +Python file, which, in the most simple case, could look like this: + +.. code-block:: python from distutils.core import setup, Extension @@ -64,7 +66,9 @@ In many cases, building an extension is more complex, since additional preprocessor defines and libraries may be needed. This is demonstrated in the -example below. :: +example below. + +.. code-block:: python from distutils.core import setup, Extension @@ -129,4 +133,3 @@ python setup.py bdist_wininst python setup.py bdist_rpm python setup.py bdist_dumb - diff --git a/Doc/extending/embedding.rst b/Doc/extending/embedding.rst --- a/Doc/extending/embedding.rst +++ b/Doc/extending/embedding.rst @@ -151,7 +151,9 @@ c = c + b return c -then the result should be:: +then the result should be: + +.. code-block:: shell-session $ call multiply multiply 3 2 Will compute 3 times 2 @@ -274,16 +276,20 @@ be directly useful to you: * ``pythonX.Y-config --cflags`` will give you the recommended flags when - compiling:: + compiling: - $ /opt/bin/python2.7-config --cflags - -I/opt/include/python2.7 -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes + .. code-block:: shell-session + + $ /opt/bin/python2.7-config --cflags + -I/opt/include/python2.7 -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes * ``pythonX.Y-config --ldflags`` will give you the recommended flags when - linking:: + linking: - $ /opt/bin/python2.7-config --ldflags - -L/opt/lib/python2.7/config -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic + .. code-block:: shell-session + + $ /opt/bin/python2.7-config --ldflags + -L/opt/lib/python2.7/config -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic .. note:: To avoid confusion between several Python installations (and especially diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -763,7 +763,9 @@ format unit, it returns whatever object is described by that format unit. To force it to return a tuple of size 0 or one, parenthesize the format string. -Examples (to the left the call, to the right the resulting Python value):: +Examples (to the left the call, to the right the resulting Python value): + +.. code-block:: none Py_BuildValue("") None Py_BuildValue("i", 123) 123 diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst --- a/Doc/extending/newtypes.rst +++ b/Doc/extending/newtypes.rst @@ -223,7 +223,9 @@ setup(name="noddy", version="1.0", ext_modules=[Extension("noddy", ["noddy.c"])]) -in a file called :file:`setup.py`; then typing :: +in a file called :file:`setup.py`; then typing + +.. code-block:: shell-session $ python setup.py build @@ -1580,4 +1582,3 @@ .. [#] Even in the third version, we aren't guaranteed to avoid cycles. Instances of string subclasses are allowed and string subclasses could allow cycles even if normal strings don't. - diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst --- a/Doc/faq/extending.rst +++ b/Doc/faq/extending.rst @@ -156,6 +156,8 @@ Sample code and use for catching stdout: +.. code-block:: pycon + >>> class StdoutCatcher: ... def __init__(self): ... self.data = '' @@ -219,11 +221,15 @@ When using GDB with dynamically loaded extensions, you can't set a breakpoint in your extension until your extension is loaded. -In your ``.gdbinit`` file (or interactively), add the command:: +In your ``.gdbinit`` file (or interactively), add the command: + +.. code-block:: none br _PyImport_LoadDynamicModule -Then, when you run GDB:: +Then, when you run GDB: + +.. code-block:: shell-session $ gdb /local/bin/python gdb) run myscript.py @@ -469,6 +475,8 @@ You can check the size of the Unicode character a Python interpreter is using by checking the value of sys.maxunicode: +.. code-block:: pycon + >>> import sys >>> if sys.maxunicode > 65535: ... print 'UCS4 build' diff --git a/Doc/howto/logging-cookbook.rst b/Doc/howto/logging-cookbook.rst --- a/Doc/howto/logging-cookbook.rst +++ b/Doc/howto/logging-cookbook.rst @@ -998,7 +998,9 @@ logger = logging.getLogger('mylogger') logger.debug('A debug message') -To run this, you will probably need to run as ``root``:: +To run this, you will probably need to run as ``root``: + +.. code-block:: shell-session $ sudo python3.3 chowntest.py $ cat chowntest.log @@ -1500,7 +1502,9 @@ completion, the status is as it was before so message #6 appears (like message #1) whereas message #7 doesn't (just like message #2). -If we run the resulting script, the result is as follows:: +If we run the resulting script, the result is as follows: + +.. code-block:: shell-session $ python logctx.py 1. This should appear just once on stderr. @@ -1510,12 +1514,16 @@ 6. This should appear just once on stderr. If we run it again, but pipe ``stderr`` to ``/dev/null``, we see the following, -which is the only message written to ``stdout``:: +which is the only message written to ``stdout``: + +.. code-block:: shell-session $ python logctx.py 2>/dev/null 5. This should appear twice - once on stderr and once on stdout. -Once again, but piping ``stdout`` to ``/dev/null``, we get:: +Once again, but piping ``stdout`` to ``/dev/null``, we get: + +.. code-block:: shell-session $ python logctx.py >/dev/null 1. This should appear just once on stderr. diff --git a/Doc/howto/logging.rst b/Doc/howto/logging.rst --- a/Doc/howto/logging.rst +++ b/Doc/howto/logging.rst @@ -106,7 +106,9 @@ logging.warning('Watch out!') # will print a message to the console logging.info('I told you so') # will not print anything -If you type these lines into a script and run it, you'll see:: +If you type these lines into a script and run it, you'll see: + +.. code-block:: none WARNING:root:Watch out! @@ -230,7 +232,9 @@ import logging logging.warning('%s before you %s', 'Look', 'leap!') -will display:: +will display: + +.. code-block:: none WARNING:root:Look before you leap! @@ -585,7 +589,9 @@ logger.error('error message') logger.critical('critical message') -Running this module from the command line produces the following output:: +Running this module from the command line produces the following output: + +.. code-block:: shell-session $ python simple_logging_module.py 2005-03-19 15:10:26,618 - simple_example - DEBUG - debug message @@ -644,7 +650,9 @@ format=%(asctime)s - %(name)s - %(levelname)s - %(message)s datefmt= -The output is nearly identical to that of the non-config-file-based example:: +The output is nearly identical to that of the non-config-file-based example: + +.. code-block:: shell-session $ python simple_logging_config.py 2005-03-19 15:38:55,977 - simpleExample - DEBUG - debug message @@ -1041,4 +1049,3 @@ Useful handlers included with the logging module. :ref:`A logging cookbook ` - diff --git a/Doc/howto/regex.rst b/Doc/howto/regex.rst --- a/Doc/howto/regex.rst +++ b/Doc/howto/regex.rst @@ -79,7 +79,9 @@ devoted to discussing various metacharacters and what they do. Here's a complete list of the metacharacters; their meanings will be discussed -in the rest of this HOWTO. :: +in the rest of this HOWTO. + +.. code-block:: none . ^ $ * + ? { } [ ] \ | ( ) diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst --- a/Doc/howto/unicode.rst +++ b/Doc/howto/unicode.rst @@ -619,7 +619,9 @@ print os.listdir('.') print os.listdir(u'.') -will produce the following output:: +will produce the following output: + +.. code-block:: shell-session amk:~$ python t.py ['.svn', 'filename\xe4\x94\x80abc', ...] diff --git a/Doc/library/2to3.rst b/Doc/library/2to3.rst --- a/Doc/library/2to3.rst +++ b/Doc/library/2to3.rst @@ -33,14 +33,18 @@ name = raw_input() greet(name) -It can be converted to Python 3.x code via 2to3 on the command line:: +It can be converted to Python 3.x code via 2to3 on the command line: + +.. code-block:: shell-session $ 2to3 example.py A diff against the original source file is printed. 2to3 can also write the needed modifications right back to the source file. (A backup of the original file is made unless :option:`-n` is also given.) Writing the changes back is -enabled with the :option:`-w` flag:: +enabled with the :option:`-w` flag: + +.. code-block:: shell-session $ 2to3 -w example.py @@ -57,17 +61,23 @@ By default, 2to3 runs a set of :ref:`predefined fixers <2to3-fixers>`. The :option:`-l` flag lists all available fixers. An explicit set of fixers to run can be given with :option:`-f`. Likewise the :option:`!-x` explicitly disables a -fixer. The following example runs only the ``imports`` and ``has_key`` fixers:: +fixer. The following example runs only the ``imports`` and ``has_key`` fixers: + +.. code-block:: shell-session $ 2to3 -f imports -f has_key example.py -This command runs every fixer except the ``apply`` fixer:: +This command runs every fixer except the ``apply`` fixer: + +.. code-block:: shell-session $ 2to3 -x apply example.py Some fixers are *explicit*, meaning they aren't run by default and must be listed on the command line to be run. Here, in addition to the default fixers, -the ``idioms`` fixer is run:: +the ``idioms`` fixer is run: + +.. code-block:: shell-session $ 2to3 -f all -f idioms example.py @@ -113,7 +123,9 @@ The :option:`--add-suffix` option specifies a string to append to all output filenames. The :option:`-n` flag is required when specifying this as backups -are not necessary when writing to different filenames. Example:: +are not necessary when writing to different filenames. Example: + +.. code-block:: shell-session $ 2to3 -n -W --add-suffix=3 example.py @@ -122,7 +134,9 @@ .. versionadded:: 2.7.3 The :option:`--add-suffix` option was added. -To translate an entire project from one directory tree to another use:: +To translate an entire project from one directory tree to another use: + +.. code-block:: shell-session $ 2to3 --output-dir=python3-version/mycode -W -n python2-version/mycode diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -44,7 +44,9 @@ print args.accumulate(args.integers) Assuming the Python code above is saved into a file called ``prog.py``, it can -be run at the command line and provides useful help messages:: +be run at the command line and provides useful help messages: + +.. code-block:: shell-session $ python prog.py -h usage: prog.py [-h] [--sum] N [N ...] @@ -59,7 +61,9 @@ --sum sum the integers (default: find the max) When run with the appropriate arguments, it prints either the sum or the max of -the command-line integers:: +the command-line integers: + +.. code-block:: shell-session $ python prog.py 1 2 3 4 4 @@ -67,7 +71,9 @@ $ python prog.py 1 2 3 4 --sum 10 -If invalid arguments are passed in, it will issue an error:: +If invalid arguments are passed in, it will issue an error: + +.. code-block:: shell-session $ python prog.py a b c usage: prog.py [-h] [--sum] N [N ...] @@ -187,7 +193,9 @@ args = parser.parse_args() The help for this program will display ``myprogram.py`` as the program name -(regardless of where the program was invoked from):: +(regardless of where the program was invoked from): + +.. code-block:: shell-session $ python myprogram.py --help usage: myprogram.py [-h] [--foo FOO] @@ -550,7 +558,9 @@ args = parser.parse_args() If ``-h`` or ``--help`` is supplied at the command line, the ArgumentParser -help will be printed:: +help will be printed: + +.. code-block:: shell-session $ python myprogram.py --help usage: myprogram.py [-h] [--foo FOO] diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -113,6 +113,7 @@ The abstract grammar is currently defined as follows: .. literalinclude:: ../../Parser/Python.asdl + :language: none :mod:`ast` Helpers diff --git a/Doc/library/cgi.rst b/Doc/library/cgi.rst --- a/Doc/library/cgi.rst +++ b/Doc/library/cgi.rst @@ -450,7 +450,9 @@ invoked as a script, the file will dump its environment and the contents of the form in HTML form. Give it the right mode etc, and send it a request. If it's installed in the standard :file:`cgi-bin` directory, it should be possible to -send it a request by entering a URL into your browser of the form:: +send it a request by entering a URL into your browser of the form: + +.. code-block:: none http://yourhostname/cgi-bin/cgi.py?name=Joe+Blow&addr=At+Home @@ -542,4 +544,3 @@ field values should be supplied in, but knowing whether a request was received from a conforming browser, or even from a browser at all, is tedious and error-prone. - diff --git a/Doc/library/doctest.rst b/Doc/library/doctest.rst --- a/Doc/library/doctest.rst +++ b/Doc/library/doctest.rst @@ -91,14 +91,18 @@ doctest.testmod() If you run :file:`example.py` directly from the command line, :mod:`doctest` -works its magic:: +works its magic: + +.. code-block:: shell-session $ python example.py $ There's no output! That's normal, and it means all the examples worked. Pass ``-v`` to the script, and :mod:`doctest` prints a detailed log of what -it's trying, and prints a summary at the end:: +it's trying, and prints a summary at the end: + +.. code-block:: shell-session $ python example.py -v Trying: @@ -117,7 +121,9 @@ [1, 1, 2, 6, 24, 120] ok -And so on, eventually ending with:: +And so on, eventually ending with: + +.. code-block:: none Trying: factorial(1e100) @@ -205,7 +211,9 @@ That short script executes and verifies any interactive Python examples contained in the file :file:`example.txt`. The file content is treated as if it were a single giant docstring; the file doesn't need to contain a Python -program! For example, perhaps :file:`example.txt` contains this:: +program! For example, perhaps :file:`example.txt` contains this: + +.. code-block:: none The ``example`` module ====================== diff --git a/Doc/library/htmlparser.rst b/Doc/library/htmlparser.rst --- a/Doc/library/htmlparser.rst +++ b/Doc/library/htmlparser.rst @@ -78,7 +78,9 @@ parser.feed('Test' '

Parse me!

') -The output will then be:: +The output will then be: + +.. code-block:: none Encountered a start tag: html Encountered a start tag: head diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -520,7 +520,7 @@ Command line usage ^^^^^^^^^^^^^^^^^^ -:: +.. code-block:: none idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ... diff --git a/Doc/library/logging.config.rst b/Doc/library/logging.config.rst --- a/Doc/library/logging.config.rst +++ b/Doc/library/logging.config.rst @@ -209,7 +209,9 @@ handler. All *other* keys are passed through as keyword arguments to the - handler's constructor. For example, given the snippet:: + handler's constructor. For example, given the snippet: + + .. code-block:: yaml handlers: console: @@ -318,7 +320,9 @@ configuration to indicate that a connection exists between the source and the destination object with that id. -So, for example, consider the following YAML snippet:: +So, for example, consider the following YAML snippet: + +.. code-block:: yaml formatters: brief: @@ -375,7 +379,9 @@ configuration dictionary and which returns the instantiated object. This is signalled by an absolute import path to the factory being made available under the special key ``'()'``. Here's a concrete -example:: +example: + +.. code-block:: yaml formatters: brief: @@ -592,7 +598,9 @@ :func:`dictConfig`, so it's worth considering transitioning to this newer API when it's convenient to do so. -Examples of these sections in the file are given below. :: +Examples of these sections in the file are given below. + +.. code-block:: ini [loggers] keys=root,log02,log03,log04,log05,log06,log07 @@ -604,7 +612,9 @@ keys=form01,form02,form03,form04,form05,form06,form07,form08,form09 The root logger must specify a level and a list of handlers. An example of a -root logger section is given below. :: +root logger section is given below. + +.. code-block:: ini [logger_root] level=NOTSET @@ -621,7 +631,9 @@ file. For loggers other than the root logger, some additional information is required. -This is illustrated by the following example. :: +This is illustrated by the following example. + +.. code-block:: ini [logger_parser] level=DEBUG @@ -639,7 +651,8 @@ say the name used by the application to get the logger. Sections which specify handler configuration are exemplified by the following. -:: + +.. code-block:: ini [handler_hand01] class=StreamHandler @@ -663,7 +676,9 @@ The ``args`` entry, when :func:`eval`\ uated in the context of the ``logging`` package's namespace, is the list of arguments to the constructor for the handler class. Refer to the constructors for the relevant handlers, or to the examples -below, to see how typical entries are constructed. :: +below, to see how typical entries are constructed. + +.. code-block:: ini [handler_hand02] class=FileHandler @@ -714,7 +729,9 @@ formatter=form09 args=('localhost:9022', '/log', 'GET') -Sections which specify formatter configuration are typified by the following. :: +Sections which specify formatter configuration are typified by the following. + +.. code-block:: ini [formatter_form01] format=F1 %(asctime)s %(levelname)s %(message)s @@ -750,5 +767,3 @@ Module :mod:`logging.handlers` Useful handlers included with the logging module. - - diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst --- a/Doc/library/optparse.rst +++ b/Doc/library/optparse.rst @@ -680,7 +680,9 @@ this option on the command line, it expands your ``version`` string (by replacing ``%prog``), prints it to stdout, and exits. -For example, if your script is called ``/usr/bin/foo``:: +For example, if your script is called ``/usr/bin/foo``: + +.. code-block:: shell-session $ /usr/bin/foo --version foo 1.0 @@ -730,14 +732,18 @@ error status 2. Consider the first example above, where the user passes ``4x`` to an option -that takes an integer:: +that takes an integer: + +.. code-block:: shell-session $ /usr/bin/foo -n 4x Usage: foo [options] foo: error: option -n: invalid integer value: '4x' -Or, where the user fails to pass a value at all:: +Or, where the user fails to pass a value at all: + +.. code-block:: shell-session $ /usr/bin/foo -n Usage: foo [options] diff --git a/Doc/library/pyexpat.rst b/Doc/library/pyexpat.rst --- a/Doc/library/pyexpat.rst +++ b/Doc/library/pyexpat.rst @@ -87,7 +87,9 @@ separator. For example, if *namespace_separator* is set to a space character (``' '``) and - the following document is parsed:: + the following document is parsed: + + .. code-block:: xml >> make_archive(archive_name, 'gztar', root_dir) '/Users/tarek/myarchive.tar.gz' -The resulting archive contains:: +The resulting archive contains: + +.. code-block:: shell-session $ tar -tzvf /Users/tarek/myarchive.tar.gz drwx------ tarek/staff 0 2010-02-01 16:23:40 ./ diff --git a/Doc/library/socketserver.rst b/Doc/library/socketserver.rst --- a/Doc/library/socketserver.rst +++ b/Doc/library/socketserver.rst @@ -481,7 +481,9 @@ The output of the example should look something like this: -Server:: +Server: + +.. code-block:: shell-session $ python TCPServer.py 127.0.0.1 wrote: @@ -489,7 +491,9 @@ 127.0.0.1 wrote: python is nice -Client:: +Client: + +.. code-block:: shell-session $ python TCPClient.py hello world with TCP Sent: hello world with TCP @@ -604,7 +608,9 @@ server.server_close() -The output of the example should look something like this:: +The output of the example should look something like this: + +.. code-block:: shell-session $ python ThreadedTCPServer.py Server loop running in thread: Thread-1 diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -706,20 +706,23 @@ Replacing /bin/sh shell backquote ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -:: +.. code-block:: bash output=`mycmd myarg` - # becomes + +becomes:: + output = check_output(["mycmd", "myarg"]) - Replacing shell pipeline ^^^^^^^^^^^^^^^^^^^^^^^^ -:: +.. code-block:: bash output=`dmesg | grep hda` - # becomes + +becomes:: + p1 = Popen(["dmesg"], stdout=PIPE) p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits. @@ -729,10 +732,14 @@ to receive a SIGPIPE if p2 exits before p1. Alternatively, for trusted input, the shell's own pipeline support may still -be used directly:: +be used directly: + +.. code-block:: bash output=`dmesg | grep hda` - # becomes + +becomes:: + output=check_output("dmesg | grep hda", shell=True) diff --git a/Doc/library/zipimport.rst b/Doc/library/zipimport.rst --- a/Doc/library/zipimport.rst +++ b/Doc/library/zipimport.rst @@ -149,7 +149,9 @@ -------- Here is an example that imports a module from a ZIP archive - note that the -:mod:`zipimport` module is not explicitly used. :: +:mod:`zipimport` module is not explicitly used. + +.. code-block:: shell-session $ unzip -l example.zip Archive: example.zip diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -715,7 +715,10 @@ .. index:: single: operators -The following tokens are operators:: +The following tokens are operators: + +.. code-block:: none + + - * ** / // % << >> & | ^ ~ @@ -732,7 +735,9 @@ .. index:: single: delimiters -The following tokens serve as delimiters in the grammar:: +The following tokens serve as delimiters in the grammar: + +.. code-block:: none ( ) [ ] { } @ , : . ` = ; @@ -745,14 +750,18 @@ but also perform an operation. The following printing ASCII characters have special meaning as part of other -tokens or are otherwise significant to the lexical analyzer:: +tokens or are otherwise significant to the lexical analyzer: + +.. code-block:: none ' " # \ .. index:: single: ASCII at ASCII The following printing ASCII characters are not used in Python. Their -occurrence outside string literals and comments is an unconditional error:: +occurrence outside string literals and comments is an unconditional error: + +.. code-block:: none $ ? diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -473,7 +473,9 @@ client="John Cleese", sketch="Cheese Shop Sketch") -and of course it would print:: +and of course it would print: + +.. code-block:: none -- Do you have any Limburger ? -- I'm sorry, we're all out of Limburger diff --git a/Doc/tutorial/interpreter.rst b/Doc/tutorial/interpreter.rst --- a/Doc/tutorial/interpreter.rst +++ b/Doc/tutorial/interpreter.rst @@ -93,7 +93,9 @@ usually three greater-than signs (``>>>``); for continuation lines it prompts with the *secondary prompt*, by default three dots (``...``). The interpreter prints a welcome message stating its version number and a copyright notice -before printing the first prompt:: +before printing the first prompt: + +.. code-block:: shell-session python Python 2.7 (#1, Feb 28 2010, 00:02:06) diff --git a/Doc/tutorial/modules.rst b/Doc/tutorial/modules.rst --- a/Doc/tutorial/modules.rst +++ b/Doc/tutorial/modules.rst @@ -135,7 +135,9 @@ you can make the file usable as a script as well as an importable module, because the code that parses the command line only runs if the module is -executed as the "main" file:: +executed as the "main" file: + +.. code-block:: shell-session $ python fibo.py 50 1 1 2 3 5 8 13 21 34 diff --git a/Doc/whatsnew/2.3.rst b/Doc/whatsnew/2.3.rst --- a/Doc/whatsnew/2.3.rst +++ b/Doc/whatsnew/2.3.rst @@ -291,7 +291,9 @@ The new :mod:`zipimport` module adds support for importing modules from a ZIP- format archive. You don't need to import the module explicitly; it will be automatically imported if a ZIP archive's filename is added to ``sys.path``. -For example:: +For example: + +.. code-block:: shell-session amk at nyman:~/src/python$ unzip -l /tmp/example.zip Archive: /tmp/example.zip @@ -1761,7 +1763,9 @@ strings containing the remaining arguments. Invoking the script with the various arguments now works as you'd expect it to. -Note that the length argument is automatically converted to an integer. :: +Note that the length argument is automatically converted to an integer. + +.. code-block:: shell-session $ ./python opt.py -i data arg1 @@ -1771,7 +1775,9 @@ [] $ -The help message is automatically generated for you:: +The help message is automatically generated for you: + +.. code-block:: shell-session $ ./python opt.py --help usage: opt.py [options] @@ -2078,4 +2084,3 @@ MacIntyre, Lalo Martins, Chad Netzer, Gustavo Niemeyer, Neal Norwitz, Hans Nowak, Chris Reedy, Francesco Ricciardi, Vinay Sajip, Neil Schemenauer, Roman Suzi, Jason Tishler, Just van Rossum. - diff --git a/Doc/whatsnew/2.4.rst b/Doc/whatsnew/2.4.rst --- a/Doc/whatsnew/2.4.rst +++ b/Doc/whatsnew/2.4.rst @@ -1425,7 +1425,9 @@ print word Running the above function's tests with :const:`doctest.REPORT_UDIFF` specified, -you get the following output:: +you get the following output: + +.. code-block:: none ********************************************************************** File "t.py", line 15, in g diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -2266,7 +2266,9 @@ written in pure Python could cause a segmentation fault by taking a :c:type:`PyCObject` from module A and somehow substituting it for the :c:type:`PyCObject` in module B. Capsules know their own name, -and getting the pointer requires providing the name:: +and getting the pointer requires providing the name: + +.. code-block:: c void *vtable; @@ -2705,4 +2707,3 @@ suggestions, corrections and assistance with various drafts of this article: Nick Coghlan, Philip Jenvey, Ryan Lovett, R. David Murray, Hugh Secker-Walker. - -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 4 22:16:06 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 05 Aug 2016 02:16:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_spelling_and_grammar_fixes_from_3=2E5?= Message-ID: <20160805021606.30699.10807.BAD0AED1@psf.io> https://hg.python.org/cpython/rev/fe189b8bd3ab changeset: 102540:fe189b8bd3ab parent: 102536:777256817c1b parent: 102539:42d84513c3f0 user: Martin Panter date: Fri Aug 05 01:51:39 2016 +0000 summary: Merge spelling and grammar fixes from 3.5 files: Doc/library/pdb.rst | 2 +- Include/unicodeobject.h | 2 +- Lib/test/test_cmd_line.py | 2 +- Lib/test/test_docxmlrpc.py | 2 +- Lib/test/test_socket.py | 2 +- Lib/threading.py | 2 +- Lib/tkinter/ttk.py | 2 +- Misc/HISTORY | 4 ++-- Misc/NEWS | 2 +- Misc/coverity_model.c | 2 +- Objects/odictobject.c | 2 +- Objects/typeobject.c | 2 +- Python/ast.c | 2 +- Tools/i18n/pygettext.py | 2 +- 14 files changed, 15 insertions(+), 15 deletions(-) diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -449,7 +449,7 @@ .. pdbcommand:: interact - Start an interative interpreter (using the :mod:`code` module) whose global + Start an interactive interpreter (using the :mod:`code` module) whose global namespace contains all the (global and local) names found in the current scope. diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h --- a/Include/unicodeobject.h +++ b/Include/unicodeobject.h @@ -2031,7 +2031,7 @@ /* Rich compare two strings and return one of the following: - NULL in case an exception was raised - - Py_True or Py_False for successfully comparisons + - Py_True or Py_False for successful comparisons - Py_NotImplemented in case the type combination is unknown Note that Py_EQ and Py_NE comparisons can cause a UnicodeWarning in diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py --- a/Lib/test/test_cmd_line.py +++ b/Lib/test/test_cmd_line.py @@ -466,7 +466,7 @@ rc, out, err = assert_python_ok('-I', '-c', 'from sys import flags as f; ' 'print(f.no_user_site, f.ignore_environment, f.isolated)', - # dummyvar to prevent extranous -E + # dummyvar to prevent extraneous -E dummyvar="") self.assertEqual(out.strip(), b'1 1 1') with test.support.temp_cwd() as tmpdir: diff --git a/Lib/test/test_docxmlrpc.py b/Lib/test/test_docxmlrpc.py --- a/Lib/test/test_docxmlrpc.py +++ b/Lib/test/test_docxmlrpc.py @@ -164,7 +164,7 @@ @make_request_and_skipIf(sys.flags.optimize >= 2, "Docstrings are omitted with -O2 and above") def test_system_methods(self): - """Test the precense of three consecutive system.* methods. + """Test the presence of three consecutive system.* methods. This also tests their use of parameter type recognition and the systems related to that process. diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -708,7 +708,7 @@ raise socket.gaierror def testSendtoErrors(self): - # Testing that sendto doesn't masks failures. See #10169. + # Testing that sendto doesn't mask failures. See #10169. s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) self.addCleanup(s.close) s.bind(('', 0)) diff --git a/Lib/threading.py b/Lib/threading.py --- a/Lib/threading.py +++ b/Lib/threading.py @@ -641,7 +641,7 @@ self._break() raise - # Wait in the barrier until we are relased. Raise an exception + # Wait in the barrier until we are released. Raise an exception # if the barrier is reset or broken. def _wait(self, timeout): if not self._cond.wait_for(lambda : self._state != 0, timeout): diff --git a/Lib/tkinter/ttk.py b/Lib/tkinter/ttk.py --- a/Lib/tkinter/ttk.py +++ b/Lib/tkinter/ttk.py @@ -153,7 +153,7 @@ def _format_layoutlist(layout, indent=0, indent_size=2): """Formats a layout list so we can pass the result to ttk::style - layout and ttk::style settings. Note that the layout doesn't has to + layout and ttk::style settings. Note that the layout doesn't have to be a list necessarily. E.g.: diff --git a/Misc/HISTORY b/Misc/HISTORY --- a/Misc/HISTORY +++ b/Misc/HISTORY @@ -1339,7 +1339,7 @@ Library ------- -- Issue #14768: os.path.expanduser('~/a') doesn't works correctly when HOME is '/'. +- Issue #14768: os.path.expanduser('~/a') doesn't work correctly when HOME is '/'. - Issue #14371: Support bzip2 in zipfile module. Patch by Serhiy Storchaka. @@ -23970,7 +23970,7 @@ - The traceback.py module has a new function tb_lineno() by Marc-Andre Lemburg which extracts the line number from the linenumber table in -the code object. Apparently the traceback object doesn't contains the +the code object. Apparently the traceback object doesn't contain the right linenumber when -O is used. Rather than guessing whether -O is on or off, the module itself uses tb_lineno() unconditionally. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -8101,7 +8101,7 @@ on Windows and adds no value over and above python -m pydoc ... - Issue #18155: The csv module now correctly handles csv files that use - a delimter character that has a special meaning in regexes, instead of + a delimiter character that has a special meaning in regexes, instead of throwing an exception. - Issue #14360: encode_quopri can now be successfully used as an encoder diff --git a/Misc/coverity_model.c b/Misc/coverity_model.c --- a/Misc/coverity_model.c +++ b/Misc/coverity_model.c @@ -30,7 +30,7 @@ typedef struct {} RFILE; /* Python/pythonrun.c - * resourece leak false positive */ + * resource leak false positive */ void Py_FatalError(const char *msg) { __coverity_panic__(); diff --git a/Objects/odictobject.c b/Objects/odictobject.c --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -194,7 +194,7 @@ Here are some ways to address this challenge: 1. Change the relevant usage of the concrete API in CPython and add - PyDict_CheckExact() calls to each of the concrete API funcions. + PyDict_CheckExact() calls to each of the concrete API functions. 2. Adjust the relevant concrete API functions to explicitly accommodate OrderedDict. 3. As with #1, add the checks, but improve the abstract API with smart fast diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -1504,7 +1504,7 @@ The next three properties are the 3 constraints in "C3". - Local precendece order. + Local precedence order. If A precedes B in C's MRO, then A will precede B in the MRO of all subclasses of C. diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -2852,7 +2852,7 @@ testlist_star_expr: (test|star_expr) (',' test|star_expr)* [','] augassign: '+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=' | '**=' | '//=' - test: ... here starts the operator precendence dance + test: ... here starts the operator precedence dance */ if (NCH(n) == 1) { diff --git a/Tools/i18n/pygettext.py b/Tools/i18n/pygettext.py --- a/Tools/i18n/pygettext.py +++ b/Tools/i18n/pygettext.py @@ -440,7 +440,7 @@ '# File: %(filename)s, line: %(lineno)d') % d, file=fp) elif options.locationstyle == options.GNU: # fit as many locations on one line, as long as the - # resulting line length doesn't exceeds 'options.width' + # resulting line length doesn't exceed 'options.width' locline = '#:' for filename, lineno in v: d = {'filename': filename, 'lineno': lineno} -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Fri Aug 5 06:21:27 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Fri, 5 Aug 2016 11:21:27 +0100 Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7 2016-08-05 Message-ID: Results for project Python 2.7, build date 2016-08-05 02:48:17 +0000 commit: 5ab52b5ea4a7 previous commit: af2a7161fa7f revision date: 2016-07-28 02:37:04 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.10% 0.71% 5.94% 4.62% :-) pybench 0.23% -0.07% 5.71% 4.87% :-( regex_v8 0.63% 0.08% -2.31% 10.49% :-) nbody 0.24% 0.05% 8.62% 1.50% :-) json_dump_v2 0.31% 0.39% 2.07% 11.94% :-( normal_startup 1.70% 0.04% -5.91% 1.73% :-) ssbench 0.19% 0.51% 2.38% 1.45% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-08-05/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Fri Aug 5 06:22:31 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Fri, 5 Aug 2016 11:22:31 +0100 Subject: [Python-checkins] GOOD Benchmark Results for Python Default 2016-08-05 Message-ID: Results for project Python default, build date 2016-08-05 02:02:27 +0000 commit: 777256817c1b previous commit: fc034d3607a8 revision date: 2016-08-04 17:14:51 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.20% 0.70% 10.61% 17.41% :-) pybench 0.12% 0.01% 5.33% 7.23% :-( regex_v8 2.65% -0.14% -3.52% 4.70% :-| nbody 0.24% -0.17% 1.47% 7.80% :-| json_dump_v2 0.26% 1.63% -1.27% 10.19% :-| normal_startup 0.97% 0.16% -1.71% 6.05% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-08-05/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Fri Aug 5 15:56:36 2016 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 05 Aug 2016 19:56:36 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogQWRkIHR5cGluZy5H?= =?utf-8?q?enerator_docs=2C_by_Michael_Lee=2E?= Message-ID: <20160805195636.10510.24482.3E6A81FA@psf.io> https://hg.python.org/cpython/rev/b5403f416836 changeset: 102541:b5403f416836 branch: 3.5 parent: 102539:42d84513c3f0 user: Guido van Rossum date: Fri Aug 05 12:56:09 2016 -0700 summary: Add typing.Generator docs, by Michael Lee. files: Doc/library/typing.rst | 29 +++++++++++++++++++++++++++++ 1 files changed, 29 insertions(+), 0 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -556,6 +556,35 @@ .. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) + A generator can be annotated by the generic type + ``Generator[YieldType, SendType, ReturnType]``. For example:: + + def echo_round() -> Generator[int, float, str]: + sent = yield 0 + while sent >= 0: + sent = yield round(sent) + return 'Done' + + Note that unlike many other generics in the typing module, the ``SendType`` + of :class:`Generator` behaves contravariantly, not covariantly or + invariantly. + + If your generator will only yield values, set the ``SendType`` and + ``ReturnType`` to ``None``:: + + def infinite_stream(start: int) -> Generator[int, None, None]: + while True: + yield start + start += 1 + + Alternatively, annotate your generator as having a return type of + ``Iterator[YieldType]``:: + + def infinite_stream(start: int) -> Iterator[int]: + while True: + yield start + start += 1 + .. class:: io Wrapper namespace for I/O stream types. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 5 15:57:50 2016 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 05 Aug 2016 19:57:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Add_typing=2EGenerator_docs=2C_by_Michael_Lee=2E_=28Merg?= =?utf-8?b?ZSAzLjUtPjMuNik=?= Message-ID: <20160805195749.11373.90636.2CE73B76@psf.io> https://hg.python.org/cpython/rev/f10029fea6ee changeset: 102542:f10029fea6ee parent: 102540:fe189b8bd3ab parent: 102541:b5403f416836 user: Guido van Rossum date: Fri Aug 05 12:57:38 2016 -0700 summary: Add typing.Generator docs, by Michael Lee. (Merge 3.5->3.6) files: Doc/library/typing.rst | 29 +++++++++++++++++++++++++++++ 1 files changed, 29 insertions(+), 0 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -561,6 +561,35 @@ .. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) + A generator can be annotated by the generic type + ``Generator[YieldType, SendType, ReturnType]``. For example:: + + def echo_round() -> Generator[int, float, str]: + sent = yield 0 + while sent >= 0: + sent = yield round(sent) + return 'Done' + + Note that unlike many other generics in the typing module, the ``SendType`` + of :class:`Generator` behaves contravariantly, not covariantly or + invariantly. + + If your generator will only yield values, set the ``SendType`` and + ``ReturnType`` to ``None``:: + + def infinite_stream(start: int) -> Generator[int, None, None]: + while True: + yield start + start += 1 + + Alternatively, annotate your generator as having a return type of + ``Iterator[YieldType]``:: + + def infinite_stream(start: int) -> Iterator[int]: + while True: + yield start + start += 1 + .. class:: io Wrapper namespace for I/O stream types. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 5 16:19:07 2016 From: python-checkins at python.org (vinay.sajip) Date: Fri, 05 Aug 2016 20:19:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIwMTYw?= =?utf-8?q?=3A_Handled_passing_of_large_structs_to_callbacks_correctly=2E?= Message-ID: <20160805201907.79569.40159.0CE04720@psf.io> https://hg.python.org/cpython/rev/09475e6135d0 changeset: 102543:09475e6135d0 branch: 2.7 parent: 102538:5ab52b5ea4a7 user: Vinay Sajip date: Fri Aug 05 21:10:26 2016 +0100 summary: Issue #20160: Handled passing of large structs to callbacks correctly. files: Lib/ctypes/test/test_callbacks.py | 35 +++++++++++++++++++ Modules/_ctypes/_ctypes_test.c | 18 +++++++++ Modules/_ctypes/libffi_msvc/ffi.c | 14 ++++++- 3 files changed, 65 insertions(+), 2 deletions(-) diff --git a/Lib/ctypes/test/test_callbacks.py b/Lib/ctypes/test/test_callbacks.py --- a/Lib/ctypes/test/test_callbacks.py +++ b/Lib/ctypes/test/test_callbacks.py @@ -1,3 +1,4 @@ +import functools import unittest from ctypes import * from ctypes.test import need_symbol @@ -246,6 +247,40 @@ self.assertEqual(result, callback(1.1*1.1, 2.2*2.2, 3.3*3.3, 4.4*4.4, 5.5*5.5)) + def test_callback_large_struct(self): + class Check: pass + + class X(Structure): + _fields_ = [ + ('first', c_ulong), + ('second', c_ulong), + ('third', c_ulong), + ] + + def callback(check, s): + check.first = s.first + check.second = s.second + check.third = s.third + + check = Check() + s = X() + s.first = 0xdeadbeef + s.second = 0xcafebabe + s.third = 0x0bad1dea + + CALLBACK = CFUNCTYPE(None, X) + dll = CDLL(_ctypes_test.__file__) + func = dll._testfunc_cbk_large_struct + func.argtypes = (X, CALLBACK) + func.restype = None + # the function just calls the callback with the passed structure + func(s, CALLBACK(functools.partial(callback, check))) + self.assertEqual(check.first, s.first) + self.assertEqual(check.second, s.second) + self.assertEqual(check.third, s.third) + self.assertEqual(check.first, 0xdeadbeef) + self.assertEqual(check.second, 0xcafebabe) + self.assertEqual(check.third, 0x0bad1dea) ################################################################ diff --git a/Modules/_ctypes/_ctypes_test.c b/Modules/_ctypes/_ctypes_test.c --- a/Modules/_ctypes/_ctypes_test.c +++ b/Modules/_ctypes/_ctypes_test.c @@ -34,6 +34,24 @@ return func(a*a, b*b, c*c, d*d, e*e); } +/* + * This structure should be the same as in test_callbacks.py and the + * method test_callback_large_struct. See issues 17310 and 20160: the + * structure must be larger than 8 bytes long. + */ + +typedef struct { + unsigned long first; + unsigned long second; + unsigned long third; +} Test; + +EXPORT(void) +_testfunc_cbk_large_struct(Test in, void (*func)(Test)) +{ + func(in); +} + EXPORT(void)testfunc_array(int values[4]) { printf("testfunc_array %d %d %d %d\n", diff --git a/Modules/_ctypes/libffi_msvc/ffi.c b/Modules/_ctypes/libffi_msvc/ffi.c --- a/Modules/_ctypes/libffi_msvc/ffi.c +++ b/Modules/_ctypes/libffi_msvc/ffi.c @@ -359,7 +359,7 @@ if ( cif->rtype->type == FFI_TYPE_STRUCT ) { *rvalue = *(void **) argp; - argp += 4; + argp += sizeof(void *); } p_argv = avalue; @@ -370,13 +370,23 @@ /* Align if necessary */ if ((sizeof(char *) - 1) & (size_t) argp) { - argp = (char *) ALIGN(argp, sizeof(char*)); + argp = (char *) ALIGN(argp, sizeof(char*)); } z = (*p_arg)->size; /* because we're little endian, this is what it turns into. */ +#ifdef _WIN64 + if (z > 8) { + /* On Win64, if a single argument takes more than 8 bytes, + * then it is always passed by reference. + */ + *p_argv = *((void**) argp); + z = 8; + } + else +#endif *p_argv = (void*) argp; p_argv++; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 5 16:45:10 2016 From: python-checkins at python.org (vinay.sajip) Date: Fri, 05 Aug 2016 20:45:10 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzIwMTYw?= =?utf-8?q?=3A_Handled_passing_of_large_structs_to_callbacks_correctly=2E?= Message-ID: <20160805204510.72136.69043.FAE35477@psf.io> https://hg.python.org/cpython/rev/4d33bccb59a8 changeset: 102544:4d33bccb59a8 branch: 3.3 parent: 102517:54bfff5503d3 user: Vinay Sajip date: Fri Aug 05 21:24:27 2016 +0100 summary: Issue #20160: Handled passing of large structs to callbacks correctly. files: Lib/ctypes/test/test_callbacks.py | 35 +++++++++++++++++++ Modules/_ctypes/_ctypes_test.c | 18 +++++++++ Modules/_ctypes/libffi_msvc/ffi.c | 14 ++++++- 3 files changed, 65 insertions(+), 2 deletions(-) diff --git a/Lib/ctypes/test/test_callbacks.py b/Lib/ctypes/test/test_callbacks.py --- a/Lib/ctypes/test/test_callbacks.py +++ b/Lib/ctypes/test/test_callbacks.py @@ -1,3 +1,4 @@ +import functools import unittest from ctypes import * import _ctypes_test @@ -243,6 +244,40 @@ self.assertEqual(result, callback(1.1*1.1, 2.2*2.2, 3.3*3.3, 4.4*4.4, 5.5*5.5)) + def test_callback_large_struct(self): + class Check: pass + + class X(Structure): + _fields_ = [ + ('first', c_ulong), + ('second', c_ulong), + ('third', c_ulong), + ] + + def callback(check, s): + check.first = s.first + check.second = s.second + check.third = s.third + + check = Check() + s = X() + s.first = 0xdeadbeef + s.second = 0xcafebabe + s.third = 0x0bad1dea + + CALLBACK = CFUNCTYPE(None, X) + dll = CDLL(_ctypes_test.__file__) + func = dll._testfunc_cbk_large_struct + func.argtypes = (X, CALLBACK) + func.restype = None + # the function just calls the callback with the passed structure + func(s, CALLBACK(functools.partial(callback, check))) + self.assertEqual(check.first, s.first) + self.assertEqual(check.second, s.second) + self.assertEqual(check.third, s.third) + self.assertEqual(check.first, 0xdeadbeef) + self.assertEqual(check.second, 0xcafebabe) + self.assertEqual(check.third, 0x0bad1dea) ################################################################ diff --git a/Modules/_ctypes/_ctypes_test.c b/Modules/_ctypes/_ctypes_test.c --- a/Modules/_ctypes/_ctypes_test.c +++ b/Modules/_ctypes/_ctypes_test.c @@ -26,6 +26,24 @@ return func(a*a, b*b, c*c, d*d, e*e); } +/* + * This structure should be the same as in test_callbacks.py and the + * method test_callback_large_struct. See issues 17310 and 20160: the + * structure must be larger than 8 bytes long. + */ + +typedef struct { + unsigned long first; + unsigned long second; + unsigned long third; +} Test; + +EXPORT(void) +_testfunc_cbk_large_struct(Test in, void (*func)(Test)) +{ + func(in); +} + EXPORT(void)testfunc_array(int values[4]) { printf("testfunc_array %d %d %d %d\n", diff --git a/Modules/_ctypes/libffi_msvc/ffi.c b/Modules/_ctypes/libffi_msvc/ffi.c --- a/Modules/_ctypes/libffi_msvc/ffi.c +++ b/Modules/_ctypes/libffi_msvc/ffi.c @@ -340,7 +340,7 @@ if ( cif->rtype->type == FFI_TYPE_STRUCT ) { *rvalue = *(void **) argp; - argp += 4; + argp += sizeof(void *); } p_argv = avalue; @@ -351,13 +351,23 @@ /* Align if necessary */ if ((sizeof(char *) - 1) & (size_t) argp) { - argp = (char *) ALIGN(argp, sizeof(char*)); + argp = (char *) ALIGN(argp, sizeof(char*)); } z = (*p_arg)->size; /* because we're little endian, this is what it turns into. */ +#ifdef _WIN64 + if (z > 8) { + /* On Win64, if a single argument takes more than 8 bytes, + * then it is always passed by reference. + */ + *p_argv = *((void**) argp); + z = 8; + } + else +#endif *p_argv = (void*) argp; p_argv++; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 5 16:45:12 2016 From: python-checkins at python.org (vinay.sajip) Date: Fri, 05 Aug 2016 20:45:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_Issue_=2320160=3A_Merged_fix_from_3=2E3=2E?= Message-ID: <20160805204511.71776.31508.E2CD8A23@psf.io> https://hg.python.org/cpython/rev/190ebf99bf45 changeset: 102545:190ebf99bf45 branch: 3.4 parent: 102518:f1aefe5947c5 parent: 102544:4d33bccb59a8 user: Vinay Sajip date: Fri Aug 05 21:43:25 2016 +0100 summary: Issue #20160: Merged fix from 3.3. files: Lib/ctypes/test/test_callbacks.py | 35 +++++++++++++++++++ Modules/_ctypes/_ctypes_test.c | 18 +++++++++ Modules/_ctypes/libffi_msvc/ffi.c | 14 ++++++- 3 files changed, 65 insertions(+), 2 deletions(-) diff --git a/Lib/ctypes/test/test_callbacks.py b/Lib/ctypes/test/test_callbacks.py --- a/Lib/ctypes/test/test_callbacks.py +++ b/Lib/ctypes/test/test_callbacks.py @@ -1,3 +1,4 @@ +import functools import unittest from ctypes import * from ctypes.test import need_symbol @@ -240,6 +241,40 @@ self.assertEqual(result, callback(1.1*1.1, 2.2*2.2, 3.3*3.3, 4.4*4.4, 5.5*5.5)) + def test_callback_large_struct(self): + class Check: pass + + class X(Structure): + _fields_ = [ + ('first', c_ulong), + ('second', c_ulong), + ('third', c_ulong), + ] + + def callback(check, s): + check.first = s.first + check.second = s.second + check.third = s.third + + check = Check() + s = X() + s.first = 0xdeadbeef + s.second = 0xcafebabe + s.third = 0x0bad1dea + + CALLBACK = CFUNCTYPE(None, X) + dll = CDLL(_ctypes_test.__file__) + func = dll._testfunc_cbk_large_struct + func.argtypes = (X, CALLBACK) + func.restype = None + # the function just calls the callback with the passed structure + func(s, CALLBACK(functools.partial(callback, check))) + self.assertEqual(check.first, s.first) + self.assertEqual(check.second, s.second) + self.assertEqual(check.third, s.third) + self.assertEqual(check.first, 0xdeadbeef) + self.assertEqual(check.second, 0xcafebabe) + self.assertEqual(check.third, 0x0bad1dea) ################################################################ diff --git a/Modules/_ctypes/_ctypes_test.c b/Modules/_ctypes/_ctypes_test.c --- a/Modules/_ctypes/_ctypes_test.c +++ b/Modules/_ctypes/_ctypes_test.c @@ -26,6 +26,24 @@ return func(a*a, b*b, c*c, d*d, e*e); } +/* + * This structure should be the same as in test_callbacks.py and the + * method test_callback_large_struct. See issues 17310 and 20160: the + * structure must be larger than 8 bytes long. + */ + +typedef struct { + unsigned long first; + unsigned long second; + unsigned long third; +} Test; + +EXPORT(void) +_testfunc_cbk_large_struct(Test in, void (*func)(Test)) +{ + func(in); +} + EXPORT(void)testfunc_array(int values[4]) { printf("testfunc_array %d %d %d %d\n", diff --git a/Modules/_ctypes/libffi_msvc/ffi.c b/Modules/_ctypes/libffi_msvc/ffi.c --- a/Modules/_ctypes/libffi_msvc/ffi.c +++ b/Modules/_ctypes/libffi_msvc/ffi.c @@ -359,7 +359,7 @@ if ( cif->rtype->type == FFI_TYPE_STRUCT ) { *rvalue = *(void **) argp; - argp += 4; + argp += sizeof(void *); } p_argv = avalue; @@ -370,13 +370,23 @@ /* Align if necessary */ if ((sizeof(char *) - 1) & (size_t) argp) { - argp = (char *) ALIGN(argp, sizeof(char*)); + argp = (char *) ALIGN(argp, sizeof(char*)); } z = (*p_arg)->size; /* because we're little endian, this is what it turns into. */ +#ifdef _WIN64 + if (z > 8) { + /* On Win64, if a single argument takes more than 8 bytes, + * then it is always passed by reference. + */ + *p_argv = *((void**) argp); + z = 8; + } + else +#endif *p_argv = (void*) argp; p_argv++; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 5 16:45:13 2016 From: python-checkins at python.org (vinay.sajip) Date: Fri, 05 Aug 2016 20:45:13 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2320160=3A_Merged_fix_from_3=2E4=2E?= Message-ID: <20160805204511.21379.3362.3473560A@psf.io> https://hg.python.org/cpython/rev/24b114d77ec8 changeset: 102546:24b114d77ec8 branch: 3.5 parent: 102541:b5403f416836 parent: 102545:190ebf99bf45 user: Vinay Sajip date: Fri Aug 05 21:44:15 2016 +0100 summary: Issue #20160: Merged fix from 3.4. files: Lib/ctypes/test/test_callbacks.py | 35 +++++++++++++++++++ Modules/_ctypes/_ctypes_test.c | 18 +++++++++ Modules/_ctypes/libffi_msvc/ffi.c | 14 ++++++- 3 files changed, 65 insertions(+), 2 deletions(-) diff --git a/Lib/ctypes/test/test_callbacks.py b/Lib/ctypes/test/test_callbacks.py --- a/Lib/ctypes/test/test_callbacks.py +++ b/Lib/ctypes/test/test_callbacks.py @@ -1,3 +1,4 @@ +import functools import unittest from ctypes import * from ctypes.test import need_symbol @@ -240,6 +241,40 @@ self.assertEqual(result, callback(1.1*1.1, 2.2*2.2, 3.3*3.3, 4.4*4.4, 5.5*5.5)) + def test_callback_large_struct(self): + class Check: pass + + class X(Structure): + _fields_ = [ + ('first', c_ulong), + ('second', c_ulong), + ('third', c_ulong), + ] + + def callback(check, s): + check.first = s.first + check.second = s.second + check.third = s.third + + check = Check() + s = X() + s.first = 0xdeadbeef + s.second = 0xcafebabe + s.third = 0x0bad1dea + + CALLBACK = CFUNCTYPE(None, X) + dll = CDLL(_ctypes_test.__file__) + func = dll._testfunc_cbk_large_struct + func.argtypes = (X, CALLBACK) + func.restype = None + # the function just calls the callback with the passed structure + func(s, CALLBACK(functools.partial(callback, check))) + self.assertEqual(check.first, s.first) + self.assertEqual(check.second, s.second) + self.assertEqual(check.third, s.third) + self.assertEqual(check.first, 0xdeadbeef) + self.assertEqual(check.second, 0xcafebabe) + self.assertEqual(check.third, 0x0bad1dea) ################################################################ diff --git a/Modules/_ctypes/_ctypes_test.c b/Modules/_ctypes/_ctypes_test.c --- a/Modules/_ctypes/_ctypes_test.c +++ b/Modules/_ctypes/_ctypes_test.c @@ -26,6 +26,24 @@ return func(a*a, b*b, c*c, d*d, e*e); } +/* + * This structure should be the same as in test_callbacks.py and the + * method test_callback_large_struct. See issues 17310 and 20160: the + * structure must be larger than 8 bytes long. + */ + +typedef struct { + unsigned long first; + unsigned long second; + unsigned long third; +} Test; + +EXPORT(void) +_testfunc_cbk_large_struct(Test in, void (*func)(Test)) +{ + func(in); +} + EXPORT(void)testfunc_array(int values[4]) { printf("testfunc_array %d %d %d %d\n", diff --git a/Modules/_ctypes/libffi_msvc/ffi.c b/Modules/_ctypes/libffi_msvc/ffi.c --- a/Modules/_ctypes/libffi_msvc/ffi.c +++ b/Modules/_ctypes/libffi_msvc/ffi.c @@ -378,7 +378,7 @@ if ( cif->rtype->type == FFI_TYPE_STRUCT ) { *rvalue = *(void **) argp; - argp += 4; + argp += sizeof(void *); } p_argv = avalue; @@ -389,13 +389,23 @@ /* Align if necessary */ if ((sizeof(char *) - 1) & (size_t) argp) { - argp = (char *) ALIGN(argp, sizeof(char*)); + argp = (char *) ALIGN(argp, sizeof(char*)); } z = (*p_arg)->size; /* because we're little endian, this is what it turns into. */ +#ifdef _WIN64 + if (z > 8) { + /* On Win64, if a single argument takes more than 8 bytes, + * then it is always passed by reference. + */ + *p_argv = *((void**) argp); + z = 8; + } + else +#endif *p_argv = (void*) argp; p_argv++; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 5 16:45:12 2016 From: python-checkins at python.org (vinay.sajip) Date: Fri, 05 Aug 2016 20:45:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2320160=3A_Merged_fix_from_3=2E5=2E?= Message-ID: <20160805204512.108726.3947.D4B4E23B@psf.io> https://hg.python.org/cpython/rev/ec9b4d93662d changeset: 102547:ec9b4d93662d parent: 102542:f10029fea6ee parent: 102546:24b114d77ec8 user: Vinay Sajip date: Fri Aug 05 21:44:52 2016 +0100 summary: Closes #20160: Merged fix from 3.5. files: Lib/ctypes/test/test_callbacks.py | 35 +++++++++++++++++++ Modules/_ctypes/_ctypes_test.c | 18 +++++++++ Modules/_ctypes/libffi_msvc/ffi.c | 14 ++++++- 3 files changed, 65 insertions(+), 2 deletions(-) diff --git a/Lib/ctypes/test/test_callbacks.py b/Lib/ctypes/test/test_callbacks.py --- a/Lib/ctypes/test/test_callbacks.py +++ b/Lib/ctypes/test/test_callbacks.py @@ -1,3 +1,4 @@ +import functools import unittest from ctypes import * from ctypes.test import need_symbol @@ -240,6 +241,40 @@ self.assertEqual(result, callback(1.1*1.1, 2.2*2.2, 3.3*3.3, 4.4*4.4, 5.5*5.5)) + def test_callback_large_struct(self): + class Check: pass + + class X(Structure): + _fields_ = [ + ('first', c_ulong), + ('second', c_ulong), + ('third', c_ulong), + ] + + def callback(check, s): + check.first = s.first + check.second = s.second + check.third = s.third + + check = Check() + s = X() + s.first = 0xdeadbeef + s.second = 0xcafebabe + s.third = 0x0bad1dea + + CALLBACK = CFUNCTYPE(None, X) + dll = CDLL(_ctypes_test.__file__) + func = dll._testfunc_cbk_large_struct + func.argtypes = (X, CALLBACK) + func.restype = None + # the function just calls the callback with the passed structure + func(s, CALLBACK(functools.partial(callback, check))) + self.assertEqual(check.first, s.first) + self.assertEqual(check.second, s.second) + self.assertEqual(check.third, s.third) + self.assertEqual(check.first, 0xdeadbeef) + self.assertEqual(check.second, 0xcafebabe) + self.assertEqual(check.third, 0x0bad1dea) ################################################################ diff --git a/Modules/_ctypes/_ctypes_test.c b/Modules/_ctypes/_ctypes_test.c --- a/Modules/_ctypes/_ctypes_test.c +++ b/Modules/_ctypes/_ctypes_test.c @@ -26,6 +26,24 @@ return func(a*a, b*b, c*c, d*d, e*e); } +/* + * This structure should be the same as in test_callbacks.py and the + * method test_callback_large_struct. See issues 17310 and 20160: the + * structure must be larger than 8 bytes long. + */ + +typedef struct { + unsigned long first; + unsigned long second; + unsigned long third; +} Test; + +EXPORT(void) +_testfunc_cbk_large_struct(Test in, void (*func)(Test)) +{ + func(in); +} + EXPORT(void)testfunc_array(int values[4]) { printf("testfunc_array %d %d %d %d\n", diff --git a/Modules/_ctypes/libffi_msvc/ffi.c b/Modules/_ctypes/libffi_msvc/ffi.c --- a/Modules/_ctypes/libffi_msvc/ffi.c +++ b/Modules/_ctypes/libffi_msvc/ffi.c @@ -378,7 +378,7 @@ if ( cif->rtype->type == FFI_TYPE_STRUCT ) { *rvalue = *(void **) argp; - argp += 4; + argp += sizeof(void *); } p_argv = avalue; @@ -389,13 +389,23 @@ /* Align if necessary */ if ((sizeof(char *) - 1) & (size_t) argp) { - argp = (char *) ALIGN(argp, sizeof(char*)); + argp = (char *) ALIGN(argp, sizeof(char*)); } z = (*p_arg)->size; /* because we're little endian, this is what it turns into. */ +#ifdef _WIN64 + if (z > 8) { + /* On Win64, if a single argument takes more than 8 bytes, + * then it is always passed by reference. + */ + *p_argv = *((void**) argp); + z = 8; + } + else +#endif *p_argv = (void*) argp; p_argv++; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 5 18:10:31 2016 From: python-checkins at python.org (ethan.furman) Date: Fri, 05 Aug 2016 22:10:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Clarify_NotImplemented_vs_?= =?utf-8?q?NotImplementedError=2E__Initial_patch_by_Emmanuel?= Message-ID: <20160805221031.30677.44886.2DFA717D@psf.io> https://hg.python.org/cpython/rev/142f5325af06 changeset: 102548:142f5325af06 user: Ethan Furman date: Fri Aug 05 15:10:16 2016 -0700 summary: Clarify NotImplemented vs NotImplementedError. Initial patch by Emmanuel Barry. Closes issue 27242. files: Doc/library/constants.rst | 22 ++++++++++++++-------- Doc/library/exceptions.rst | 25 +++++++++++++++++++++++-- 2 files changed, 37 insertions(+), 10 deletions(-) diff --git a/Doc/library/constants.rst b/Doc/library/constants.rst --- a/Doc/library/constants.rst +++ b/Doc/library/constants.rst @@ -33,16 +33,22 @@ (e.g. :meth:`__imul__`, :meth:`__iand__`, etc.) for the same purpose. Its truth value is true. -.. note:: + .. note:: - When ``NotImplemented`` is returned, the interpreter will then try the - reflected operation on the other type, or some other fallback, depending - on the operator. If all attempted operations return ``NotImplemented``, the - interpreter will raise an appropriate exception. + When a binary (or in-place) method returns ``NotImplemented`` the + interpreter will try the reflected operation on the other type (or some + other fallback, depending on the operator). If all attempts return + ``NotImplemented``, the interpreter will raise an appropriate exception. + Incorrectly returning ``NotImplemented`` will result in a misleading + error message or the ``NotImplemented`` value being returned to Python code. - See - :ref:`implementing-the-arithmetic-operations` - for more details. + See :ref:`implementing-the-arithmetic-operations` for examples. + + .. note:: + + ``NotImplentedError`` and ``NotImplemented`` are not interchangeable, + even though they have similar names and purposes. + See :exc:`NotImplementedError` for details on when to use it. .. data:: Ellipsis diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -228,9 +228,21 @@ .. exception:: NotImplementedError This exception is derived from :exc:`RuntimeError`. In user defined base - classes, abstract methods should raise this exception when they require derived - classes to override the method. + classes, abstract methods should raise this exception when they require + derived classes to override the method, or while the class is being + developed to indicate that the real implementation still needs to be added. + .. note:: + + It should not be used to indicate that an operater or method is not + meant to be supported at all -- in that case either leave the operator / + method undefined or, if a subclass, set it to :data:`None`. + + .. note:: + + ``NotImplementedError`` and ``NotImplemented`` are not interchangeable, + even though they have similar names and purposes. See + :data:`NotImplemented` for details on when to use it. .. exception:: OSError([arg]) OSError(errno, strerror[, filename[, winerror[, filename2]]]) @@ -436,6 +448,15 @@ Raised when an operation or function is applied to an object of inappropriate type. The associated value is a string giving details about the type mismatch. + This exception may be raised by user code to indicate that an attempted + operation on an object is not supported, and is not meant to be. If an object + is meant to support a given operation but has not yet provided an + implementation, :exc:`NotImplementedError` is the proper exception to raise. + + Passing arguments of the wrong type (e.g. passing a :class:`list` when an + :class:`int` is expected) should result in a :exc:`TypeError`, but passing + arguments with the wrong value (e.g. a number outside expected boundaries) + should result in a :exc:`ValueError`. .. exception:: UnboundLocalError -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 5 19:18:42 2016 From: python-checkins at python.org (ethan.furman) Date: Fri, 05 Aug 2016 23:18:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_AutoEnum=3A_automatica?= =?utf-8?q?lly_provides_next_value_if_missing=2E__Issue_26988=2E?= Message-ID: <20160805231842.11028.1377.0D5F608D@psf.io> https://hg.python.org/cpython/rev/7ed7d7f58fcd changeset: 102549:7ed7d7f58fcd user: Ethan Furman date: Fri Aug 05 16:03:16 2016 -0700 summary: Add AutoEnum: automatically provides next value if missing. Issue 26988. files: Doc/library/enum.rst | 279 +++++++++++++++++++----- Lib/enum.py | 133 +++++++++++- Lib/test/test_enum.py | 324 +++++++++++++++++++++++++++++- Misc/NEWS | 2 + 4 files changed, 661 insertions(+), 77 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -37,6 +37,13 @@ Base class for creating enumerated constants that are also subclasses of :class:`int`. +.. class:: AutoEnum + + Base class for creating automatically numbered members (may + be combined with IntEnum if desired). + + .. versionadded:: 3.6 + .. function:: unique Enum class decorator that ensures only one name is bound to any one value. @@ -47,14 +54,14 @@ Enumerations are created using the :keyword:`class` syntax, which makes them easy to read and write. An alternative creation method is described in -`Functional API`_. To define an enumeration, subclass :class:`Enum` as -follows:: +`Functional API`_. To define a simple enumeration, subclass :class:`AutoEnum` +as follows:: - >>> from enum import Enum - >>> class Color(Enum): - ... red = 1 - ... green = 2 - ... blue = 3 + >>> from enum import AutoEnum + >>> class Color(AutoEnum): + ... red + ... green + ... blue ... .. note:: Nomenclature @@ -72,6 +79,33 @@ are not normal Python classes. See `How are Enums different?`_ for more details. +To create your own automatic :class:`Enum` classes, you need to add a +:meth:`_generate_next_value_` method; it will be used to create missing values +for any members after its definition. + +.. versionadded:: 3.6 + +If you need full control of the member values, use :class:`Enum` as the base +class and specify the values manually:: + + >>> from enum import Enum + >>> class Color(Enum): + ... red = 19 + ... green = 7.9182 + ... blue = 'periwinkle' + ... + +We'll use the following Enum for the examples below:: + + >>> class Color(Enum): + ... red = 1 + ... green = 2 + ... blue = 3 + ... + +Enum Details +------------ + Enumeration members have human readable string representations:: >>> print(Color.red) @@ -235,7 +269,11 @@ The ``__members__`` attribute can be used for detailed programmatic access to the enumeration members. For example, finding all the aliases:: - >>> [name for name, member in Shape.__members__.items() if member.name != name] + >>> [ + ... name + ... for name, member in Shape.__members__.items() + ... if member.name != name + ... ] ['alias_for_square'] @@ -257,7 +295,7 @@ >>> Color.red < Color.blue Traceback (most recent call last): File "", line 1, in - TypeError: unorderable types: Color() < Color() + TypeError: '<' not supported between instances of 'Color' and 'Color' Equality comparisons are defined though:: @@ -280,10 +318,10 @@ ---------------------------------------------- The examples above use integers for enumeration values. Using integers is -short and handy (and provided by default by the `Functional API`_), but not -strictly enforced. In the vast majority of use-cases, one doesn't care what -the actual value of an enumeration is. But if the value *is* important, -enumerations can have arbitrary values. +short and handy (and provided by default by :class:`AutoEnum` and the +`Functional API`_), but not strictly enforced. In the vast majority of +use-cases, one doesn't care what the actual value of an enumeration is. +But if the value *is* important, enumerations can have arbitrary values. Enumerations are Python classes, and can have methods and special methods as usual. If we have this enumeration:: @@ -393,17 +431,21 @@ >>> list(Animal) [, , , ] -The semantics of this API resemble :class:`~collections.namedtuple`. The first -argument of the call to :class:`Enum` is the name of the enumeration. +The semantics of this API resemble :class:`~collections.namedtuple`. -The second argument is the *source* of enumeration member names. It can be a -whitespace-separated string of names, a sequence of names, a sequence of -2-tuples with key/value pairs, or a mapping (e.g. dictionary) of names to -values. The last two options enable assigning arbitrary values to -enumerations; the others auto-assign increasing integers starting with 1 (use -the ``start`` parameter to specify a different starting value). A -new class derived from :class:`Enum` is returned. In other words, the above -assignment to :class:`Animal` is equivalent to:: +- the first argument of the call to :class:`Enum` is the name of the + enumeration; + +- the second argument is the *source* of enumeration member names. It can be a + whitespace-separated string of names, a sequence of names, a sequence of + 2-tuples with key/value pairs, or a mapping (e.g. dictionary) of names to + values; + +- the last two options enable assigning arbitrary values to enumerations; the + others auto-assign increasing integers starting with 1 (use the ``start`` + parameter to specify a different starting value). A new class derived from + :class:`Enum` is returned. In other words, the above assignment to + :class:`Animal` is equivalent to:: >>> class Animal(Enum): ... ant = 1 @@ -419,7 +461,7 @@ Pickling enums created with the functional API can be tricky as frame stack implementation details are used to try and figure out which module the enumeration is being created in (e.g. it will fail if you use a utility -function in separate module, and also may not work on IronPython or Jython). +function in a separate module, and also may not work on IronPython or Jython). The solution is to specify the module name explicitly as follows:: >>> Animal = Enum('Animal', 'ant bee cat dog', module=__name__) @@ -439,7 +481,15 @@ The complete signature is:: - Enum(value='NewEnumName', names=<...>, *, module='...', qualname='...', type=, start=1) + Enum( + value='NewEnumName', + names=<...>, + *, + module='...', + qualname='...', + type=, + start=1, + ) :value: What the new Enum class will record as its name. @@ -475,10 +525,41 @@ Derived Enumerations -------------------- +AutoEnum +^^^^^^^^ + +This version of :class:`Enum` automatically assigns numbers as the values +for the enumeration members, while still allowing values to be specified +when needed:: + + >>> from enum import AutoEnum + >>> class Color(AutoEnum): + ... red + ... green = 5 + ... blue + ... + >>> list(Color) + [, , ] + +.. note:: Name Lookup + + By default the names :func:`property`, :func:`classmethod`, and + :func:`staticmethod` are shielded from becoming members. To enable + them, or to specify a different set of shielded names, specify the + ignore parameter:: + + >>> class AddressType(AutoEnum, ignore='classmethod staticmethod'): + ... pobox + ... mailbox + ... property + ... + +.. versionadded:: 3.6 + IntEnum ^^^^^^^ -A variation of :class:`Enum` is provided which is also a subclass of +Another variation of :class:`Enum` which is also a subclass of :class:`int`. Members of an :class:`IntEnum` can be compared to integers; by extension, integer enumerations of different types can also be compared to each other:: @@ -521,14 +602,13 @@ >>> [i for i in range(Shape.square)] [0, 1] -For the vast majority of code, :class:`Enum` is strongly recommended, -since :class:`IntEnum` breaks some semantic promises of an enumeration (by -being comparable to integers, and thus by transitivity to other -unrelated enumerations). It should be used only in special cases where -there's no other choice; for example, when integer constants are -replaced with enumerations and backwards compatibility is required with code -that still expects integers. - +For the vast majority of code, :class:`Enum` and :class:`AutoEnum` are strongly +recommended, since :class:`IntEnum` breaks some semantic promises of an +enumeration (by being comparable to integers, and thus by transitivity to other +unrelated ``IntEnum`` enumerations). It should be used only in special cases +where there's no other choice; for example, when integer constants are replaced +with enumerations and backwards compatibility is required with code that still +expects integers. Others ^^^^^^ @@ -540,7 +620,9 @@ pass This demonstrates how similar derived enumerations can be defined; for example -a :class:`StrEnum` that mixes in :class:`str` instead of :class:`int`. +an :class:`AutoIntEnum` that mixes in :class:`int` with :class:`AutoEnum` +to get members that are :class:`int` (but keep in mind the warnings for +:class:`IntEnum`). Some rules: @@ -567,31 +649,35 @@ Interesting examples -------------------- -While :class:`Enum` and :class:`IntEnum` are expected to cover the majority of -use-cases, they cannot cover them all. Here are recipes for some different -types of enumerations that can be used directly, or as examples for creating -one's own. +While :class:`Enum`, :class:`AutoEnum`, and :class:`IntEnum` are expected +to cover the majority of use-cases, they cannot cover them all. Here are +recipes for some different types of enumerations that can be used directly, +or as examples for creating one's own. -AutoNumber -^^^^^^^^^^ +AutoDocEnum +^^^^^^^^^^^ -Avoids having to specify the value for each enumeration member:: +Automatically numbers the members, and uses the given value as the +:attr:`__doc__` string:: - >>> class AutoNumber(Enum): - ... def __new__(cls): + >>> class AutoDocEnum(Enum): + ... def __new__(cls, doc): ... value = len(cls.__members__) + 1 ... obj = object.__new__(cls) ... obj._value_ = value + ... obj.__doc__ = doc ... return obj ... - >>> class Color(AutoNumber): - ... red = () - ... green = () - ... blue = () + >>> class Color(AutoDocEnum): + ... red = 'stop' + ... green = 'go' + ... blue = 'what?' ... >>> Color.green.value == 2 True + >>> Color.green.__doc__ + 'go' .. note:: @@ -599,6 +685,23 @@ members; it is then replaced by Enum's :meth:`__new__` which is used after class creation for lookup of existing members. +AutoNameEnum +^^^^^^^^^^^^ + +Automatically sets the member's value to its name:: + + >>> class AutoNameEnum(Enum): + ... def _generate_next_value_(name, start, count, last_value): + ... return name + ... + >>> class Color(AutoNameEnum): + ... red + ... green + ... blue + ... + >>> Color.green.value == 'green' + True + OrderedEnum ^^^^^^^^^^^ @@ -731,10 +834,61 @@ Finer Points ^^^^^^^^^^^^ -:class:`Enum` members are instances of an :class:`Enum` class, and even -though they are accessible as `EnumClass.member`, they should not be accessed +Enum class signature +~~~~~~~~~~~~~~~~~~~~ + + ``class SomeName( + AnEnum, + start=None, + ignore='staticmethod classmethod property', + ):`` + +``start`` can be used by a :meth:`_generate_next_value_` method to specify a +starting value. + +``ignore`` specifies which names, if any, will not attempt to auto-generate +a new value (they will also be removed from the class body). + + +Supported ``__dunder__`` names +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The :attr:`__members__` attribute is only available on the class. + +:meth:`__new__`, if specified, must create and return the enum members; it is +also a very good idea to set the member's :attr:`_value_` appropriately. Once +all the members are created it is no longer used. + + +Supported ``_sunder_`` names +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- ``_order_`` -- used in Python 2/3 code to ensure member order is consistent [class attribute] + +- ``_name_`` -- name of the member (but use ``name`` for normal access) +- ``_value_`` -- value of the member; can be set / modified in ``__new__`` (see ``_name_``) +- ``_missing_`` -- a lookup function used when a value is not found (only after class creation) +- ``_generate_next_value_`` -- a function to generate missing values (only during class creation) + + +:meth:`_generate_next_value_` signature +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + ``def _generate_next_value_(name, start, count, last_value):`` + +- ``name`` is the name of the member +- ``start`` is the initital start value (if any) or None +- ``count`` is the number of existing members in the enumeration +- ``last_value`` is the value of the last enum member (if any) or None + + +Enum member type +~~~~~~~~~~~~~~~~ + +``Enum`` members are instances of an ``Enum`` class, and even +though they are accessible as ``EnumClass.member``, they should not be accessed directly from the member as that lookup may fail or, worse, return something -besides the :class:`Enum` member you looking for:: +besides the ``Enum`` member you are looking for:: >>> class FieldTypes(Enum): ... name = 0 @@ -748,18 +902,24 @@ .. versionchanged:: 3.5 -Boolean evaluation: Enum classes that are mixed with non-Enum types (such as + +Boolean value of ``Enum`` classes and members +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Enum classes that are mixed with non-Enum types (such as :class:`int`, :class:`str`, etc.) are evaluated according to the mixed-in -type's rules; otherwise, all members evaluate as ``True``. To make your own +type's rules; otherwise, all members evaluate as :data:`True`. To make your own Enum's boolean evaluation depend on the member's value add the following to your class:: def __bool__(self): return bool(self.value) -The :attr:`__members__` attribute is only available on the class. -If you give your :class:`Enum` subclass extra methods, like the `Planet`_ +Enum classes with methods +~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you give your ``Enum`` subclass extra methods, like the `Planet`_ class above, those methods will show up in a :func:`dir` of the member, but not of the class:: @@ -767,12 +927,3 @@ ['EARTH', 'JUPITER', 'MARS', 'MERCURY', 'NEPTUNE', 'SATURN', 'URANUS', 'VENUS', '__class__', '__doc__', '__members__', '__module__'] >>> dir(Planet.EARTH) ['__class__', '__doc__', '__module__', 'name', 'surface_gravity', 'value'] - -The :meth:`__new__` method will only be used for the creation of the -:class:`Enum` members -- after that it is replaced. Any custom :meth:`__new__` -method must create the object and set the :attr:`_value_` attribute -appropriately. - -If you wish to change how :class:`Enum` members are looked up you should either -write a helper function or a :func:`classmethod` for the :class:`Enum` -subclass. diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -8,7 +8,9 @@ from collections import OrderedDict -__all__ = ['EnumMeta', 'Enum', 'IntEnum', 'unique'] +__all__ = [ + 'EnumMeta', 'Enum', 'IntEnum', 'AutoEnum', 'unique', + ] def _is_descriptor(obj): @@ -52,7 +54,30 @@ """ def __init__(self): super().__init__() + # list of enum members self._member_names = [] + # starting value + self._start = None + # last assigned value + self._last_value = None + # when the magic turns off + self._locked = True + # list of temporary names + self._ignore = [] + + def __getitem__(self, key): + if ( + self._generate_next_value_ is None + or self._locked + or key in self + or key in self._ignore + or _is_sunder(key) + or _is_dunder(key) + ): + return super(_EnumDict, self).__getitem__(key) + next_value = self._generate_next_value_(key, self._start, len(self._member_names), self._last_value) + self[key] = next_value + return next_value def __setitem__(self, key, value): """Changes anything not dundered or not a descriptor. @@ -64,19 +89,55 @@ """ if _is_sunder(key): - raise ValueError('_names_ are reserved for future Enum use') + if key not in ('_settings_', '_order_', '_ignore_', '_start_', '_generate_next_value_'): + raise ValueError('_names_ are reserved for future Enum use') + elif key == '_generate_next_value_': + if isinstance(value, staticmethod): + value = value.__get__(None, self) + self._generate_next_value_ = value + self._locked = False + elif key == '_ignore_': + if isinstance(value, str): + value = value.split() + else: + value = list(value) + self._ignore = value + already = set(value) & set(self._member_names) + if already: + raise ValueError( + '_ignore_ cannot specify already set names: %r' + % (already, )) + elif key == '_start_': + self._start = value + self._locked = False elif _is_dunder(key): - pass + if key == '__order__': + key = '_order_' + if _is_descriptor(value): + self._locked = True elif key in self._member_names: # descriptor overwriting an enum? raise TypeError('Attempted to reuse key: %r' % key) + elif key in self._ignore: + pass elif not _is_descriptor(value): if key in self: # enum overwriting a descriptor? - raise TypeError('Key already defined as: %r' % self[key]) + raise TypeError('%r already defined as: %r' % (key, self[key])) self._member_names.append(key) + if self._generate_next_value_ is not None: + self._last_value = value + else: + # not a new member, turn off the autoassign magic + self._locked = True super().__setitem__(key, value) + # for magic "auto values" an Enum class should specify a `_generate_next_value_` + # method; that method will be used to generate missing values, and is + # implicitly a staticmethod; + # the signature should be `def _generate_next_value_(name, last_value)` + # last_value will be the last value created and/or assigned, or None + _generate_next_value_ = None # Dummy value for Enum as EnumMeta explicitly checks for it, but of course @@ -84,14 +145,31 @@ # This is also why there are checks in EnumMeta like `if Enum is not None` Enum = None - +_ignore_sentinel = object() class EnumMeta(type): """Metaclass for Enum""" @classmethod - def __prepare__(metacls, cls, bases): - return _EnumDict() + def __prepare__(metacls, cls, bases, start=None, ignore=_ignore_sentinel): + # create the namespace dict + enum_dict = _EnumDict() + # inherit previous flags and _generate_next_value_ function + member_type, first_enum = metacls._get_mixins_(bases) + if first_enum is not None: + enum_dict['_generate_next_value_'] = getattr(first_enum, '_generate_next_value_', None) + if start is None: + start = getattr(first_enum, '_start_', None) + if ignore is _ignore_sentinel: + enum_dict['_ignore_'] = 'property classmethod staticmethod'.split() + elif ignore: + enum_dict['_ignore_'] = ignore + if start is not None: + enum_dict['_start_'] = start + return enum_dict - def __new__(metacls, cls, bases, classdict): + def __init__(cls, *args , **kwds): + super(EnumMeta, cls).__init__(*args) + + def __new__(metacls, cls, bases, classdict, **kwds): # an Enum class is final once enumeration items have been defined; it # cannot be mixed with other types (int, float, etc.) if it has an # inherited __new__ unless a new __new__ is defined (or the resulting @@ -102,12 +180,24 @@ # save enum items into separate mapping so they don't get baked into # the new class - members = {k: classdict[k] for k in classdict._member_names} + enum_members = {k: classdict[k] for k in classdict._member_names} for name in classdict._member_names: del classdict[name] + # adjust the sunders + _order_ = classdict.pop('_order_', None) + classdict.pop('_ignore_', None) + + # py3 support for definition order (helps keep py2/py3 code in sync) + if _order_ is not None: + if isinstance(_order_, str): + _order_ = _order_.replace(',', ' ').split() + unique_members = [n for n in clsdict._member_names if n in _order_] + if _order_ != unique_members: + raise TypeError('member order does not match _order_') + # check for illegal enum names (any others?) - invalid_names = set(members) & {'mro', } + invalid_names = set(enum_members) & {'mro', } if invalid_names: raise ValueError('Invalid enum member name: {0}'.format( ','.join(invalid_names))) @@ -151,7 +241,7 @@ # a custom __new__ is doing something funky with the values -- such as # auto-numbering ;) for member_name in classdict._member_names: - value = members[member_name] + value = enum_members[member_name] if not isinstance(value, tuple): args = (value, ) else: @@ -165,7 +255,10 @@ else: enum_member = __new__(enum_class, *args) if not hasattr(enum_member, '_value_'): - enum_member._value_ = member_type(*args) + if member_type is object: + enum_member._value_ = value + else: + enum_member._value_ = member_type(*args) value = enum_member._value_ enum_member._name_ = member_name enum_member.__objclass__ = enum_class @@ -572,6 +665,22 @@ def _reduce_ex_by_name(self, proto): return self.name +class AutoEnum(Enum): + """Enum where values are automatically assigned.""" + def _generate_next_value_(name, start, count, last_value): + """ + Generate the next value when not given. + + name: the name of the member + start: the initital start value or None + count: the number of existing members + last_value: the last value assigned or None + """ + # add one to the last assigned value + if not count: + return start if start is not None else 1 + return last_value + 1 + def unique(enumeration): """Class decorator for enumerations ensuring unique member values.""" duplicates = [] diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -3,7 +3,7 @@ import pydoc import unittest from collections import OrderedDict -from enum import Enum, IntEnum, EnumMeta, unique +from enum import EnumMeta, Enum, IntEnum, AutoEnum, unique from io import StringIO from pickle import dumps, loads, PicklingError, HIGHEST_PROTOCOL from test import support @@ -1570,6 +1570,328 @@ self.assertEqual(LabelledList.unprocessed, 1) self.assertEqual(LabelledList(1), LabelledList.unprocessed) + def test_ignore_as_str(self): + from datetime import timedelta + class Period(Enum, ignore='Period i'): + """ + different lengths of time + """ + def __new__(cls, value, period): + obj = object.__new__(cls) + obj._value_ = value + obj.period = period + return obj + Period = vars() + for i in range(367): + Period['Day%d' % i] = timedelta(days=i), 'day' + for i in range(53): + Period['Week%d' % i] = timedelta(days=i*7), 'week' + for i in range(13): + Period['Month%d' % i] = i, 'month' + OneDay = Day1 + OneWeek = Week1 + self.assertEqual(Period.Day7.value, timedelta(days=7)) + self.assertEqual(Period.Day7.period, 'day') + + def test_ignore_as_list(self): + from datetime import timedelta + class Period(Enum, ignore=['Period', 'i']): + """ + different lengths of time + """ + def __new__(cls, value, period): + obj = object.__new__(cls) + obj._value_ = value + obj.period = period + return obj + Period = vars() + for i in range(367): + Period['Day%d' % i] = timedelta(days=i), 'day' + for i in range(53): + Period['Week%d' % i] = timedelta(days=i*7), 'week' + for i in range(13): + Period['Month%d' % i] = i, 'month' + OneDay = Day1 + OneWeek = Week1 + self.assertEqual(Period.Day7.value, timedelta(days=7)) + self.assertEqual(Period.Day7.period, 'day') + + def test_new_with_no_value_and_int_base_class(self): + class NoValue(int, Enum): + def __new__(cls, value): + obj = int.__new__(cls, value) + obj.index = len(cls.__members__) + return obj + this = 1 + that = 2 + self.assertEqual(list(NoValue), [NoValue.this, NoValue.that]) + self.assertEqual(NoValue.this, 1) + self.assertEqual(NoValue.this.value, 1) + self.assertEqual(NoValue.this.index, 0) + self.assertEqual(NoValue.that, 2) + self.assertEqual(NoValue.that.value, 2) + self.assertEqual(NoValue.that.index, 1) + + def test_new_with_no_value(self): + class NoValue(Enum): + def __new__(cls, value): + obj = object.__new__(cls) + obj.index = len(cls.__members__) + return obj + this = 1 + that = 2 + self.assertEqual(list(NoValue), [NoValue.this, NoValue.that]) + self.assertEqual(NoValue.this.value, 1) + self.assertEqual(NoValue.this.index, 0) + self.assertEqual(NoValue.that.value, 2) + self.assertEqual(NoValue.that.index, 1) + + +class TestAutoNumber(unittest.TestCase): + + def test_autonumbering(self): + class Color(AutoEnum): + red + green + blue + self.assertEqual(list(Color), [Color.red, Color.green, Color.blue]) + self.assertEqual(Color.red.value, 1) + self.assertEqual(Color.green.value, 2) + self.assertEqual(Color.blue.value, 3) + + def test_autointnumbering(self): + class Color(int, AutoEnum): + red + green + blue + self.assertTrue(isinstance(Color.red, int)) + self.assertEqual(Color.green, 2) + self.assertTrue(Color.blue > Color.red) + + def test_autonumbering_with_start(self): + class Color(AutoEnum, start=7): + red + green + blue + self.assertEqual(list(Color), [Color.red, Color.green, Color.blue]) + self.assertEqual(Color.red.value, 7) + self.assertEqual(Color.green.value, 8) + self.assertEqual(Color.blue.value, 9) + + def test_autonumbering_with_start_and_skip(self): + class Color(AutoEnum, start=7): + red + green + blue = 11 + brown + self.assertEqual(list(Color), [Color.red, Color.green, Color.blue, Color.brown]) + self.assertEqual(Color.red.value, 7) + self.assertEqual(Color.green.value, 8) + self.assertEqual(Color.blue.value, 11) + self.assertEqual(Color.brown.value, 12) + + + def test_badly_overridden_ignore(self): + with self.assertRaisesRegex(TypeError, "'int' object is not callable"): + class Color(AutoEnum): + _ignore_ = () + red + green + blue + @property + def whatever(self): + pass + with self.assertRaisesRegex(TypeError, "'int' object is not callable"): + class Color(AutoEnum, ignore=None): + red + green + blue + @property + def whatever(self): + pass + with self.assertRaisesRegex(TypeError, "'int' object is not callable"): + class Color(AutoEnum, ignore='classmethod staticmethod'): + red + green + blue + @property + def whatever(self): + pass + + def test_property(self): + class Color(AutoEnum): + red + green + blue + @property + def cap_name(self): + return self.name.title() + self.assertEqual(Color.blue.cap_name, 'Blue') + + def test_magic_turns_off(self): + with self.assertRaisesRegex(NameError, "brown"): + class Color(AutoEnum): + red + green + blue + @property + def cap_name(self): + return self.name.title() + brown + + with self.assertRaisesRegex(NameError, "rose"): + class Color(AutoEnum): + red + green + blue + def hello(self): + print('Hello! My serial is %s.' % self.value) + rose + + with self.assertRaisesRegex(NameError, "cyan"): + class Color(AutoEnum): + red + green + blue + def __init__(self, *args): + pass + cyan + + +class TestGenerateMethod(unittest.TestCase): + + def test_autonaming(self): + class Color(Enum): + def _generate_next_value_(name, start, count, last_value): + return name + Red + Green + Blue + self.assertEqual(list(Color), [Color.Red, Color.Green, Color.Blue]) + self.assertEqual(Color.Red.value, 'Red') + self.assertEqual(Color.Green.value, 'Green') + self.assertEqual(Color.Blue.value, 'Blue') + + def test_autonamestr(self): + class Color(str, Enum): + def _generate_next_value_(name, start, count, last_value): + return name + Red + Green + Blue + self.assertTrue(isinstance(Color.Red, str)) + self.assertEqual(Color.Green, 'Green') + self.assertTrue(Color.Blue < Color.Red) + + def test_generate_as_staticmethod(self): + class Color(str, Enum): + @staticmethod + def _generate_next_value_(name, start, count, last_value): + return name.lower() + Red + Green + Blue + self.assertTrue(isinstance(Color.Red, str)) + self.assertEqual(Color.Green, 'green') + self.assertTrue(Color.Blue < Color.Red) + + + def test_overridden_ignore(self): + with self.assertRaisesRegex(TypeError, "'str' object is not callable"): + class Color(Enum): + def _generate_next_value_(name, start, count, last_value): + return name + _ignore_ = () + red + green + blue + @property + def whatever(self): + pass + with self.assertRaisesRegex(TypeError, "'str' object is not callable"): + class Color(Enum, ignore=None): + def _generate_next_value_(name, start, count, last_value): + return name + red + green + blue + @property + def whatever(self): + pass + + def test_property(self): + class Color(Enum): + def _generate_next_value_(name, start, count, last_value): + return name + red + green + blue + @property + def upper_name(self): + return self.name.upper() + self.assertEqual(Color.blue.upper_name, 'BLUE') + + def test_magic_turns_off(self): + with self.assertRaisesRegex(NameError, "brown"): + class Color(Enum): + def _generate_next_value_(name, start, count, last_value): + return name + red + green + blue + @property + def cap_name(self): + return self.name.title() + brown + + with self.assertRaisesRegex(NameError, "rose"): + class Color(Enum): + def _generate_next_value_(name, start, count, last_value): + return name + red + green + blue + def hello(self): + print('Hello! My value %s.' % self.value) + rose + + with self.assertRaisesRegex(NameError, "cyan"): + class Color(Enum): + def _generate_next_value_(name, start, count, last_value): + return name + red + green + blue + def __init__(self, *args): + pass + cyan + + def test_powers_of_two(self): + class Bits(Enum): + def _generate_next_value_(name, start, count, last_value): + return 2 ** count + one + two + four + eight + self.assertEqual(Bits.one.value, 1) + self.assertEqual(Bits.two.value, 2) + self.assertEqual(Bits.four.value, 4) + self.assertEqual(Bits.eight.value, 8) + + def test_powers_of_two_as_int(self): + class Bits(int, Enum): + def _generate_next_value_(name, start, count, last_value): + return 2 ** count + one + two + four + eight + self.assertEqual(Bits.one, 1) + self.assertEqual(Bits.two, 2) + self.assertEqual(Bits.four, 4) + self.assertEqual(Bits.eight, 8) + class TestUnique(unittest.TestCase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -78,6 +78,8 @@ - Issue 27512: Fix a segfault when os.fspath() called a an __fspath__() method that raised an exception. Patch by Xiang Zhang. +- Issue 26988: Add AutoEnum. + Tests ----- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 5 19:59:29 2016 From: python-checkins at python.org (berker.peksag) Date: Fri, 05 Aug 2016 23:59:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2318548=3A_Merge_from_3=2E5?= Message-ID: <20160805235929.20122.12060.98D5A5FB@psf.io> https://hg.python.org/cpython/rev/e17e3f620709 changeset: 102551:e17e3f620709 parent: 102549:7ed7d7f58fcd parent: 102550:d0402caa0ef5 user: Berker Peksag date: Sat Aug 06 03:00:30 2016 +0300 summary: Issue #18548: Merge from 3.5 files: Doc/library/unittest.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -352,7 +352,7 @@ import unittest - class SimpleWidgetTestCase(unittest.TestCase): + class WidgetTestCase(unittest.TestCase): def setUp(self): self.widget = Widget('The widget') @@ -379,7 +379,7 @@ import unittest - class SimpleWidgetTestCase(unittest.TestCase): + class WidgetTestCase(unittest.TestCase): def setUp(self): self.widget = Widget('The widget') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 5 19:59:29 2016 From: python-checkins at python.org (berker.peksag) Date: Fri, 05 Aug 2016 23:59:29 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzE4NTQ4?= =?utf-8?q?=3A_Fix_unittest=2ETestSuite=28=29_example?= Message-ID: <20160805235929.108297.34562.41B95A66@psf.io> https://hg.python.org/cpython/rev/d0402caa0ef5 changeset: 102550:d0402caa0ef5 branch: 3.5 parent: 102546:24b114d77ec8 user: Berker Peksag date: Sat Aug 06 03:00:03 2016 +0300 summary: Issue #18548: Fix unittest.TestSuite() example Initial patch by py.user. files: Doc/library/unittest.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -352,7 +352,7 @@ import unittest - class SimpleWidgetTestCase(unittest.TestCase): + class WidgetTestCase(unittest.TestCase): def setUp(self): self.widget = Widget('The widget') @@ -379,7 +379,7 @@ import unittest - class SimpleWidgetTestCase(unittest.TestCase): + class WidgetTestCase(unittest.TestCase): def setUp(self): self.widget = Widget('The widget') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 6 05:28:46 2016 From: python-checkins at python.org (vinay.sajip) Date: Sat, 06 Aug 2016 09:28:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2327650=3A_Impleme?= =?utf-8?q?nted_repr_methods_for_logging_objects=2E?= Message-ID: <20160806092846.21379.36889.DCF98B33@psf.io> https://hg.python.org/cpython/rev/6c304a2ca139 changeset: 102552:6c304a2ca139 user: Vinay Sajip date: Sat Aug 06 10:28:31 2016 +0100 summary: Closes #27650: Implemented repr methods for logging objects. files: Lib/logging/__init__.py | 27 +++++++++++++++++++++++++++ 1 files changed, 27 insertions(+), 0 deletions(-) diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -935,6 +935,10 @@ finally: del t, v, tb + def __repr__(self): + level = getLevelName(self.level) + return '<%s (%s)>' % (self.__class__.__name__, level) + class StreamHandler(Handler): """ A handler class which writes logging records, appropriately formatted, @@ -986,6 +990,14 @@ except Exception: self.handleError(record) + def __repr__(self): + level = getLevelName(self.level) + name = getattr(self.stream, 'name', '') + if name: + name += ' ' + return '<%s %s(%s)>' % (self.__class__.__name__, name, level) + + class FileHandler(StreamHandler): """ A handler class which writes formatted logging records to disk files. @@ -1050,6 +1062,11 @@ self.stream = self._open() StreamHandler.emit(self, record) + def __repr__(self): + level = getLevelName(self.level) + return '<%s %s (%s)>' % (self.__class__.__name__, self.baseFilename, level) + + class _StderrHandler(StreamHandler): """ This class is like a StreamHandler using sys.stderr, but always uses @@ -1542,6 +1559,11 @@ suffix = '.'.join((self.name, suffix)) return self.manager.getLogger(suffix) + def __repr__(self): + level = getLevelName(self.getEffectiveLevel()) + return '<%s %s (%s)>' % (self.__class__.__name__, self.name, level) + + class RootLogger(Logger): """ A root logger is not that different to any other logger, except that @@ -1668,6 +1690,11 @@ """ return self.logger.hasHandlers() + def __repr__(self): + logger = self.logger + level = getLevelName(logger.getEffectiveLevel()) + return '<%s %s (%s)>' % (self.__class__.__name__, logger.name, level) + root = RootLogger(WARNING) Logger.root = root Logger.manager = Manager(Logger.root) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 6 05:43:58 2016 From: python-checkins at python.org (vinay.sajip) Date: Sat, 06 Aug 2016 09:43:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2322829=3A_Added_-?= =?utf-8?q?-prompt_option_to_venv=2E?= Message-ID: <20160806094358.30895.81835.142FCD7E@psf.io> https://hg.python.org/cpython/rev/6703847eeb31 changeset: 102553:6703847eeb31 user: Vinay Sajip date: Sat Aug 06 10:43:44 2016 +0100 summary: Closes #22829: Added --prompt option to venv. files: Doc/library/venv.rst | 10 +++++++++- Doc/whatsnew/3.6.rst | 8 ++++++++ Lib/test/test_venv.py | 11 +++++++++++ Lib/venv/__init__.py | 18 +++++++++++++----- 4 files changed, 41 insertions(+), 6 deletions(-) diff --git a/Doc/library/venv.rst b/Doc/library/venv.rst --- a/Doc/library/venv.rst +++ b/Doc/library/venv.rst @@ -89,7 +89,8 @@ creation according to their needs, the :class:`EnvBuilder` class. .. class:: EnvBuilder(system_site_packages=False, clear=False, \ - symlinks=False, upgrade=False, with_pip=False) + symlinks=False, upgrade=False, with_pip=False, \ + prompt=None) The :class:`EnvBuilder` class accepts the following keyword arguments on instantiation: @@ -113,9 +114,16 @@ installed in the virtual environment. This uses :mod:`ensurepip` with the ``--default-pip`` option. + * ``prompt`` -- a String to be used after virtual environment is activated + (defaults to ``None`` which means directory name of the environment would + be used). + .. versionchanged:: 3.4 Added the ``with_pip`` parameter + .. versionadded:: 3.6 + Added the ``prompt`` parameter + Creators of third-party virtual environment tools will be free to use the provided ``EnvBuilder`` class as a base class. diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -284,6 +284,14 @@ (Contributed by Brett Cannon in :issue:`25609`.) +venv +---- + +:mod:`venv` accepts a new parameter ``--prompt``. This parameter provides an +alternative prefix for the virtual environment. (Proposed by ?ukasz.Balcerzak +and ported to 3.6 by St?phane Wirtel in :issue:`22829`.) + + datetime -------- diff --git a/Lib/test/test_venv.py b/Lib/test/test_venv.py --- a/Lib/test/test_venv.py +++ b/Lib/test/test_venv.py @@ -114,6 +114,17 @@ print(' %r' % os.listdir(bd)) self.assertTrue(os.path.exists(fn), 'File %r should exist.' % fn) + def test_prompt(self): + env_name = os.path.split(self.env_dir)[1] + + builder = venv.EnvBuilder() + context = builder.ensure_directories(self.env_dir) + self.assertEqual(context.prompt, '(%s) ' % env_name) + + builder = venv.EnvBuilder(prompt='My prompt') + context = builder.ensure_directories(self.env_dir) + self.assertEqual(context.prompt, '(My prompt) ') + @skipInVenv def test_prefixes(self): """ diff --git a/Lib/venv/__init__.py b/Lib/venv/__init__.py --- a/Lib/venv/__init__.py +++ b/Lib/venv/__init__.py @@ -36,15 +36,17 @@ :param upgrade: If True, upgrade an existing virtual environment. :param with_pip: If True, ensure pip is installed in the virtual environment + :param prompt: Alternative terminal prefix for the environment. """ def __init__(self, system_site_packages=False, clear=False, - symlinks=False, upgrade=False, with_pip=False): + symlinks=False, upgrade=False, with_pip=False, prompt=None): self.system_site_packages = system_site_packages self.clear = clear self.symlinks = symlinks self.upgrade = upgrade self.with_pip = with_pip + self.prompt = prompt def create(self, env_dir): """ @@ -90,7 +92,8 @@ context = types.SimpleNamespace() context.env_dir = env_dir context.env_name = os.path.split(env_dir)[1] - context.prompt = '(%s) ' % context.env_name + prompt = self.prompt if self.prompt is not None else context.env_name + context.prompt = '(%s) ' % prompt create_if_needed(env_dir) env = os.environ if sys.platform == 'darwin' and '__PYVENV_LAUNCHER__' in env: @@ -326,10 +329,11 @@ def create(env_dir, system_site_packages=False, clear=False, - symlinks=False, with_pip=False): + symlinks=False, with_pip=False, prompt=None): """Create a virtual environment in a directory.""" builder = EnvBuilder(system_site_packages=system_site_packages, - clear=clear, symlinks=symlinks, with_pip=with_pip) + clear=clear, symlinks=symlinks, with_pip=with_pip, + prompt=prompt) builder.create(env_dir) def main(args=None): @@ -389,6 +393,9 @@ help='Skips installing or upgrading pip in the ' 'virtual environment (pip is bootstrapped ' 'by default)') + parser.add_argument('--prompt', + help='Provides an alternative prompt prefix for ' + 'this environment.') options = parser.parse_args(args) if options.upgrade and options.clear: raise ValueError('you cannot supply --upgrade and --clear together.') @@ -396,7 +403,8 @@ clear=options.clear, symlinks=options.symlinks, upgrade=options.upgrade, - with_pip=options.with_pip) + with_pip=options.with_pip, + prompt=options.prompt) for d in options.dirs: builder.create(d) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 6 06:36:07 2016 From: python-checkins at python.org (berker.peksag) Date: Sat, 06 Aug 2016 10:36:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Silence_warnings_from_=27m?= =?utf-8?q?ake_suspicious=27_to_make_the_docs_buildbot_happy?= Message-ID: <20160806103607.10875.88111.95AD90C4@psf.io> https://hg.python.org/cpython/rev/8c04300acdf0 changeset: 102554:8c04300acdf0 user: Berker Peksag date: Sat Aug 06 13:37:22 2016 +0300 summary: Silence warnings from 'make suspicious' to make the docs buildbot happy files: Doc/library/enum.rst | 16 +++++++++------- 1 files changed, 9 insertions(+), 7 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -837,16 +837,18 @@ Enum class signature ~~~~~~~~~~~~~~~~~~~~ - ``class SomeName( - AnEnum, - start=None, - ignore='staticmethod classmethod property', - ):`` +:: -``start`` can be used by a :meth:`_generate_next_value_` method to specify a + class SomeName( + AnEnum, + start=None, + ignore='staticmethod classmethod property', + ): + +*start* can be used by a :meth:`_generate_next_value_` method to specify a starting value. -``ignore`` specifies which names, if any, will not attempt to auto-generate +*ignore* specifies which names, if any, will not attempt to auto-generate a new value (they will also be removed from the class body). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 6 16:22:50 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 06 Aug 2016 20:22:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326800=3A_Undocume?= =?utf-8?q?nted_support_of_general_bytes-like_objects?= Message-ID: <20160806202250.21021.9254.826B4D29@psf.io> https://hg.python.org/cpython/rev/1e01ca34a42c changeset: 102555:1e01ca34a42c user: Serhiy Storchaka date: Sat Aug 06 23:22:08 2016 +0300 summary: Issue #26800: Undocumented support of general bytes-like objects as paths in os functions is now deprecated. files: Doc/whatsnew/3.6.rst | 5 ++++ Lib/test/test_os.py | 9 +++++++- Lib/test/test_posix.py | 4 ++- Misc/NEWS | 3 ++ Modules/posixmodule.c | 31 ++++++++++++++++++++++++++++- 5 files changed, 48 insertions(+), 4 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -638,6 +638,11 @@ and will be removed in 3.8. (Contributed by Serhiy Storchaka in :issue:`21708`.) +* Undocumented support of general :term:`bytes-like objects ` + as paths in :mod:`os` functions is now deprecated. + (Contributed by Serhiy Storchaka in :issue:`25791`.) + + Deprecated Python behavior -------------------------- diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -2626,6 +2626,7 @@ else: encoded = os.fsencode(support.TESTFN) self.bytes_filenames.append(encoded) + self.bytes_filenames.append(bytearray(encoded)) self.bytes_filenames.append(memoryview(encoded)) self.filenames = self.bytes_filenames + self.unicode_filenames @@ -2699,8 +2700,14 @@ for filenames, func, *func_args in funcs: for name in filenames: try: - with bytes_filename_warn(False): + if isinstance(name, str): func(name, *func_args) + elif isinstance(name, bytes): + with bytes_filename_warn(False): + func(name, *func_args) + else: + with self.assertWarnsRegex(DeprecationWarning, 'should be'): + func(name, *func_args) except OSError as err: self.assertIs(err.filename, name) else: diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -407,8 +407,10 @@ def test_stat(self): self.assertTrue(posix.stat(support.TESTFN)) self.assertTrue(posix.stat(os.fsencode(support.TESTFN))) - self.assertTrue(posix.stat(bytearray(os.fsencode(support.TESTFN)))) + self.assertWarnsRegex(DeprecationWarning, + 'should be string, bytes or integer, not', + posix.stat, bytearray(os.fsencode(support.TESTFN))) self.assertRaisesRegex(TypeError, 'should be string, bytes or integer, not', posix.stat, None) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -43,6 +43,9 @@ Library ------- +- Issue #26800: Undocumented support of general bytes-like objects + as paths in os functions is now deprecated. + - Issue #27661: Added tzinfo keyword argument to datetime.combine. - Issue #27568: Prevent HTTPoxy attack (CVE-2016-1000110). Ignore the diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -891,7 +891,28 @@ } #endif } + else if (PyBytes_Check(o)) { +#ifdef MS_WINDOWS + if (win32_warn_bytes_api()) { + return 0; + } +#endif + bytes = o; + Py_INCREF(bytes); + } else if (PyObject_CheckBuffer(o)) { + if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1, + "%s%s%s should be %s, not %.200s", + path->function_name ? path->function_name : "", + path->function_name ? ": " : "", + path->argument_name ? path->argument_name : "path", + path->allow_fd && path->nullable ? "string, bytes, integer or None" : + path->allow_fd ? "string, bytes or integer" : + path->nullable ? "string, bytes or None" : + "string or bytes", + Py_TYPE(o)->tp_name)) { + return 0; + } #ifdef MS_WINDOWS if (win32_warn_bytes_api()) { return 0; @@ -946,8 +967,14 @@ path->length = length; path->object = o; path->fd = -1; - path->cleanup = bytes; - return Py_CLEANUP_SUPPORTED; + if (bytes == o) { + Py_DECREF(bytes); + return 1; + } + else { + path->cleanup = bytes; + return Py_CLEANUP_SUPPORTED; + } } static void -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 6 16:29:47 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 06 Aug 2016 20:29:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326754=3A_Undocume?= =?utf-8?q?nted_support_of_general_bytes-like_objects?= Message-ID: <20160806202946.71800.75741.DF61DE4C@psf.io> https://hg.python.org/cpython/rev/818f22f9ab02 changeset: 102556:818f22f9ab02 user: Serhiy Storchaka date: Sat Aug 06 23:29:29 2016 +0300 summary: Issue #26754: Undocumented support of general bytes-like objects as path in compile() and similar functions is now deprecated. files: Doc/whatsnew/3.6.rst | 5 +++-- Lib/test/test_compile.py | 7 +++++-- Lib/test/test_parser.py | 10 ++++++++-- Lib/test/test_symtable.py | 6 ++++-- Lib/test/test_zipimport.py | 6 ++++-- Misc/NEWS | 3 +++ Objects/unicodeobject.c | 13 +++++++------ 7 files changed, 34 insertions(+), 16 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -639,8 +639,9 @@ (Contributed by Serhiy Storchaka in :issue:`21708`.) * Undocumented support of general :term:`bytes-like objects ` - as paths in :mod:`os` functions is now deprecated. - (Contributed by Serhiy Storchaka in :issue:`25791`.) + as paths in :mod:`os` functions, :func:`compile` and similar functions is + now deprecated. + (Contributed by Serhiy Storchaka in :issue:`25791` and :issue:`26754`.) Deprecated Python behavior diff --git a/Lib/test/test_compile.py b/Lib/test/test_compile.py --- a/Lib/test/test_compile.py +++ b/Lib/test/test_compile.py @@ -473,10 +473,13 @@ self.assertEqual(d, {1: 2, 3: 4}) def test_compile_filename(self): - for filename in ('file.py', b'file.py', - bytearray(b'file.py'), memoryview(b'file.py')): + for filename in 'file.py', b'file.py': code = compile('pass', filename, 'exec') self.assertEqual(code.co_filename, 'file.py') + for filename in bytearray(b'file.py'), memoryview(b'file.py'): + with self.assertWarns(DeprecationWarning): + code = compile('pass', filename, 'exec') + self.assertEqual(code.co_filename, 'file.py') self.assertRaises(TypeError, compile, 'pass', list(b'file.py'), 'exec') @support.cpython_only diff --git a/Lib/test/test_parser.py b/Lib/test/test_parser.py --- a/Lib/test/test_parser.py +++ b/Lib/test/test_parser.py @@ -632,12 +632,18 @@ self.assertEqual(code.co_filename, '') code = st.compile() self.assertEqual(code.co_filename, '') - for filename in ('file.py', b'file.py', - bytearray(b'file.py'), memoryview(b'file.py')): + for filename in 'file.py', b'file.py': code = parser.compilest(st, filename) self.assertEqual(code.co_filename, 'file.py') code = st.compile(filename) self.assertEqual(code.co_filename, 'file.py') + for filename in bytearray(b'file.py'), memoryview(b'file.py'): + with self.assertWarns(DeprecationWarning): + code = parser.compilest(st, filename) + self.assertEqual(code.co_filename, 'file.py') + with self.assertWarns(DeprecationWarning): + code = st.compile(filename) + self.assertEqual(code.co_filename, 'file.py') self.assertRaises(TypeError, parser.compilest, st, list(b'file.py')) self.assertRaises(TypeError, st.compile, list(b'file.py')) diff --git a/Lib/test/test_symtable.py b/Lib/test/test_symtable.py --- a/Lib/test/test_symtable.py +++ b/Lib/test/test_symtable.py @@ -158,9 +158,11 @@ checkfilename("def f(x): foo)(") # parse-time checkfilename("def f(x): global x") # symtable-build-time symtable.symtable("pass", b"spam", "exec") - with self.assertRaises(TypeError): + with self.assertWarns(DeprecationWarning), \ + self.assertRaises(TypeError): symtable.symtable("pass", bytearray(b"spam"), "exec") - symtable.symtable("pass", memoryview(b"spam"), "exec") + with self.assertWarns(DeprecationWarning): + symtable.symtable("pass", memoryview(b"spam"), "exec") with self.assertRaises(TypeError): symtable.symtable("pass", list(b"spam"), "exec") diff --git a/Lib/test/test_zipimport.py b/Lib/test/test_zipimport.py --- a/Lib/test/test_zipimport.py +++ b/Lib/test/test_zipimport.py @@ -629,8 +629,10 @@ zipimport.zipimporter(filename) zipimport.zipimporter(os.fsencode(filename)) - zipimport.zipimporter(bytearray(os.fsencode(filename))) - zipimport.zipimporter(memoryview(os.fsencode(filename))) + with self.assertWarns(DeprecationWarning): + zipimport.zipimporter(bytearray(os.fsencode(filename))) + with self.assertWarns(DeprecationWarning): + zipimport.zipimporter(memoryview(os.fsencode(filename))) @support.requires_zlib diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -43,6 +43,9 @@ Library ------- +- Issue #26754: Undocumented support of general bytes-like objects + as path in compile() and similar functions is now deprecated. + - Issue #26800: Undocumented support of general bytes-like objects as paths in os functions is now deprecated. diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -3837,7 +3837,13 @@ output = arg; Py_INCREF(output); } - else if (PyObject_CheckBuffer(arg)) { + else if (PyBytes_Check(arg) || PyObject_CheckBuffer(arg)) { + if (!PyBytes_Check(arg) && + PyErr_WarnFormat(PyExc_DeprecationWarning, 1, + "path should be string or bytes, not %.200s", + Py_TYPE(arg)->tp_name)) { + return 0; + } arg = PyBytes_FromObject(arg); if (!arg) return 0; @@ -3846,11 +3852,6 @@ Py_DECREF(arg); if (!output) return 0; - if (!PyUnicode_Check(output)) { - Py_DECREF(output); - PyErr_SetString(PyExc_TypeError, "decoder failed to return unicode"); - return 0; - } } else { PyErr_Format(PyExc_TypeError, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 6 16:48:22 2016 From: python-checkins at python.org (guido.van.rossum) Date: Sat, 06 Aug 2016 20:48:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Better_docs_fo?= =?utf-8?q?r_typing=2EAny_by_Michael_Lee=2E_Fixes_issue_=2327688=2E?= Message-ID: <20160806204822.11208.36203.3F8656D5@psf.io> https://hg.python.org/cpython/rev/cbcab1fb1b8a changeset: 102557:cbcab1fb1b8a branch: 3.5 parent: 102550:d0402caa0ef5 user: Guido van Rossum date: Sat Aug 06 13:46:48 2016 -0700 summary: Better docs for typing.Any by Michael Lee. Fixes issue #27688. files: Doc/library/typing.rst | 80 ++++++++++++++++++++++++++--- 1 files changed, 71 insertions(+), 9 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -279,17 +279,79 @@ The :class:`Any` type --------------------- -A special kind of type is :class:`Any`. Every type is a subtype of -:class:`Any`. This is also true for the builtin type object. However, to the -static type checker these are completely different. +A special kind of type is :class:`Any`. A static type checker will treat +every type as being compatible with :class:`Any` and :class:`Any` as being +compatible with every type. -When the type of a value is :class:`object`, the type checker will reject -almost all operations on it, and assigning it to a variable (or using it as a -return value) of a more specialized type is a type error. On the other hand, -when a value has type :class:`Any`, the type checker will allow all operations -on it, and a value of type :class:`Any` can be assigned to a variable (or used -as a return value) of a more constrained type. +This means that it is possible to perform any operation or method call on a +value of type on :class:`Any` and assign it to any variable:: + from typing import Any + + a = None # type: Any + a = [] # OK + a = 2 # OK + + s = '' # type: str + s = a # OK + + def foo(item: Any) -> int: + # Typechecks; `item` could be any type, + # and that type might have a `bar` method + item.bar() + ... + +Notice that no typechecking is performed when assigning a value of type +:class:`Any` to a more precise type. For example, the static type checker did +not report an error when assigning ``a`` to ``s`` even though ``s`` was +declared to be of type :class:`str` and receives an :class:`int` value at +runtime! + +Furthermore, all functions without a return type or parameter types will +implicitly default to using :class:`Any`:: + + def legacy_parser(text): + ... + return data + + # A static type checker will treat the above + # as having the same signature as: + def legacy_parser(text: Any) -> Any: + ... + return data + +This behavior allows :class:`Any` to be used as an *escape hatch* when you +need to mix dynamically and statically typed code. + +Contrast the behavior of :class:`Any` with the behavior of :class:`object`. +Similar to :class:`Any`, every type is a subtype of :class:`object`. However, +unlike :class:`Any`, the reverse is not true: :class:`object` is *not* a +subtype of every other type. + +That means when the type of a value is :class:`object`, a type checker will +reject almost all operations on it, and assigning it to a variable (or using +it as a return value) of a more specialized type is a type error. For example:: + + def hash_a(item: object) -> int: + # Fails; an object does not have a `magic` method. + item.magic() + ... + + def hash_b(item: Any) -> int: + # Typechecks + item.magic() + ... + + # Typechecks, since ints and strs are subclasses of object + hash_a(42) + hash_a("foo") + + # Typechecks, since Any is compatible with all types + hash_b(42) + hash_b("foo") + +Use :class:`object` to indicate that a value could be any type in a typesafe +manner. Use :class:`Any` to indicate that a value is dynamically typed. Classes, functions, and decorators ---------------------------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 6 16:48:22 2016 From: python-checkins at python.org (guido.van.rossum) Date: Sat, 06 Aug 2016 20:48:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Better_docs_for_typing=2EAny_by_Michael_Lee=2E_Fixes_iss?= =?utf-8?b?dWUgIzI3Njg4LiAoTWVyZ2UgMy41LT4zLjYp?= Message-ID: <20160806204822.26668.57390.0B5CC98B@psf.io> https://hg.python.org/cpython/rev/f60e16663f6e changeset: 102558:f60e16663f6e parent: 102556:818f22f9ab02 parent: 102557:cbcab1fb1b8a user: Guido van Rossum date: Sat Aug 06 13:48:10 2016 -0700 summary: Better docs for typing.Any by Michael Lee. Fixes issue #27688. (Merge 3.5->3.6) files: Doc/library/typing.rst | 80 ++++++++++++++++++++++++++--- 1 files changed, 71 insertions(+), 9 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -279,17 +279,79 @@ The :class:`Any` type --------------------- -A special kind of type is :class:`Any`. Every type is a subtype of -:class:`Any`. This is also true for the builtin type object. However, to the -static type checker these are completely different. +A special kind of type is :class:`Any`. A static type checker will treat +every type as being compatible with :class:`Any` and :class:`Any` as being +compatible with every type. -When the type of a value is :class:`object`, the type checker will reject -almost all operations on it, and assigning it to a variable (or using it as a -return value) of a more specialized type is a type error. On the other hand, -when a value has type :class:`Any`, the type checker will allow all operations -on it, and a value of type :class:`Any` can be assigned to a variable (or used -as a return value) of a more constrained type. +This means that it is possible to perform any operation or method call on a +value of type on :class:`Any` and assign it to any variable:: + from typing import Any + + a = None # type: Any + a = [] # OK + a = 2 # OK + + s = '' # type: str + s = a # OK + + def foo(item: Any) -> int: + # Typechecks; `item` could be any type, + # and that type might have a `bar` method + item.bar() + ... + +Notice that no typechecking is performed when assigning a value of type +:class:`Any` to a more precise type. For example, the static type checker did +not report an error when assigning ``a`` to ``s`` even though ``s`` was +declared to be of type :class:`str` and receives an :class:`int` value at +runtime! + +Furthermore, all functions without a return type or parameter types will +implicitly default to using :class:`Any`:: + + def legacy_parser(text): + ... + return data + + # A static type checker will treat the above + # as having the same signature as: + def legacy_parser(text: Any) -> Any: + ... + return data + +This behavior allows :class:`Any` to be used as an *escape hatch* when you +need to mix dynamically and statically typed code. + +Contrast the behavior of :class:`Any` with the behavior of :class:`object`. +Similar to :class:`Any`, every type is a subtype of :class:`object`. However, +unlike :class:`Any`, the reverse is not true: :class:`object` is *not* a +subtype of every other type. + +That means when the type of a value is :class:`object`, a type checker will +reject almost all operations on it, and assigning it to a variable (or using +it as a return value) of a more specialized type is a type error. For example:: + + def hash_a(item: object) -> int: + # Fails; an object does not have a `magic` method. + item.magic() + ... + + def hash_b(item: Any) -> int: + # Typechecks + item.magic() + ... + + # Typechecks, since ints and strs are subclasses of object + hash_a(42) + hash_a("foo") + + # Typechecks, since Any is compatible with all types + hash_b(42) + hash_b("foo") + +Use :class:`object` to indicate that a value could be any type in a typesafe +manner. Use :class:`Any` to indicate that a value is dynamically typed. Classes, functions, and decorators ---------------------------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 7 12:06:38 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sun, 07 Aug 2016 16:06:38 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2NzUw?= =?utf-8?q?=3A_unittest=2Emock=2Ecreate=5Fautospec=28=29_now_works_properl?= =?utf-8?q?y?= Message-ID: <20160807160638.79748.68932.D80528BA@psf.io> https://hg.python.org/cpython/rev/e947248100ae changeset: 102559:e947248100ae branch: 3.5 parent: 102557:cbcab1fb1b8a user: Gregory P. Smith date: Sun Aug 07 08:52:26 2016 -0700 summary: Issue #26750: unittest.mock.create_autospec() now works properly for subclasses of property() and other data descriptors. files: Lib/unittest/mock.py | 10 +- Lib/unittest/test/testmock/testhelpers.py | 60 +++++++--- Misc/NEWS | 3 + 3 files changed, 51 insertions(+), 22 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -64,12 +64,20 @@ __slots__ = ['a'] +# Do not use this tuple. It was never documented as a public API. +# It will be removed. It has no obvious signs of users on github. DescriptorTypes = ( type(_slotted.a), property, ) +def _is_data_descriptor(obj): + # Data descriptors are Properties, slots, getsets and C data members. + return ((hasattr(obj, '__set__') or hasattr(obj, '__del__')) and + hasattr(obj, '__get__')) + + def _get_signature_object(func, as_instance, eat_self): """ Given an arbitrary, possibly callable object, try to create a suitable @@ -2130,7 +2138,7 @@ _kwargs.update(kwargs) Klass = MagicMock - if type(spec) in DescriptorTypes: + if _is_data_descriptor(spec): # descriptors don't have a spec # because we don't know what type they return _kwargs = {} diff --git a/Lib/unittest/test/testmock/testhelpers.py b/Lib/unittest/test/testmock/testhelpers.py --- a/Lib/unittest/test/testmock/testhelpers.py +++ b/Lib/unittest/test/testmock/testhelpers.py @@ -802,35 +802,53 @@ a.f.assert_called_with(self=10) - def test_autospec_property(self): + def test_autospec_data_descriptor(self): + class Descriptor(object): + def __init__(self, value): + self.value = value + + def __get__(self, obj, cls=None): + if obj is None: + return self + return self.value + + def __set__(self, obj, value): + pass + + class MyProperty(property): + pass + class Foo(object): + __slots__ = ['slot'] + @property - def foo(self): + def prop(self): return 3 - foo = create_autospec(Foo) - mock_property = foo.foo + @MyProperty + def subprop(self): + return 4 - # no spec on properties - self.assertIsInstance(mock_property, MagicMock) - mock_property(1, 2, 3) - mock_property.abc(4, 5, 6) - mock_property.assert_called_once_with(1, 2, 3) - mock_property.abc.assert_called_once_with(4, 5, 6) - - - def test_autospec_slots(self): - class Foo(object): - __slots__ = ['a'] + desc = Descriptor(42) foo = create_autospec(Foo) - mock_slot = foo.a - # no spec on slots - mock_slot(1, 2, 3) - mock_slot.abc(4, 5, 6) - mock_slot.assert_called_once_with(1, 2, 3) - mock_slot.abc.assert_called_once_with(4, 5, 6) + def check_data_descriptor(mock_attr): + # Data descriptors don't have a spec. + self.assertIsInstance(mock_attr, MagicMock) + mock_attr(1, 2, 3) + mock_attr.abc(4, 5, 6) + mock_attr.assert_called_once_with(1, 2, 3) + mock_attr.abc.assert_called_once_with(4, 5, 6) + + # property + check_data_descriptor(foo.prop) + # property subclass + check_data_descriptor(foo.subprop) + # class __slot__ + check_data_descriptor(foo.slot) + # plain data descriptor + check_data_descriptor(foo.desc) class TestCallList(unittest.TestCase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -34,6 +34,9 @@ Library ------- +- Issue #26750: unittest.mock.create_autospec() now works properly for + subclasses of property() and other data descriptors. + - Issue #27568: Prevent HTTPoxy attack (CVE-2016-1000110). Ignore the HTTP_PROXY variable when REQUEST_METHOD environment is set, which indicates that the script is in CGI mode. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 7 12:06:38 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sun, 07 Aug 2016 16:06:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326750=3A_unittest=2Emock=2Ecreate=5Fautospec=28?= =?utf-8?q?=29_now_works_properly_for?= Message-ID: <20160807160638.108469.3180.FB6F0F78@psf.io> https://hg.python.org/cpython/rev/0bc14c91ef7e changeset: 102560:0bc14c91ef7e parent: 102558:f60e16663f6e parent: 102559:e947248100ae user: Gregory P. Smith date: Sun Aug 07 09:06:27 2016 -0700 summary: Issue #26750: unittest.mock.create_autospec() now works properly for subclasses of property() and other data descriptors. Removes the never publicly used, never documented unittest.mock.DescriptorTypes tuple. files: Lib/unittest/mock.py | 14 +- Lib/unittest/test/testmock/testhelpers.py | 60 +++++++--- Misc/NEWS | 4 + 3 files changed, 48 insertions(+), 30 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -60,14 +60,10 @@ ) -class _slotted(object): - __slots__ = ['a'] - - -DescriptorTypes = ( - type(_slotted.a), - property, -) +def _is_data_descriptor(obj): + # Data descriptors are Properties, slots, getsets and C data members. + return ((hasattr(obj, '__set__') or hasattr(obj, '__del__')) and + hasattr(obj, '__get__')) def _get_signature_object(func, as_instance, eat_self): @@ -2153,7 +2149,7 @@ _kwargs.update(kwargs) Klass = MagicMock - if type(spec) in DescriptorTypes: + if _is_data_descriptor(spec): # descriptors don't have a spec # because we don't know what type they return _kwargs = {} diff --git a/Lib/unittest/test/testmock/testhelpers.py b/Lib/unittest/test/testmock/testhelpers.py --- a/Lib/unittest/test/testmock/testhelpers.py +++ b/Lib/unittest/test/testmock/testhelpers.py @@ -802,35 +802,53 @@ a.f.assert_called_with(self=10) - def test_autospec_property(self): + def test_autospec_data_descriptor(self): + class Descriptor(object): + def __init__(self, value): + self.value = value + + def __get__(self, obj, cls=None): + if obj is None: + return self + return self.value + + def __set__(self, obj, value): + pass + + class MyProperty(property): + pass + class Foo(object): + __slots__ = ['slot'] + @property - def foo(self): + def prop(self): return 3 - foo = create_autospec(Foo) - mock_property = foo.foo + @MyProperty + def subprop(self): + return 4 - # no spec on properties - self.assertIsInstance(mock_property, MagicMock) - mock_property(1, 2, 3) - mock_property.abc(4, 5, 6) - mock_property.assert_called_once_with(1, 2, 3) - mock_property.abc.assert_called_once_with(4, 5, 6) - - - def test_autospec_slots(self): - class Foo(object): - __slots__ = ['a'] + desc = Descriptor(42) foo = create_autospec(Foo) - mock_slot = foo.a - # no spec on slots - mock_slot(1, 2, 3) - mock_slot.abc(4, 5, 6) - mock_slot.assert_called_once_with(1, 2, 3) - mock_slot.abc.assert_called_once_with(4, 5, 6) + def check_data_descriptor(mock_attr): + # Data descriptors don't have a spec. + self.assertIsInstance(mock_attr, MagicMock) + mock_attr(1, 2, 3) + mock_attr.abc(4, 5, 6) + mock_attr.assert_called_once_with(1, 2, 3) + mock_attr.abc.assert_called_once_with(4, 5, 6) + + # property + check_data_descriptor(foo.prop) + # property subclass + check_data_descriptor(foo.subprop) + # class __slot__ + check_data_descriptor(foo.slot) + # plain data descriptor + check_data_descriptor(foo.desc) class TestCallList(unittest.TestCase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -43,6 +43,10 @@ Library ------- +- Issue #26750: unittest.mock.create_autospec() now works properly for + subclasses of property() and other data descriptors. Removes the never + publicly used, never documented unittest.mock.DescriptorTypes tuple. + - Issue #26754: Undocumented support of general bytes-like objects as path in compile() and similar functions is now deprecated. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 7 13:19:28 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sun, 07 Aug 2016 17:19:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327664=3A_Add_to_c?= =?utf-8?q?oncurrent=2Efutures=2Ethread=2EThreadPoolExecutor=28=29?= Message-ID: <20160807171928.79902.45911.E08061DF@psf.io> https://hg.python.org/cpython/rev/1002a1bdc5b1 changeset: 102561:1002a1bdc5b1 user: Gregory P. Smith date: Sun Aug 07 10:19:20 2016 -0700 summary: Issue #27664: Add to concurrent.futures.thread.ThreadPoolExecutor() the ability to specify a thread name prefix. files: Doc/library/concurrent.futures.rst | 6 +++- Lib/concurrent/futures/thread.py | 11 +++++-- Lib/test/test_concurrent_futures.py | 24 +++++++++++++++++ Misc/NEWS | 3 ++ 4 files changed, 40 insertions(+), 4 deletions(-) diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst --- a/Doc/library/concurrent.futures.rst +++ b/Doc/library/concurrent.futures.rst @@ -124,7 +124,7 @@ executor.submit(wait_on_future) -.. class:: ThreadPoolExecutor(max_workers=None) +.. class:: ThreadPoolExecutor(max_workers=None, thread_name_prefix='') An :class:`Executor` subclass that uses a pool of at most *max_workers* threads to execute calls asynchronously. @@ -137,6 +137,10 @@ should be higher than the number of workers for :class:`ProcessPoolExecutor`. + .. versionadded:: 3.6 + The *thread_name_prefix* argument was added to allow users to + control the threading.Thread names for worker threads created by + the pool for easier debugging. .. _threadpoolexecutor-example: diff --git a/Lib/concurrent/futures/thread.py b/Lib/concurrent/futures/thread.py --- a/Lib/concurrent/futures/thread.py +++ b/Lib/concurrent/futures/thread.py @@ -81,12 +81,13 @@ _base.LOGGER.critical('Exception in worker', exc_info=True) class ThreadPoolExecutor(_base.Executor): - def __init__(self, max_workers=None): + def __init__(self, max_workers=None, thread_name_prefix=''): """Initializes a new ThreadPoolExecutor instance. Args: max_workers: The maximum number of threads that can be used to execute the given calls. + thread_name_prefix: An optional name prefix to give our threads. """ if max_workers is None: # Use this number because ThreadPoolExecutor is often @@ -100,6 +101,7 @@ self._threads = set() self._shutdown = False self._shutdown_lock = threading.Lock() + self._thread_name_prefix = thread_name_prefix def submit(self, fn, *args, **kwargs): with self._shutdown_lock: @@ -121,8 +123,11 @@ q.put(None) # TODO(bquinlan): Should avoid creating new threads if there are more # idle threads than items in the work queue. - if len(self._threads) < self._max_workers: - t = threading.Thread(target=_worker, + num_threads = len(self._threads) + if num_threads < self._max_workers: + thread_name = '%s_%d' % (self._thread_name_prefix or self, + num_threads) + t = threading.Thread(name=thread_name, target=_worker, args=(weakref.ref(self, weakref_cb), self._work_queue)) t.daemon = True diff --git a/Lib/test/test_concurrent_futures.py b/Lib/test/test_concurrent_futures.py --- a/Lib/test/test_concurrent_futures.py +++ b/Lib/test/test_concurrent_futures.py @@ -154,6 +154,30 @@ for t in threads: t.join() + def test_thread_names_assigned(self): + executor = futures.ThreadPoolExecutor( + max_workers=5, thread_name_prefix='SpecialPool') + executor.map(abs, range(-5, 5)) + threads = executor._threads + del executor + + for t in threads: + self.assertRegex(t.name, r'^SpecialPool_[0-4]$') + t.join() + + def test_thread_names_default(self): + executor = futures.ThreadPoolExecutor(max_workers=5) + executor.map(abs, range(-5, 5)) + threads = executor._threads + del executor + + for t in threads: + # We don't particularly care what the default name is, just that + # it has a default name implying that it is a ThreadPoolExecutor + # followed by what looks like a thread number. + self.assertRegex(t.name, r'^.*ThreadPoolExecutor.*_[0-4]$') + t.join() + class ProcessPoolShutdownTest(ProcessPoolMixin, ExecutorShutdownTest, unittest.TestCase): def _prime_executor(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -43,6 +43,9 @@ Library ------- +- Issue #27664: Add to concurrent.futures.thread.ThreadPoolExecutor() + the ability to specify a thread name prefix. + - Issue #26750: unittest.mock.create_autospec() now works properly for subclasses of property() and other data descriptors. Removes the never publicly used, never documented unittest.mock.DescriptorTypes tuple. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 7 20:11:14 2016 From: python-checkins at python.org (andrew.svetlov) Date: Mon, 08 Aug 2016 00:11:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20160808001114.26385.36123.CE5F7C94@psf.io> https://hg.python.org/cpython/rev/d86eabe2c457 changeset: 102563:d86eabe2c457 parent: 102561:1002a1bdc5b1 parent: 102562:59c9b99f545e user: Andrew Svetlov date: Mon Aug 08 03:11:06 2016 +0300 summary: Merge 3.5 files: Doc/library/asyncio-task.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -486,7 +486,7 @@ asyncio.ensure_future(factorial("A", 2)), asyncio.ensure_future(factorial("B", 3)), asyncio.ensure_future(factorial("C", 4))] - loop.run_until_complete(asyncio.wait(tasks)) + loop.run_until_complete(asyncio.gather(*tasks)) loop.close() Output:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 7 20:11:14 2016 From: python-checkins at python.org (andrew.svetlov) Date: Mon, 08 Aug 2016 00:11:14 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogVXNlIGFzeW5jaW8u?= =?utf-8?q?gather_instead_of_asyncio=2Ewait_in_example=2E?= Message-ID: <20160808001114.20210.97443.1C32D9F1@psf.io> https://hg.python.org/cpython/rev/59c9b99f545e changeset: 102562:59c9b99f545e branch: 3.5 parent: 102559:e947248100ae user: Andrew Svetlov date: Mon Aug 08 03:10:39 2016 +0300 summary: Use asyncio.gather instead of asyncio.wait in example. gather will raise exception in case of problem but wait just returns done and pending lists of futures. For getting error every future result should be retrieved, which is boring and error prone. files: Doc/library/asyncio-task.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -486,7 +486,7 @@ asyncio.ensure_future(factorial("A", 2)), asyncio.ensure_future(factorial("B", 3)), asyncio.ensure_future(factorial("C", 4))] - loop.run_until_complete(asyncio.wait(tasks)) + loop.run_until_complete(asyncio.gather(*tasks)) loop.close() Output:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 7 23:20:38 2016 From: python-checkins at python.org (raymond.hettinger) Date: Mon, 08 Aug 2016 03:20:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Re-linewrap_comments?= Message-ID: <20160808032038.20510.36657.A0D9FE41@psf.io> https://hg.python.org/cpython/rev/226d6c1485e9 changeset: 102564:226d6c1485e9 user: Raymond Hettinger date: Sun Aug 07 20:20:33 2016 -0700 summary: Re-linewrap comments files: Python/peephole.c | 21 ++++++++------------- 1 files changed, 8 insertions(+), 13 deletions(-) diff --git a/Python/peephole.c b/Python/peephole.c --- a/Python/peephole.c +++ b/Python/peephole.c @@ -638,22 +638,17 @@ j = codestr[tgt]; if (CONDITIONAL_JUMP(j)) { - /* NOTE: all possible jumps here are - absolute! */ + /* NOTE: all possible jumps here are absolute. */ if (JUMPS_ON_TRUE(j) == JUMPS_ON_TRUE(opcode)) { - /* The second jump will be - taken iff the first is. - The current opcode inherits - its target's stack effect */ + /* The second jump will be taken iff the first is. + The current opcode inherits its target's + stack effect */ h = set_arg(codestr, i, get_arg(codestr, tgt)); } else { - /* The second jump is not taken - if the first is (so jump past - it), and all conditional - jumps pop their argument when - they're not taken (so change - the first jump to pop its - argument when it's taken). */ + /* The second jump is not taken if the first is (so + jump past it), and all conditional jumps pop their + argument when they're not taken (so change the + first jump to pop its argument when it's taken). */ h = set_arg(codestr, i, tgt + 2); j = opcode == JUMP_IF_TRUE_OR_POP ? POP_JUMP_IF_TRUE : POP_JUMP_IF_FALSE; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 8 06:06:10 2016 From: python-checkins at python.org (berker.peksag) Date: Mon, 08 Aug 2016 10:06:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2323322=3A_Merge_from_3=2E5?= Message-ID: <20160808100610.11373.59072.796AF79A@psf.io> https://hg.python.org/cpython/rev/099fd7954941 changeset: 102566:099fd7954941 parent: 102564:226d6c1485e9 parent: 102565:b479e74f7706 user: Berker Peksag date: Mon Aug 08 13:07:35 2016 +0300 summary: Issue #23322: Merge from 3.5 files: Doc/library/token.rst | 7 ------- 1 files changed, 0 insertions(+), 7 deletions(-) diff --git a/Doc/library/token.rst b/Doc/library/token.rst --- a/Doc/library/token.rst +++ b/Doc/library/token.rst @@ -108,10 +108,3 @@ Added :data:`AWAIT` and :data:`ASYNC` tokens. Starting with Python 3.7, "async" and "await" will be tokenized as :data:`NAME` tokens, and :data:`AWAIT` and :data:`ASYNC` will be removed. - -.. seealso:: - - Module :mod:`parser` - The second example for the :mod:`parser` module shows how to use the - :mod:`symbol` module. - -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 8 06:06:10 2016 From: python-checkins at python.org (berker.peksag) Date: Mon, 08 Aug 2016 10:06:10 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIzMzIy?= =?utf-8?q?=3A_Remove_outdated_reference_to_an_example_in_parser_docs?= Message-ID: <20160808100610.21340.34748.81BBBFDB@psf.io> https://hg.python.org/cpython/rev/b479e74f7706 changeset: 102565:b479e74f7706 branch: 3.5 parent: 102562:59c9b99f545e user: Berker Peksag date: Mon Aug 08 13:07:08 2016 +0300 summary: Issue #23322: Remove outdated reference to an example in parser docs Initial patch by Sahil Chelaramani. files: Doc/library/token.rst | 7 ------- 1 files changed, 0 insertions(+), 7 deletions(-) diff --git a/Doc/library/token.rst b/Doc/library/token.rst --- a/Doc/library/token.rst +++ b/Doc/library/token.rst @@ -108,10 +108,3 @@ Added :data:`AWAIT` and :data:`ASYNC` tokens. Starting with Python 3.7, "async" and "await" will be tokenized as :data:`NAME` tokens, and :data:`AWAIT` and :data:`ASYNC` will be removed. - -.. seealso:: - - Module :mod:`parser` - The second example for the :mod:`parser` module shows how to use the - :mod:`symbol` module. - -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 8 06:34:03 2016 From: python-checkins at python.org (berker.peksag) Date: Mon, 08 Aug 2016 10:34:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_use_of_def?= =?utf-8?q?ault_reST_role?= Message-ID: <20160808103403.71977.58380.942D2642@psf.io> https://hg.python.org/cpython/rev/9f8666d2ff5d changeset: 102567:9f8666d2ff5d branch: 3.5 parent: 102565:b479e74f7706 user: Berker Peksag date: Mon Aug 08 13:34:49 2016 +0300 summary: Fix use of default reST role files: Doc/library/typing.rst | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -296,8 +296,8 @@ s = a # OK def foo(item: Any) -> int: - # Typechecks; `item` could be any type, - # and that type might have a `bar` method + # Typechecks; 'item' could be any type, + # and that type might have a 'bar' method item.bar() ... @@ -333,7 +333,7 @@ it as a return value) of a more specialized type is a type error. For example:: def hash_a(item: object) -> int: - # Fails; an object does not have a `magic` method. + # Fails; an object does not have a 'magic' method. item.magic() ... -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 8 06:34:03 2016 From: python-checkins at python.org (berker.peksag) Date: Mon, 08 Aug 2016 10:34:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E5?= Message-ID: <20160808103403.11326.4686.0006715F@psf.io> https://hg.python.org/cpython/rev/ccbcc0e5ce53 changeset: 102568:ccbcc0e5ce53 parent: 102566:099fd7954941 parent: 102567:9f8666d2ff5d user: Berker Peksag date: Mon Aug 08 13:35:27 2016 +0300 summary: Merge from 3.5 files: Doc/library/typing.rst | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -296,8 +296,8 @@ s = a # OK def foo(item: Any) -> int: - # Typechecks; `item` could be any type, - # and that type might have a `bar` method + # Typechecks; 'item' could be any type, + # and that type might have a 'bar' method item.bar() ... @@ -333,7 +333,7 @@ it as a return value) of a more specialized type is a type error. For example:: def hash_a(item: object) -> int: - # Fails; an object does not have a `magic` method. + # Fails; an object does not have a 'magic' method. item.magic() ... -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 8 06:38:19 2016 From: python-checkins at python.org (berker.peksag) Date: Mon, 08 Aug 2016 10:38:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Expose_EPOLLRDHUP_conditio?= =?utf-8?q?nally?= Message-ID: <20160808103819.30715.67477.BD46788A@psf.io> https://hg.python.org/cpython/rev/831ce2a717eb changeset: 102569:831ce2a717eb user: Berker Peksag date: Mon Aug 08 13:39:43 2016 +0300 summary: Expose EPOLLRDHUP conditionally files: Modules/selectmodule.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Modules/selectmodule.c b/Modules/selectmodule.c --- a/Modules/selectmodule.c +++ b/Modules/selectmodule.c @@ -2481,7 +2481,10 @@ PyModule_AddIntMacro(m, EPOLLPRI); PyModule_AddIntMacro(m, EPOLLERR); PyModule_AddIntMacro(m, EPOLLHUP); +#ifdef EPOLLRDHUP + /* Kernel 2.6.17 */ PyModule_AddIntMacro(m, EPOLLRDHUP); +#endif PyModule_AddIntMacro(m, EPOLLET); #ifdef EPOLLONESHOT /* Kernel 2.6.2+ */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 8 07:05:41 2016 From: python-checkins at python.org (berker.peksag) Date: Mon, 08 Aug 2016 11:05:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327702=3A_Only_exp?= =?utf-8?q?ose_SOCK=5FRAW_when_defined?= Message-ID: <20160808110540.30937.90396.CB7383E4@psf.io> https://hg.python.org/cpython/rev/535f88ad80d8 changeset: 102570:535f88ad80d8 user: Berker Peksag date: Mon Aug 08 14:07:05 2016 +0300 summary: Issue #27702: Only expose SOCK_RAW when defined SOCK_RAW is marked as optional in the POSIX specification: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_socket.h.html Patch by Ed Schouten. files: Modules/socketmodule.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -6495,7 +6495,10 @@ PyModule_AddIntMacro(m, SOCK_STREAM); PyModule_AddIntMacro(m, SOCK_DGRAM); /* We have incomplete socket support. */ +#ifdef SOCK_RAW + /* SOCK_RAW is marked as optional in the POSIX specification */ PyModule_AddIntMacro(m, SOCK_RAW); +#endif PyModule_AddIntMacro(m, SOCK_SEQPACKET); #if defined(SOCK_RDM) PyModule_AddIntMacro(m, SOCK_RDM); -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Mon Aug 8 07:41:14 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Mon, 8 Aug 2016 12:41:14 +0100 Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-08-08 Message-ID: <4e8dd5b2-fd8d-434f-a6e6-8dea1e53e617@irsmsx101.ger.corp.intel.com> Results for project Python 2.7, build date 2016-08-08 02:48:06 +0000 commit: 09475e6135d0 previous commit: 5ab52b5ea4a7 revision date: 2016-08-05 20:10:26 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.14% -2.28% 3.79% 8.49% :-) pybench 0.20% -0.05% 5.66% 5.57% :-( regex_v8 0.62% -0.06% -2.37% 11.11% :-) nbody 0.29% -0.13% 8.50% 2.01% :-| json_dump_v2 0.29% -0.15% 1.92% 11.93% :-( normal_startup 1.88% -0.06% -5.97% 2.17% :-) ssbench 0.17% 0.30% 2.68% 1.69% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-08-08/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Mon Aug 8 07:45:21 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Mon, 8 Aug 2016 12:45:21 +0100 Subject: [Python-checkins] NEUTRAL Benchmark Results for Python Default 2016-08-08 Message-ID: Results for project Python default, build date 2016-08-08 02:01:56 +0000 commit: d86eabe2c457 previous commit: 777256817c1b revision date: 2016-08-08 00:11:06 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.22% 0.29% 10.87% 15.77% :-) pybench 0.10% 0.63% 5.93% 7.15% :-( regex_v8 2.70% 0.71% -2.79% 8.01% :-| nbody 0.73% -0.27% 1.21% 9.64% :-| json_dump_v2 0.34% 0.51% -0.75% 10.73% :-| normal_startup 0.94% -0.00% -1.72% 5.72% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-default-2016-08-08/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Mon Aug 8 17:05:42 2016 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 08 Aug 2016 21:05:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2327710=3A_Disallo?= =?utf-8?q?w_fold_not_in_=5B0=2C_1=5D_in_time_and_datetime_constructors=2E?= Message-ID: <20160808210542.12941.65978.087F737A@psf.io> https://hg.python.org/cpython/rev/035778fdc641 changeset: 102571:035778fdc641 user: Alexander Belopolsky date: Mon Aug 08 17:05:40 2016 -0400 summary: Closes #27710: Disallow fold not in [0, 1] in time and datetime constructors. files: Lib/datetime.py | 14 +++++++----- Lib/test/datetimetester.py | 5 ++++ Modules/_datetimemodule.c | 28 +++++++++++++++++++++---- 3 files changed, 36 insertions(+), 11 deletions(-) diff --git a/Lib/datetime.py b/Lib/datetime.py --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -288,7 +288,7 @@ raise ValueError('day must be in 1..%d' % dim, day) return year, month, day -def _check_time_fields(hour, minute, second, microsecond): +def _check_time_fields(hour, minute, second, microsecond, fold): hour = _check_int_field(hour) minute = _check_int_field(minute) second = _check_int_field(second) @@ -301,7 +301,9 @@ raise ValueError('second must be in 0..59', second) if not 0 <= microsecond <= 999999: raise ValueError('microsecond must be in 0..999999', microsecond) - return hour, minute, second, microsecond + if fold not in (0, 1): + raise ValueError('fold must be either 0 or 1', fold) + return hour, minute, second, microsecond, fold def _check_tzinfo_arg(tz): if tz is not None and not isinstance(tz, tzinfo): @@ -1059,8 +1061,8 @@ self.__setstate(hour, minute or None) self._hashcode = -1 return self - hour, minute, second, microsecond = _check_time_fields( - hour, minute, second, microsecond) + hour, minute, second, microsecond, fold = _check_time_fields( + hour, minute, second, microsecond, fold) _check_tzinfo_arg(tzinfo) self = object.__new__(cls) self._hour = hour @@ -1369,8 +1371,8 @@ self._hashcode = -1 return self year, month, day = _check_date_fields(year, month, day) - hour, minute, second, microsecond = _check_time_fields( - hour, minute, second, microsecond) + hour, minute, second, microsecond, fold = _check_time_fields( + hour, minute, second, microsecond, fold) _check_tzinfo_arg(tzinfo) self = object.__new__(cls) self._year = year diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -1724,6 +1724,11 @@ self.assertRaises(ValueError, self.theclass, 2000, 1, 31, 23, 59, 59, 1000000) + # bad fold + self.assertRaises(ValueError, self.theclass, + 2000, 1, 31, fold=-1) + self.assertRaises(ValueError, self.theclass, + 2000, 1, 31, fold=2) # Positional fold: self.assertRaises(TypeError, self.theclass, 2000, 1, 31, 23, 59, 59, 0, None, 1) diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -427,7 +427,7 @@ * aren't, raise ValueError and return -1. */ static int -check_time_args(int h, int m, int s, int us) +check_time_args(int h, int m, int s, int us, int fold) { if (h < 0 || h > 23) { PyErr_SetString(PyExc_ValueError, @@ -449,6 +449,11 @@ "microsecond must be in 0..999999"); return -1; } + if (fold != 0 && fold != 1) { + PyErr_SetString(PyExc_ValueError, + "fold must be either 0 or 1"); + return -1; + } return 0; } @@ -3598,7 +3603,7 @@ if (PyArg_ParseTupleAndKeywords(args, kw, "|iiiiO$i", time_kws, &hour, &minute, &second, &usecond, &tzinfo, &fold)) { - if (check_time_args(hour, minute, second, usecond) < 0) + if (check_time_args(hour, minute, second, usecond, fold) < 0) return NULL; if (check_tzinfo_subclass(tzinfo) < 0) return NULL; @@ -3926,8 +3931,14 @@ if (tuple == NULL) return NULL; clone = time_new(Py_TYPE(self), tuple, NULL); - if (clone != NULL) + if (clone != NULL) { + if (fold != 0 && fold != 1) { + PyErr_SetString(PyExc_ValueError, + "fold must be either 0 or 1"); + return NULL; + } TIME_SET_FOLD(clone, fold); + } Py_DECREF(tuple); return clone; } @@ -4175,7 +4186,7 @@ &second, &usecond, &tzinfo, &fold)) { if (check_date_args(year, month, day) < 0) return NULL; - if (check_time_args(hour, minute, second, usecond) < 0) + if (check_time_args(hour, minute, second, usecond, fold) < 0) return NULL; if (check_tzinfo_subclass(tzinfo) < 0) return NULL; @@ -5006,8 +5017,15 @@ if (tuple == NULL) return NULL; clone = datetime_new(Py_TYPE(self), tuple, NULL); - if (clone != NULL) + + if (clone != NULL) { + if (fold != 0 && fold != 1) { + PyErr_SetString(PyExc_ValueError, + "fold must be either 0 or 1"); + return NULL; + } DATE_SET_FOLD(clone, fold); + } Py_DECREF(tuple); return clone; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 8 17:44:11 2016 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 08 Aug 2016 21:44:11 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NzAw?= =?utf-8?q?=3A_Document_AbstractEventLoop=2C_not_BaseEventLoop=2E?= Message-ID: <20160808214411.99594.83281.FD104D83@psf.io> https://hg.python.org/cpython/rev/87e3a58ed3c3 changeset: 102572:87e3a58ed3c3 branch: 3.5 parent: 102567:9f8666d2ff5d user: Guido van Rossum date: Mon Aug 08 09:41:21 2016 -0700 summary: Issue #27700: Document AbstractEventLoop, not BaseEventLoop. files: Doc/library/asyncio-dev.rst | 22 +- Doc/library/asyncio-eventloop.rst | 133 +++++++++------- Doc/library/asyncio-eventloops.rst | 28 +- Doc/library/asyncio-protocol.rst | 18 +- Doc/library/asyncio-stream.rst | 18 +- Doc/library/asyncio-subprocess.rst | 26 +- Doc/library/asyncio-task.rst | 22 +- 7 files changed, 138 insertions(+), 129 deletions(-) diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -21,7 +21,7 @@ To enable all debug checks for an application: * Enable the asyncio debug mode globally by setting the environment variable - :envvar:`PYTHONASYNCIODEBUG` to ``1``, or by calling :meth:`BaseEventLoop.set_debug`. + :envvar:`PYTHONASYNCIODEBUG` to ``1``, or by calling :meth:`AbstractEventLoop.set_debug`. * Set the log level of the :ref:`asyncio logger ` to :py:data:`logging.DEBUG`. For example, call ``logging.basicConfig(level=logging.DEBUG)`` at startup. @@ -33,18 +33,18 @@ * Log :ref:`coroutines defined but never "yielded from" ` -* :meth:`~BaseEventLoop.call_soon` and :meth:`~BaseEventLoop.call_at` methods +* :meth:`~AbstractEventLoop.call_soon` and :meth:`~AbstractEventLoop.call_at` methods raise an exception if they are called from the wrong thread. * Log the execution time of the selector * Log callbacks taking more than 100 ms to be executed. The - :attr:`BaseEventLoop.slow_callback_duration` attribute is the minimum + :attr:`AbstractEventLoop.slow_callback_duration` attribute is the minimum duration in seconds of "slow" callbacks. * :exc:`ResourceWarning` warnings are emitted when transports and event loops are :ref:`not closed explicitly `. .. seealso:: - The :meth:`BaseEventLoop.set_debug` method and the :ref:`asyncio logger + The :meth:`AbstractEventLoop.set_debug` method and the :ref:`asyncio logger `. @@ -68,7 +68,7 @@ Don't schedule directly a call to the :meth:`~Future.set_result` or the :meth:`~Future.set_exception` method of a future with -:meth:`BaseEventLoop.call_soon`: the future can be cancelled before its method +:meth:`AbstractEventLoop.call_soon`: the future can be cancelled before its method is called. If you wait for a future, you should check early if the future was cancelled to @@ -96,7 +96,7 @@ and the event loop executes the next task. To schedule a callback from a different thread, the -:meth:`BaseEventLoop.call_soon_threadsafe` method should be used. Example:: +:meth:`AbstractEventLoop.call_soon_threadsafe` method should be used. Example:: loop.call_soon_threadsafe(callback, *args) @@ -116,7 +116,7 @@ future = asyncio.run_coroutine_threadsafe(coro_func(), loop) result = future.result(timeout) # Wait for the result with a timeout -The :meth:`BaseEventLoop.run_in_executor` method can be used with a thread pool +The :meth:`AbstractEventLoop.run_in_executor` method can be used with a thread pool executor to execute a callback in different thread to not block the thread of the event loop. @@ -145,7 +145,7 @@ An executor can be used to run a task in a different thread or even in a different process, to not block the thread of the event loop. See the -:meth:`BaseEventLoop.run_in_executor` method. +:meth:`AbstractEventLoop.run_in_executor` method. .. seealso:: @@ -168,7 +168,7 @@ ---------------------------------------- When a coroutine function is called and its result is not passed to -:func:`ensure_future` or to the :meth:`BaseEventLoop.create_task` method, +:func:`ensure_future` or to the :meth:`AbstractEventLoop.create_task` method, the execution of the coroutine object will never be scheduled which is probably a bug. :ref:`Enable the debug mode of asyncio ` to :ref:`log a warning ` to detect it. @@ -191,7 +191,7 @@ test() The fix is to call the :func:`ensure_future` function or the -:meth:`BaseEventLoop.create_task` method with the coroutine object. +:meth:`AbstractEventLoop.create_task` method with the coroutine object. .. seealso:: @@ -267,7 +267,7 @@ loop.run_forever() loop.close() -Another option is to use the :meth:`BaseEventLoop.run_until_complete` +Another option is to use the :meth:`AbstractEventLoop.run_until_complete` function:: task = asyncio.ensure_future(bug()) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -20,14 +20,23 @@ .. class:: BaseEventLoop - Base class of event loops. + This class is an implementation detail. It is a subclass of + :class:`AbstractEventLoop` and may be a base class of concrete + event loop implementations found in :mod:`asyncio`. It should not + be used directly; use :class:`AbstractEventLoop` instead. + ``BaseEventLoop`` should not be subclassed by third-party code; the + internal interface is not stable. + +.. class:: AbstractEventLoop + + Abstract base class of event loops. This class is :ref:`not thread safe `. Run an event loop ----------------- -.. method:: BaseEventLoop.run_forever() +.. method:: AbstractEventLoop.run_forever() Run until :meth:`stop` is called. If :meth:`stop` is called before :meth:`run_forever()` is called, this polls the I/O selector once @@ -40,7 +49,7 @@ .. versionchanged:: 3.5.1 -.. method:: BaseEventLoop.run_until_complete(future) +.. method:: AbstractEventLoop.run_until_complete(future) Run until the :class:`Future` is done. @@ -49,11 +58,11 @@ Return the Future's result, or raise its exception. -.. method:: BaseEventLoop.is_running() +.. method:: AbstractEventLoop.is_running() Returns running status of event loop. -.. method:: BaseEventLoop.stop() +.. method:: AbstractEventLoop.stop() Stop running the event loop. @@ -62,13 +71,13 @@ .. versionchanged:: 3.5.1 -.. method:: BaseEventLoop.is_closed() +.. method:: AbstractEventLoop.is_closed() Returns ``True`` if the event loop was closed. .. versionadded:: 3.4.2 -.. method:: BaseEventLoop.close() +.. method:: AbstractEventLoop.close() Close the event loop. The loop must not be running. Pending callbacks will be lost. @@ -95,7 +104,7 @@ parameters in debug mode, whereas ``lambda`` functions have a poor representation. -.. method:: BaseEventLoop.call_soon(callback, \*args) +.. method:: AbstractEventLoop.call_soon(callback, \*args) Arrange for a callback to be called as soon as possible. The callback is called after :meth:`call_soon` returns, when control returns to the event @@ -113,7 +122,7 @@ :ref:`Use functools.partial to pass keywords to the callback `. -.. method:: BaseEventLoop.call_soon_threadsafe(callback, \*args) +.. method:: AbstractEventLoop.call_soon_threadsafe(callback, \*args) Like :meth:`call_soon`, but thread safe. @@ -136,7 +145,7 @@ Timeouts (relative *delay* or absolute *when*) should not exceed one day. -.. method:: BaseEventLoop.call_later(delay, callback, *args) +.. method:: AbstractEventLoop.call_later(delay, callback, *args) Arrange for the *callback* to be called after the given *delay* seconds (either an int or float). @@ -155,11 +164,11 @@ :ref:`Use functools.partial to pass keywords to the callback `. -.. method:: BaseEventLoop.call_at(when, callback, *args) +.. method:: AbstractEventLoop.call_at(when, callback, *args) Arrange for the *callback* to be called at the given absolute timestamp *when* (an int or float), using the same time reference as - :meth:`BaseEventLoop.time`. + :meth:`AbstractEventLoop.time`. This method's behavior is the same as :meth:`call_later`. @@ -169,7 +178,7 @@ :ref:`Use functools.partial to pass keywords to the callback `. -.. method:: BaseEventLoop.time() +.. method:: AbstractEventLoop.time() Return the current time, as a :class:`float` value, according to the event loop's internal clock. @@ -182,7 +191,7 @@ Futures ------- -.. method:: BaseEventLoop.create_future() +.. method:: AbstractEventLoop.create_future() Create an :class:`asyncio.Future` object attached to the loop. @@ -196,7 +205,7 @@ Tasks ----- -.. method:: BaseEventLoop.create_task(coro) +.. method:: AbstractEventLoop.create_task(coro) Schedule the execution of a :ref:`coroutine object `: wrap it in a future. Return a :class:`Task` object. @@ -210,10 +219,10 @@ .. versionadded:: 3.4.2 -.. method:: BaseEventLoop.set_task_factory(factory) +.. method:: AbstractEventLoop.set_task_factory(factory) Set a task factory that will be used by - :meth:`BaseEventLoop.create_task`. + :meth:`AbstractEventLoop.create_task`. If *factory* is ``None`` the default task factory will be set. @@ -224,7 +233,7 @@ .. versionadded:: 3.4.4 -.. method:: BaseEventLoop.get_task_factory() +.. method:: AbstractEventLoop.get_task_factory() Return a task factory, or ``None`` if the default one is in use. @@ -234,7 +243,7 @@ Creating connections -------------------- -.. coroutinemethod:: BaseEventLoop.create_connection(protocol_factory, host=None, port=None, \*, ssl=None, family=0, proto=0, flags=0, sock=None, local_addr=None, server_hostname=None) +.. coroutinemethod:: AbstractEventLoop.create_connection(protocol_factory, host=None, port=None, \*, ssl=None, family=0, proto=0, flags=0, sock=None, local_addr=None, server_hostname=None) Create a streaming transport connection to a given Internet *host* and *port*: socket family :py:data:`~socket.AF_INET` or @@ -309,7 +318,7 @@ (:class:`StreamReader`, :class:`StreamWriter`) instead of a protocol. -.. coroutinemethod:: BaseEventLoop.create_datagram_endpoint(protocol_factory, local_addr=None, remote_addr=None, \*, family=0, proto=0, flags=0, reuse_address=None, reuse_port=None, allow_broadcast=None, sock=None) +.. coroutinemethod:: AbstractEventLoop.create_datagram_endpoint(protocol_factory, local_addr=None, remote_addr=None, \*, family=0, proto=0, flags=0, reuse_address=None, reuse_port=None, allow_broadcast=None, sock=None) Create datagram connection: socket family :py:data:`~socket.AF_INET` or :py:data:`~socket.AF_INET6` depending on *host* (or *family* if specified), @@ -360,7 +369,7 @@ :ref:`UDP echo server protocol ` examples. -.. coroutinemethod:: BaseEventLoop.create_unix_connection(protocol_factory, path, \*, ssl=None, sock=None, server_hostname=None) +.. coroutinemethod:: AbstractEventLoop.create_unix_connection(protocol_factory, path, \*, ssl=None, sock=None, server_hostname=None) Create UNIX connection: socket family :py:data:`~socket.AF_UNIX`, socket type :py:data:`~socket.SOCK_STREAM`. The :py:data:`~socket.AF_UNIX` socket @@ -371,7 +380,7 @@ establish the connection in the background. When successful, the coroutine returns a ``(transport, protocol)`` pair. - See the :meth:`BaseEventLoop.create_connection` method for parameters. + See the :meth:`AbstractEventLoop.create_connection` method for parameters. Availability: UNIX. @@ -379,7 +388,7 @@ Creating listening connections ------------------------------ -.. coroutinemethod:: BaseEventLoop.create_server(protocol_factory, host=None, port=None, \*, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None, reuse_port=None) +.. coroutinemethod:: AbstractEventLoop.create_server(protocol_factory, host=None, port=None, \*, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None, reuse_port=None) Create a TCP server (socket type :data:`~socket.SOCK_STREAM`) bound to *host* and *port*. @@ -439,9 +448,9 @@ The *host* parameter can now be a sequence of strings. -.. coroutinemethod:: BaseEventLoop.create_unix_server(protocol_factory, path=None, \*, sock=None, backlog=100, ssl=None) +.. coroutinemethod:: AbstractEventLoop.create_unix_server(protocol_factory, path=None, \*, sock=None, backlog=100, ssl=None) - Similar to :meth:`BaseEventLoop.create_server`, but specific to the + Similar to :meth:`AbstractEventLoop.create_server`, but specific to the socket family :py:data:`~socket.AF_UNIX`. This method is a :ref:`coroutine `. @@ -457,7 +466,7 @@ On Windows with :class:`ProactorEventLoop`, these methods are not supported. -.. method:: BaseEventLoop.add_reader(fd, callback, \*args) +.. method:: AbstractEventLoop.add_reader(fd, callback, \*args) Start watching the file descriptor for read availability and then call the *callback* with specified arguments. @@ -465,11 +474,11 @@ :ref:`Use functools.partial to pass keywords to the callback `. -.. method:: BaseEventLoop.remove_reader(fd) +.. method:: AbstractEventLoop.remove_reader(fd) Stop watching the file descriptor for read availability. -.. method:: BaseEventLoop.add_writer(fd, callback, \*args) +.. method:: AbstractEventLoop.add_writer(fd, callback, \*args) Start watching the file descriptor for write availability and then call the *callback* with specified arguments. @@ -477,19 +486,19 @@ :ref:`Use functools.partial to pass keywords to the callback `. -.. method:: BaseEventLoop.remove_writer(fd) +.. method:: AbstractEventLoop.remove_writer(fd) Stop watching the file descriptor for write availability. The :ref:`watch a file descriptor for read events ` -example uses the low-level :meth:`BaseEventLoop.add_reader` method to register +example uses the low-level :meth:`AbstractEventLoop.add_reader` method to register the file descriptor of a socket. Low-level socket operations --------------------------- -.. coroutinemethod:: BaseEventLoop.sock_recv(sock, nbytes) +.. coroutinemethod:: AbstractEventLoop.sock_recv(sock, nbytes) Receive data from the socket. Modeled after blocking :meth:`socket.socket.recv` method. @@ -503,7 +512,7 @@ This method is a :ref:`coroutine `. -.. coroutinemethod:: BaseEventLoop.sock_sendall(sock, data) +.. coroutinemethod:: AbstractEventLoop.sock_sendall(sock, data) Send data to the socket. Modeled after blocking :meth:`socket.socket.sendall` method. @@ -519,7 +528,7 @@ This method is a :ref:`coroutine `. -.. coroutinemethod:: BaseEventLoop.sock_connect(sock, address) +.. coroutinemethod:: AbstractEventLoop.sock_connect(sock, address) Connect to a remote socket at *address*. Modeled after blocking :meth:`socket.socket.connect` method. @@ -533,16 +542,16 @@ ``address`` no longer needs to be resolved. ``sock_connect`` will try to check if the *address* is already resolved by calling :func:`socket.inet_pton`. If not, - :meth:`BaseEventLoop.getaddrinfo` will be used to resolve the + :meth:`AbstractEventLoop.getaddrinfo` will be used to resolve the *address*. .. seealso:: - :meth:`BaseEventLoop.create_connection` + :meth:`AbstractEventLoop.create_connection` and :func:`asyncio.open_connection() `. -.. coroutinemethod:: BaseEventLoop.sock_accept(sock) +.. coroutinemethod:: AbstractEventLoop.sock_accept(sock) Accept a connection. Modeled after blocking :meth:`socket.socket.accept`. @@ -559,18 +568,18 @@ .. seealso:: - :meth:`BaseEventLoop.create_server` and :func:`start_server`. + :meth:`AbstractEventLoop.create_server` and :func:`start_server`. Resolve host name ----------------- -.. coroutinemethod:: BaseEventLoop.getaddrinfo(host, port, \*, family=0, type=0, proto=0, flags=0) +.. coroutinemethod:: AbstractEventLoop.getaddrinfo(host, port, \*, family=0, type=0, proto=0, flags=0) This method is a :ref:`coroutine `, similar to :meth:`socket.getaddrinfo` function but non-blocking. -.. coroutinemethod:: BaseEventLoop.getnameinfo(sockaddr, flags=0) +.. coroutinemethod:: AbstractEventLoop.getnameinfo(sockaddr, flags=0) This method is a :ref:`coroutine `, similar to :meth:`socket.getnameinfo` function but non-blocking. @@ -582,7 +591,7 @@ On Windows with :class:`SelectorEventLoop`, these methods are not supported. Use :class:`ProactorEventLoop` to support pipes on Windows. -.. coroutinemethod:: BaseEventLoop.connect_read_pipe(protocol_factory, pipe) +.. coroutinemethod:: AbstractEventLoop.connect_read_pipe(protocol_factory, pipe) Register read pipe in eventloop. @@ -596,7 +605,7 @@ This method is a :ref:`coroutine `. -.. coroutinemethod:: BaseEventLoop.connect_write_pipe(protocol_factory, pipe) +.. coroutinemethod:: AbstractEventLoop.connect_write_pipe(protocol_factory, pipe) Register write pipe in eventloop. @@ -612,8 +621,8 @@ .. seealso:: - The :meth:`BaseEventLoop.subprocess_exec` and - :meth:`BaseEventLoop.subprocess_shell` methods. + The :meth:`AbstractEventLoop.subprocess_exec` and + :meth:`AbstractEventLoop.subprocess_shell` methods. UNIX signals @@ -621,7 +630,7 @@ Availability: UNIX only. -.. method:: BaseEventLoop.add_signal_handler(signum, callback, \*args) +.. method:: AbstractEventLoop.add_signal_handler(signum, callback, \*args) Add a handler for a signal. @@ -631,7 +640,7 @@ :ref:`Use functools.partial to pass keywords to the callback `. -.. method:: BaseEventLoop.remove_signal_handler(sig) +.. method:: AbstractEventLoop.remove_signal_handler(sig) Remove a handler for a signal. @@ -649,7 +658,7 @@ pool of processes). By default, an event loop uses a thread pool executor (:class:`~concurrent.futures.ThreadPoolExecutor`). -.. coroutinemethod:: BaseEventLoop.run_in_executor(executor, func, \*args) +.. coroutinemethod:: AbstractEventLoop.run_in_executor(executor, func, \*args) Arrange for a *func* to be called in the specified executor. @@ -661,7 +670,7 @@ This method is a :ref:`coroutine `. -.. method:: BaseEventLoop.set_default_executor(executor) +.. method:: AbstractEventLoop.set_default_executor(executor) Set the default executor used by :meth:`run_in_executor`. @@ -671,7 +680,7 @@ Allows customizing how exceptions are handled in the event loop. -.. method:: BaseEventLoop.set_exception_handler(handler) +.. method:: AbstractEventLoop.set_exception_handler(handler) Set *handler* as the new event loop exception handler. @@ -684,14 +693,14 @@ will be a ``dict`` object (see :meth:`call_exception_handler` documentation for details about context). -.. method:: BaseEventLoop.get_exception_handler() +.. method:: AbstractEventLoop.get_exception_handler() Return the exception handler, or ``None`` if the default one is in use. .. versionadded:: 3.5.2 -.. method:: BaseEventLoop.default_exception_handler(context) +.. method:: AbstractEventLoop.default_exception_handler(context) Default exception handler. @@ -702,7 +711,7 @@ *context* parameter has the same meaning as in :meth:`call_exception_handler`. -.. method:: BaseEventLoop.call_exception_handler(context) +.. method:: AbstractEventLoop.call_exception_handler(context) Call the current event loop exception handler. @@ -726,7 +735,7 @@ Debug mode ---------- -.. method:: BaseEventLoop.get_debug() +.. method:: AbstractEventLoop.get_debug() Get the debug mode (:class:`bool`) of the event loop. @@ -736,7 +745,7 @@ .. versionadded:: 3.4.2 -.. method:: BaseEventLoop.set_debug(enabled: bool) +.. method:: AbstractEventLoop.set_debug(enabled: bool) Set the debug mode of the event loop. @@ -753,7 +762,7 @@ Server listening on sockets. - Object created by the :meth:`BaseEventLoop.create_server` method and the + Object created by the :meth:`AbstractEventLoop.create_server` method and the :func:`start_server` function. Don't instantiate the class directly. .. method:: close() @@ -784,9 +793,9 @@ .. class:: Handle - A callback wrapper object returned by :func:`BaseEventLoop.call_soon`, - :func:`BaseEventLoop.call_soon_threadsafe`, :func:`BaseEventLoop.call_later`, - and :func:`BaseEventLoop.call_at`. + A callback wrapper object returned by :func:`AbstractEventLoop.call_soon`, + :func:`AbstractEventLoop.call_soon_threadsafe`, :func:`AbstractEventLoop.call_later`, + and :func:`AbstractEventLoop.call_at`. .. method:: cancel() @@ -802,7 +811,7 @@ Hello World with call_soon() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Example using the :meth:`BaseEventLoop.call_soon` method to schedule a +Example using the :meth:`AbstractEventLoop.call_soon` method to schedule a callback. The callback displays ``"Hello World"`` and then stops the event loop:: @@ -833,7 +842,7 @@ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Example of callback displaying the current date every second. The callback uses -the :meth:`BaseEventLoop.call_later` method to reschedule itself during 5 +the :meth:`AbstractEventLoop.call_later` method to reschedule itself during 5 seconds, and then stops the event loop:: import asyncio @@ -869,7 +878,7 @@ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Wait until a file descriptor received some data using the -:meth:`BaseEventLoop.add_reader` method and then close the event loop:: +:meth:`AbstractEventLoop.add_reader` method and then close the event loop:: import asyncio try: @@ -907,7 +916,7 @@ The :ref:`register an open socket to wait for data using a protocol ` example uses a low-level protocol created by the - :meth:`BaseEventLoop.create_connection` method. + :meth:`AbstractEventLoop.create_connection` method. The :ref:`register an open socket to wait for data using streams ` example uses high-level streams @@ -918,7 +927,7 @@ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Register handlers for signals :py:data:`SIGINT` and :py:data:`SIGTERM` using -the :meth:`BaseEventLoop.add_signal_handler` method:: +the :meth:`AbstractEventLoop.add_signal_handler` method:: import asyncio import functools diff --git a/Doc/library/asyncio-eventloops.rst b/Doc/library/asyncio-eventloops.rst --- a/Doc/library/asyncio-eventloops.rst +++ b/Doc/library/asyncio-eventloops.rst @@ -35,7 +35,7 @@ .. class:: SelectorEventLoop Event loop based on the :mod:`selectors` module. Subclass of - :class:`BaseEventLoop`. + :class:`AbstractEventLoop`. Use the most efficient selector available on the platform. @@ -46,7 +46,7 @@ .. class:: ProactorEventLoop Proactor event loop for Windows using "I/O Completion Ports" aka IOCP. - Subclass of :class:`BaseEventLoop`. + Subclass of :class:`AbstractEventLoop`. Availability: Windows. @@ -76,11 +76,11 @@ Common limits of Windows event loops: -- :meth:`~BaseEventLoop.create_unix_connection` and - :meth:`~BaseEventLoop.create_unix_server` are not supported: the socket +- :meth:`~AbstractEventLoop.create_unix_connection` and + :meth:`~AbstractEventLoop.create_unix_server` are not supported: the socket family :data:`socket.AF_UNIX` is specific to UNIX -- :meth:`~BaseEventLoop.add_signal_handler` and - :meth:`~BaseEventLoop.remove_signal_handler` are not supported +- :meth:`~AbstractEventLoop.add_signal_handler` and + :meth:`~AbstractEventLoop.remove_signal_handler` are not supported - :meth:`EventLoopPolicy.set_child_watcher` is not supported. :class:`ProactorEventLoop` supports subprocesses. It has only one implementation to watch child processes, there is no need to configure it. @@ -89,19 +89,19 @@ - :class:`~selectors.SelectSelector` is used which only supports sockets and is limited to 512 sockets. -- :meth:`~BaseEventLoop.add_reader` and :meth:`~BaseEventLoop.add_writer` only +- :meth:`~AbstractEventLoop.add_reader` and :meth:`~AbstractEventLoop.add_writer` only accept file descriptors of sockets - Pipes are not supported - (ex: :meth:`~BaseEventLoop.connect_read_pipe`, - :meth:`~BaseEventLoop.connect_write_pipe`) + (ex: :meth:`~AbstractEventLoop.connect_read_pipe`, + :meth:`~AbstractEventLoop.connect_write_pipe`) - :ref:`Subprocesses ` are not supported - (ex: :meth:`~BaseEventLoop.subprocess_exec`, - :meth:`~BaseEventLoop.subprocess_shell`) + (ex: :meth:`~AbstractEventLoop.subprocess_exec`, + :meth:`~AbstractEventLoop.subprocess_shell`) :class:`ProactorEventLoop` specific limits: -- :meth:`~BaseEventLoop.create_datagram_endpoint` (UDP) is not supported -- :meth:`~BaseEventLoop.add_reader` and :meth:`~BaseEventLoop.add_writer` are +- :meth:`~AbstractEventLoop.create_datagram_endpoint` (UDP) is not supported +- :meth:`~AbstractEventLoop.add_reader` and :meth:`~AbstractEventLoop.add_writer` are not supported The resolution of the monotonic clock on Windows is usually around 15.6 msec. @@ -167,7 +167,7 @@ Get the event loop for the current context. - Returns an event loop object implementing the :class:`BaseEventLoop` + Returns an event loop object implementing the :class:`AbstractEventLoop` interface. Raises an exception in case no event loop has been set for the current diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -11,7 +11,7 @@ Transports are classes provided by :mod:`asyncio` in order to abstract various kinds of communication channels. You generally won't instantiate -a transport yourself; instead, you will call a :class:`BaseEventLoop` method +a transport yourself; instead, you will call a :class:`AbstractEventLoop` method which will create the transport and try to initiate the underlying communication channel, calling you back when it succeeds. @@ -475,7 +475,7 @@ TCP echo client protocol ------------------------ -TCP echo client using the :meth:`BaseEventLoop.create_connection` method, send +TCP echo client using the :meth:`AbstractEventLoop.create_connection` method, send data and wait until the connection is closed:: import asyncio @@ -506,10 +506,10 @@ loop.close() The event loop is running twice. The -:meth:`~BaseEventLoop.run_until_complete` method is preferred in this short +:meth:`~AbstractEventLoop.run_until_complete` method is preferred in this short example to raise an exception if the server is not listening, instead of having to write a short coroutine to handle the exception and stop the -running loop. At :meth:`~BaseEventLoop.run_until_complete` exit, the loop is +running loop. At :meth:`~AbstractEventLoop.run_until_complete` exit, the loop is no longer running, so there is no need to stop the loop in case of an error. .. seealso:: @@ -523,7 +523,7 @@ TCP echo server protocol ------------------------ -TCP echo server using the :meth:`BaseEventLoop.create_server` method, send back +TCP echo server using the :meth:`AbstractEventLoop.create_server` method, send back received data and close the connection:: import asyncio @@ -577,7 +577,7 @@ UDP echo client protocol ------------------------ -UDP echo client using the :meth:`BaseEventLoop.create_datagram_endpoint` +UDP echo client using the :meth:`AbstractEventLoop.create_datagram_endpoint` method, send data and close the transport when we received the answer:: import asyncio @@ -623,7 +623,7 @@ UDP echo server protocol ------------------------ -UDP echo server using the :meth:`BaseEventLoop.create_datagram_endpoint` +UDP echo server using the :meth:`AbstractEventLoop.create_datagram_endpoint` method, send back received data:: import asyncio @@ -660,7 +660,7 @@ --------------------------------------------------------- Wait until a socket receives data using the -:meth:`BaseEventLoop.create_connection` method with a protocol, and then close +:meth:`AbstractEventLoop.create_connection` method with a protocol, and then close the event loop :: import asyncio @@ -708,7 +708,7 @@ The :ref:`watch a file descriptor for read events ` example uses the low-level - :meth:`BaseEventLoop.add_reader` method to register the file descriptor of a + :meth:`AbstractEventLoop.add_reader` method to register the file descriptor of a socket. The :ref:`register an open socket to wait for data using streams diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -18,14 +18,14 @@ .. coroutinefunction:: open_connection(host=None, port=None, \*, loop=None, limit=None, \*\*kwds) - A wrapper for :meth:`~BaseEventLoop.create_connection()` returning a (reader, + A wrapper for :meth:`~AbstractEventLoop.create_connection()` returning a (reader, writer) pair. The reader returned is a :class:`StreamReader` instance; the writer is a :class:`StreamWriter` instance. The arguments are all the usual arguments to - :meth:`BaseEventLoop.create_connection` except *protocol_factory*; most + :meth:`AbstractEventLoop.create_connection` except *protocol_factory*; most common are positional host and port, with various optional keyword arguments following. @@ -38,7 +38,7 @@ .. coroutinefunction:: start_server(client_connected_cb, host=None, port=None, \*, loop=None, limit=None, \*\*kwds) Start a socket server, with a callback for each client connected. The return - value is the same as :meth:`~BaseEventLoop.create_server()`. + value is the same as :meth:`~AbstractEventLoop.create_server()`. The *client_connected_cb* parameter is called with two parameters: *client_reader*, *client_writer*. *client_reader* is a @@ -49,7 +49,7 @@ converted into a :class:`Task`. The rest of the arguments are all the usual arguments to - :meth:`~BaseEventLoop.create_server()` except *protocol_factory*; most + :meth:`~AbstractEventLoop.create_server()` except *protocol_factory*; most common are positional *host* and *port*, with various optional keyword arguments following. @@ -61,7 +61,7 @@ .. coroutinefunction:: open_unix_connection(path=None, \*, loop=None, limit=None, **kwds) - A wrapper for :meth:`~BaseEventLoop.create_unix_connection()` returning + A wrapper for :meth:`~AbstractEventLoop.create_unix_connection()` returning a (reader, writer) pair. See :func:`open_connection` for information about return value and other @@ -321,7 +321,7 @@ .. seealso:: The :ref:`TCP echo client protocol ` - example uses the :meth:`BaseEventLoop.create_connection` method. + example uses the :meth:`AbstractEventLoop.create_connection` method. .. _asyncio-tcp-echo-server-streams: @@ -366,7 +366,7 @@ .. seealso:: The :ref:`TCP echo server protocol ` - example uses the :meth:`BaseEventLoop.create_server` method. + example uses the :meth:`AbstractEventLoop.create_server` method. Get HTTP headers @@ -458,10 +458,10 @@ The :ref:`register an open socket to wait for data using a protocol ` example uses a low-level protocol created by the - :meth:`BaseEventLoop.create_connection` method. + :meth:`AbstractEventLoop.create_connection` method. The :ref:`watch a file descriptor for read events ` example uses the low-level - :meth:`BaseEventLoop.add_reader` method to register the file descriptor of a + :meth:`AbstractEventLoop.add_reader` method to register the file descriptor of a socket. diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -32,7 +32,7 @@ Create a subprocess. The *limit* parameter sets the buffer limit passed to the - :class:`StreamReader`. See :meth:`BaseEventLoop.subprocess_exec` for other + :class:`StreamReader`. See :meth:`AbstractEventLoop.subprocess_exec` for other parameters. Return a :class:`~asyncio.subprocess.Process` instance. @@ -44,7 +44,7 @@ Run the shell command *cmd*. The *limit* parameter sets the buffer limit passed to the - :class:`StreamReader`. See :meth:`BaseEventLoop.subprocess_shell` for other + :class:`StreamReader`. See :meth:`AbstractEventLoop.subprocess_shell` for other parameters. Return a :class:`~asyncio.subprocess.Process` instance. @@ -58,8 +58,8 @@ This function is a :ref:`coroutine `. -Use the :meth:`BaseEventLoop.connect_read_pipe` and -:meth:`BaseEventLoop.connect_write_pipe` methods to connect pipes. +Use the :meth:`AbstractEventLoop.connect_read_pipe` and +:meth:`AbstractEventLoop.connect_write_pipe` methods to connect pipes. Create a subprocess: low-level API using subprocess.Popen @@ -67,7 +67,7 @@ Run subprocesses asynchronously using the :mod:`subprocess` module. -.. coroutinemethod:: BaseEventLoop.subprocess_exec(protocol_factory, \*args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) +.. coroutinemethod:: AbstractEventLoop.subprocess_exec(protocol_factory, \*args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) Create a subprocess from one or more string arguments (character strings or bytes strings encoded to the :ref:`filesystem encoding @@ -87,19 +87,19 @@ * *stdin*: Either a file-like object representing the pipe to be connected to the subprocess's standard input stream using - :meth:`~BaseEventLoop.connect_write_pipe`, or the constant + :meth:`~AbstractEventLoop.connect_write_pipe`, or the constant :const:`subprocess.PIPE` (the default). By default a new pipe will be created and connected. * *stdout*: Either a file-like object representing the pipe to be connected to the subprocess's standard output stream using - :meth:`~BaseEventLoop.connect_read_pipe`, or the constant + :meth:`~AbstractEventLoop.connect_read_pipe`, or the constant :const:`subprocess.PIPE` (the default). By default a new pipe will be created and connected. * *stderr*: Either a file-like object representing the pipe to be connected to the subprocess's standard error stream using - :meth:`~BaseEventLoop.connect_read_pipe`, or one of the constants + :meth:`~AbstractEventLoop.connect_read_pipe`, or one of the constants :const:`subprocess.PIPE` (the default) or :const:`subprocess.STDOUT`. By default a new pipe will be created and connected. When :const:`subprocess.STDOUT` is specified, the subprocess's standard error @@ -116,7 +116,7 @@ See the constructor of the :class:`subprocess.Popen` class for parameters. -.. coroutinemethod:: BaseEventLoop.subprocess_shell(protocol_factory, cmd, \*, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) +.. coroutinemethod:: AbstractEventLoop.subprocess_shell(protocol_factory, cmd, \*, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) Create a subprocess from *cmd*, which is a character string or a bytes string encoded to the :ref:`filesystem encoding `, @@ -126,7 +126,7 @@ The *protocol_factory* must instanciate a subclass of the :class:`asyncio.SubprocessProtocol` class. - See :meth:`~BaseEventLoop.subprocess_exec` for more details about + See :meth:`~AbstractEventLoop.subprocess_exec` for more details about the remaining arguments. Returns a pair of ``(transport, protocol)``, where *transport* is an @@ -143,8 +143,8 @@ .. seealso:: - The :meth:`BaseEventLoop.connect_read_pipe` and - :meth:`BaseEventLoop.connect_write_pipe` methods. + The :meth:`AbstractEventLoop.connect_read_pipe` and + :meth:`AbstractEventLoop.connect_write_pipe` methods. Constants @@ -329,7 +329,7 @@ Example of a subprocess protocol using to get the output of a subprocess and to wait for the subprocess exit. The subprocess is created by the -:meth:`BaseEventLoop.subprocess_exec` method:: +:meth:`AbstractEventLoop.subprocess_exec` method:: import asyncio import sys diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -59,7 +59,7 @@ schedule its execution. There are two basic ways to start it running: call ``await coroutine`` or ``yield from coroutine`` from another coroutine (assuming the other coroutine is already running!), or schedule its execution -using the :func:`ensure_future` function or the :meth:`BaseEventLoop.create_task` +using the :func:`ensure_future` function or the :meth:`AbstractEventLoop.create_task` method. @@ -108,7 +108,7 @@ .. seealso:: The :ref:`Hello World with call_soon() ` - example uses the :meth:`BaseEventLoop.call_soon` method to schedule a + example uses the :meth:`AbstractEventLoop.call_soon` method to schedule a callback. @@ -151,7 +151,7 @@ The :ref:`display the current date with call_later() ` example uses a callback with the - :meth:`BaseEventLoop.call_later` method. + :meth:`AbstractEventLoop.call_later` method. Example: Chain coroutines @@ -182,12 +182,12 @@ .. image:: tulip_coro.png :align: center -The "Task" is created by the :meth:`BaseEventLoop.run_until_complete` method +The "Task" is created by the :meth:`AbstractEventLoop.run_until_complete` method when it gets a coroutine object instead of a task. The diagram shows the control flow, it does not describe exactly how things work internally. For example, the sleep coroutine creates an internal future -which uses :meth:`BaseEventLoop.call_later` to wake up the task in 1 second. +which uses :meth:`AbstractEventLoop.call_later` to wake up the task in 1 second. InvalidStateError @@ -223,7 +223,7 @@ raise an exception when the future isn't done yet. - Callbacks registered with :meth:`add_done_callback` are always called - via the event loop's :meth:`~BaseEventLoop.call_soon_threadsafe`. + via the event loop's :meth:`~AbstractEventLoop.call_soon_threadsafe`. - This class is not compatible with the :func:`~concurrent.futures.wait` and :func:`~concurrent.futures.as_completed` functions in the @@ -273,7 +273,7 @@ The callback is called with a single argument - the future object. If the future is already done when this is called, the callback is scheduled - with :meth:`~BaseEventLoop.call_soon`. + with :meth:`~AbstractEventLoop.call_soon`. :ref:`Use functools.partial to pass parameters to the callback `. For example, @@ -323,11 +323,11 @@ The coroutine function is responsible for the computation (which takes 1 second) and it stores the result into the future. The -:meth:`~BaseEventLoop.run_until_complete` method waits for the completion of +:meth:`~AbstractEventLoop.run_until_complete` method waits for the completion of the future. .. note:: - The :meth:`~BaseEventLoop.run_until_complete` method uses internally the + The :meth:`~AbstractEventLoop.run_until_complete` method uses internally the :meth:`~Future.add_done_callback` method to be notified when the future is done. @@ -396,7 +396,7 @@ logged: see :ref:`Pending task destroyed `. Don't directly create :class:`Task` instances: use the :func:`ensure_future` - function or the :meth:`BaseEventLoop.create_task` method. + function or the :meth:`AbstractEventLoop.create_task` method. This class is :ref:`not thread safe `. @@ -546,7 +546,7 @@ .. seealso:: - The :meth:`BaseEventLoop.create_task` method. + The :meth:`AbstractEventLoop.create_task` method. .. function:: async(coro_or_future, \*, loop=None) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 8 17:44:11 2016 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 08 Aug 2016 21:44:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327700=3A_Document_AbstractEventLoop=2C_not_Base?= =?utf-8?b?RXZlbnRMb29wLiAoTWVyZ2UgMy41LT4zLjYp?= Message-ID: <20160808214411.11667.94123.88193C3B@psf.io> https://hg.python.org/cpython/rev/d69f782d642d changeset: 102573:d69f782d642d parent: 102571:035778fdc641 parent: 102572:87e3a58ed3c3 user: Guido van Rossum date: Mon Aug 08 14:43:46 2016 -0700 summary: Issue #27700: Document AbstractEventLoop, not BaseEventLoop. (Merge 3.5->3.6) files: Doc/library/asyncio-dev.rst | 22 +- Doc/library/asyncio-eventloop.rst | 133 +++++++++------- Doc/library/asyncio-eventloops.rst | 28 +- Doc/library/asyncio-protocol.rst | 18 +- Doc/library/asyncio-stream.rst | 18 +- Doc/library/asyncio-subprocess.rst | 26 +- Doc/library/asyncio-task.rst | 22 +- 7 files changed, 138 insertions(+), 129 deletions(-) diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -21,7 +21,7 @@ To enable all debug checks for an application: * Enable the asyncio debug mode globally by setting the environment variable - :envvar:`PYTHONASYNCIODEBUG` to ``1``, or by calling :meth:`BaseEventLoop.set_debug`. + :envvar:`PYTHONASYNCIODEBUG` to ``1``, or by calling :meth:`AbstractEventLoop.set_debug`. * Set the log level of the :ref:`asyncio logger ` to :py:data:`logging.DEBUG`. For example, call ``logging.basicConfig(level=logging.DEBUG)`` at startup. @@ -33,18 +33,18 @@ * Log :ref:`coroutines defined but never "yielded from" ` -* :meth:`~BaseEventLoop.call_soon` and :meth:`~BaseEventLoop.call_at` methods +* :meth:`~AbstractEventLoop.call_soon` and :meth:`~AbstractEventLoop.call_at` methods raise an exception if they are called from the wrong thread. * Log the execution time of the selector * Log callbacks taking more than 100 ms to be executed. The - :attr:`BaseEventLoop.slow_callback_duration` attribute is the minimum + :attr:`AbstractEventLoop.slow_callback_duration` attribute is the minimum duration in seconds of "slow" callbacks. * :exc:`ResourceWarning` warnings are emitted when transports and event loops are :ref:`not closed explicitly `. .. seealso:: - The :meth:`BaseEventLoop.set_debug` method and the :ref:`asyncio logger + The :meth:`AbstractEventLoop.set_debug` method and the :ref:`asyncio logger `. @@ -68,7 +68,7 @@ Don't schedule directly a call to the :meth:`~Future.set_result` or the :meth:`~Future.set_exception` method of a future with -:meth:`BaseEventLoop.call_soon`: the future can be cancelled before its method +:meth:`AbstractEventLoop.call_soon`: the future can be cancelled before its method is called. If you wait for a future, you should check early if the future was cancelled to @@ -96,7 +96,7 @@ and the event loop executes the next task. To schedule a callback from a different thread, the -:meth:`BaseEventLoop.call_soon_threadsafe` method should be used. Example:: +:meth:`AbstractEventLoop.call_soon_threadsafe` method should be used. Example:: loop.call_soon_threadsafe(callback, *args) @@ -116,7 +116,7 @@ future = asyncio.run_coroutine_threadsafe(coro_func(), loop) result = future.result(timeout) # Wait for the result with a timeout -The :meth:`BaseEventLoop.run_in_executor` method can be used with a thread pool +The :meth:`AbstractEventLoop.run_in_executor` method can be used with a thread pool executor to execute a callback in different thread to not block the thread of the event loop. @@ -145,7 +145,7 @@ An executor can be used to run a task in a different thread or even in a different process, to not block the thread of the event loop. See the -:meth:`BaseEventLoop.run_in_executor` method. +:meth:`AbstractEventLoop.run_in_executor` method. .. seealso:: @@ -168,7 +168,7 @@ ---------------------------------------- When a coroutine function is called and its result is not passed to -:func:`ensure_future` or to the :meth:`BaseEventLoop.create_task` method, +:func:`ensure_future` or to the :meth:`AbstractEventLoop.create_task` method, the execution of the coroutine object will never be scheduled which is probably a bug. :ref:`Enable the debug mode of asyncio ` to :ref:`log a warning ` to detect it. @@ -191,7 +191,7 @@ test() The fix is to call the :func:`ensure_future` function or the -:meth:`BaseEventLoop.create_task` method with the coroutine object. +:meth:`AbstractEventLoop.create_task` method with the coroutine object. .. seealso:: @@ -267,7 +267,7 @@ loop.run_forever() loop.close() -Another option is to use the :meth:`BaseEventLoop.run_until_complete` +Another option is to use the :meth:`AbstractEventLoop.run_until_complete` function:: task = asyncio.ensure_future(bug()) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -20,14 +20,23 @@ .. class:: BaseEventLoop - Base class of event loops. + This class is an implementation detail. It is a subclass of + :class:`AbstractEventLoop` and may be a base class of concrete + event loop implementations found in :mod:`asyncio`. It should not + be used directly; use :class:`AbstractEventLoop` instead. + ``BaseEventLoop`` should not be subclassed by third-party code; the + internal interface is not stable. + +.. class:: AbstractEventLoop + + Abstract base class of event loops. This class is :ref:`not thread safe `. Run an event loop ----------------- -.. method:: BaseEventLoop.run_forever() +.. method:: AbstractEventLoop.run_forever() Run until :meth:`stop` is called. If :meth:`stop` is called before :meth:`run_forever()` is called, this polls the I/O selector once @@ -40,7 +49,7 @@ .. versionchanged:: 3.5.1 -.. method:: BaseEventLoop.run_until_complete(future) +.. method:: AbstractEventLoop.run_until_complete(future) Run until the :class:`Future` is done. @@ -49,11 +58,11 @@ Return the Future's result, or raise its exception. -.. method:: BaseEventLoop.is_running() +.. method:: AbstractEventLoop.is_running() Returns running status of event loop. -.. method:: BaseEventLoop.stop() +.. method:: AbstractEventLoop.stop() Stop running the event loop. @@ -62,13 +71,13 @@ .. versionchanged:: 3.5.1 -.. method:: BaseEventLoop.is_closed() +.. method:: AbstractEventLoop.is_closed() Returns ``True`` if the event loop was closed. .. versionadded:: 3.4.2 -.. method:: BaseEventLoop.close() +.. method:: AbstractEventLoop.close() Close the event loop. The loop must not be running. Pending callbacks will be lost. @@ -95,7 +104,7 @@ parameters in debug mode, whereas ``lambda`` functions have a poor representation. -.. method:: BaseEventLoop.call_soon(callback, \*args) +.. method:: AbstractEventLoop.call_soon(callback, \*args) Arrange for a callback to be called as soon as possible. The callback is called after :meth:`call_soon` returns, when control returns to the event @@ -114,7 +123,7 @@ :ref:`Use functools.partial to pass keywords to the callback `. -.. method:: BaseEventLoop.call_soon_threadsafe(callback, \*args) +.. method:: AbstractEventLoop.call_soon_threadsafe(callback, \*args) Like :meth:`call_soon`, but thread safe. @@ -137,7 +146,7 @@ Timeouts (relative *delay* or absolute *when*) should not exceed one day. -.. method:: BaseEventLoop.call_later(delay, callback, *args) +.. method:: AbstractEventLoop.call_later(delay, callback, *args) Arrange for the *callback* to be called after the given *delay* seconds (either an int or float). @@ -156,11 +165,11 @@ :ref:`Use functools.partial to pass keywords to the callback `. -.. method:: BaseEventLoop.call_at(when, callback, *args) +.. method:: AbstractEventLoop.call_at(when, callback, *args) Arrange for the *callback* to be called at the given absolute timestamp *when* (an int or float), using the same time reference as - :meth:`BaseEventLoop.time`. + :meth:`AbstractEventLoop.time`. This method's behavior is the same as :meth:`call_later`. @@ -170,7 +179,7 @@ :ref:`Use functools.partial to pass keywords to the callback `. -.. method:: BaseEventLoop.time() +.. method:: AbstractEventLoop.time() Return the current time, as a :class:`float` value, according to the event loop's internal clock. @@ -183,7 +192,7 @@ Futures ------- -.. method:: BaseEventLoop.create_future() +.. method:: AbstractEventLoop.create_future() Create an :class:`asyncio.Future` object attached to the loop. @@ -197,7 +206,7 @@ Tasks ----- -.. method:: BaseEventLoop.create_task(coro) +.. method:: AbstractEventLoop.create_task(coro) Schedule the execution of a :ref:`coroutine object `: wrap it in a future. Return a :class:`Task` object. @@ -211,10 +220,10 @@ .. versionadded:: 3.4.2 -.. method:: BaseEventLoop.set_task_factory(factory) +.. method:: AbstractEventLoop.set_task_factory(factory) Set a task factory that will be used by - :meth:`BaseEventLoop.create_task`. + :meth:`AbstractEventLoop.create_task`. If *factory* is ``None`` the default task factory will be set. @@ -225,7 +234,7 @@ .. versionadded:: 3.4.4 -.. method:: BaseEventLoop.get_task_factory() +.. method:: AbstractEventLoop.get_task_factory() Return a task factory, or ``None`` if the default one is in use. @@ -235,7 +244,7 @@ Creating connections -------------------- -.. coroutinemethod:: BaseEventLoop.create_connection(protocol_factory, host=None, port=None, \*, ssl=None, family=0, proto=0, flags=0, sock=None, local_addr=None, server_hostname=None) +.. coroutinemethod:: AbstractEventLoop.create_connection(protocol_factory, host=None, port=None, \*, ssl=None, family=0, proto=0, flags=0, sock=None, local_addr=None, server_hostname=None) Create a streaming transport connection to a given Internet *host* and *port*: socket family :py:data:`~socket.AF_INET` or @@ -310,7 +319,7 @@ (:class:`StreamReader`, :class:`StreamWriter`) instead of a protocol. -.. coroutinemethod:: BaseEventLoop.create_datagram_endpoint(protocol_factory, local_addr=None, remote_addr=None, \*, family=0, proto=0, flags=0, reuse_address=None, reuse_port=None, allow_broadcast=None, sock=None) +.. coroutinemethod:: AbstractEventLoop.create_datagram_endpoint(protocol_factory, local_addr=None, remote_addr=None, \*, family=0, proto=0, flags=0, reuse_address=None, reuse_port=None, allow_broadcast=None, sock=None) Create datagram connection: socket family :py:data:`~socket.AF_INET` or :py:data:`~socket.AF_INET6` depending on *host* (or *family* if specified), @@ -361,7 +370,7 @@ :ref:`UDP echo server protocol ` examples. -.. coroutinemethod:: BaseEventLoop.create_unix_connection(protocol_factory, path, \*, ssl=None, sock=None, server_hostname=None) +.. coroutinemethod:: AbstractEventLoop.create_unix_connection(protocol_factory, path, \*, ssl=None, sock=None, server_hostname=None) Create UNIX connection: socket family :py:data:`~socket.AF_UNIX`, socket type :py:data:`~socket.SOCK_STREAM`. The :py:data:`~socket.AF_UNIX` socket @@ -372,7 +381,7 @@ establish the connection in the background. When successful, the coroutine returns a ``(transport, protocol)`` pair. - See the :meth:`BaseEventLoop.create_connection` method for parameters. + See the :meth:`AbstractEventLoop.create_connection` method for parameters. Availability: UNIX. @@ -380,7 +389,7 @@ Creating listening connections ------------------------------ -.. coroutinemethod:: BaseEventLoop.create_server(protocol_factory, host=None, port=None, \*, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None, reuse_port=None) +.. coroutinemethod:: AbstractEventLoop.create_server(protocol_factory, host=None, port=None, \*, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None, reuse_port=None) Create a TCP server (socket type :data:`~socket.SOCK_STREAM`) bound to *host* and *port*. @@ -440,9 +449,9 @@ The *host* parameter can now be a sequence of strings. -.. coroutinemethod:: BaseEventLoop.create_unix_server(protocol_factory, path=None, \*, sock=None, backlog=100, ssl=None) +.. coroutinemethod:: AbstractEventLoop.create_unix_server(protocol_factory, path=None, \*, sock=None, backlog=100, ssl=None) - Similar to :meth:`BaseEventLoop.create_server`, but specific to the + Similar to :meth:`AbstractEventLoop.create_server`, but specific to the socket family :py:data:`~socket.AF_UNIX`. This method is a :ref:`coroutine `. @@ -458,7 +467,7 @@ On Windows with :class:`ProactorEventLoop`, these methods are not supported. -.. method:: BaseEventLoop.add_reader(fd, callback, \*args) +.. method:: AbstractEventLoop.add_reader(fd, callback, \*args) Start watching the file descriptor for read availability and then call the *callback* with specified arguments. @@ -466,11 +475,11 @@ :ref:`Use functools.partial to pass keywords to the callback `. -.. method:: BaseEventLoop.remove_reader(fd) +.. method:: AbstractEventLoop.remove_reader(fd) Stop watching the file descriptor for read availability. -.. method:: BaseEventLoop.add_writer(fd, callback, \*args) +.. method:: AbstractEventLoop.add_writer(fd, callback, \*args) Start watching the file descriptor for write availability and then call the *callback* with specified arguments. @@ -478,19 +487,19 @@ :ref:`Use functools.partial to pass keywords to the callback `. -.. method:: BaseEventLoop.remove_writer(fd) +.. method:: AbstractEventLoop.remove_writer(fd) Stop watching the file descriptor for write availability. The :ref:`watch a file descriptor for read events ` -example uses the low-level :meth:`BaseEventLoop.add_reader` method to register +example uses the low-level :meth:`AbstractEventLoop.add_reader` method to register the file descriptor of a socket. Low-level socket operations --------------------------- -.. coroutinemethod:: BaseEventLoop.sock_recv(sock, nbytes) +.. coroutinemethod:: AbstractEventLoop.sock_recv(sock, nbytes) Receive data from the socket. Modeled after blocking :meth:`socket.socket.recv` method. @@ -504,7 +513,7 @@ This method is a :ref:`coroutine `. -.. coroutinemethod:: BaseEventLoop.sock_sendall(sock, data) +.. coroutinemethod:: AbstractEventLoop.sock_sendall(sock, data) Send data to the socket. Modeled after blocking :meth:`socket.socket.sendall` method. @@ -520,7 +529,7 @@ This method is a :ref:`coroutine `. -.. coroutinemethod:: BaseEventLoop.sock_connect(sock, address) +.. coroutinemethod:: AbstractEventLoop.sock_connect(sock, address) Connect to a remote socket at *address*. Modeled after blocking :meth:`socket.socket.connect` method. @@ -534,16 +543,16 @@ ``address`` no longer needs to be resolved. ``sock_connect`` will try to check if the *address* is already resolved by calling :func:`socket.inet_pton`. If not, - :meth:`BaseEventLoop.getaddrinfo` will be used to resolve the + :meth:`AbstractEventLoop.getaddrinfo` will be used to resolve the *address*. .. seealso:: - :meth:`BaseEventLoop.create_connection` + :meth:`AbstractEventLoop.create_connection` and :func:`asyncio.open_connection() `. -.. coroutinemethod:: BaseEventLoop.sock_accept(sock) +.. coroutinemethod:: AbstractEventLoop.sock_accept(sock) Accept a connection. Modeled after blocking :meth:`socket.socket.accept`. @@ -560,18 +569,18 @@ .. seealso:: - :meth:`BaseEventLoop.create_server` and :func:`start_server`. + :meth:`AbstractEventLoop.create_server` and :func:`start_server`. Resolve host name ----------------- -.. coroutinemethod:: BaseEventLoop.getaddrinfo(host, port, \*, family=0, type=0, proto=0, flags=0) +.. coroutinemethod:: AbstractEventLoop.getaddrinfo(host, port, \*, family=0, type=0, proto=0, flags=0) This method is a :ref:`coroutine `, similar to :meth:`socket.getaddrinfo` function but non-blocking. -.. coroutinemethod:: BaseEventLoop.getnameinfo(sockaddr, flags=0) +.. coroutinemethod:: AbstractEventLoop.getnameinfo(sockaddr, flags=0) This method is a :ref:`coroutine `, similar to :meth:`socket.getnameinfo` function but non-blocking. @@ -583,7 +592,7 @@ On Windows with :class:`SelectorEventLoop`, these methods are not supported. Use :class:`ProactorEventLoop` to support pipes on Windows. -.. coroutinemethod:: BaseEventLoop.connect_read_pipe(protocol_factory, pipe) +.. coroutinemethod:: AbstractEventLoop.connect_read_pipe(protocol_factory, pipe) Register read pipe in eventloop. @@ -597,7 +606,7 @@ This method is a :ref:`coroutine `. -.. coroutinemethod:: BaseEventLoop.connect_write_pipe(protocol_factory, pipe) +.. coroutinemethod:: AbstractEventLoop.connect_write_pipe(protocol_factory, pipe) Register write pipe in eventloop. @@ -613,8 +622,8 @@ .. seealso:: - The :meth:`BaseEventLoop.subprocess_exec` and - :meth:`BaseEventLoop.subprocess_shell` methods. + The :meth:`AbstractEventLoop.subprocess_exec` and + :meth:`AbstractEventLoop.subprocess_shell` methods. UNIX signals @@ -622,7 +631,7 @@ Availability: UNIX only. -.. method:: BaseEventLoop.add_signal_handler(signum, callback, \*args) +.. method:: AbstractEventLoop.add_signal_handler(signum, callback, \*args) Add a handler for a signal. @@ -632,7 +641,7 @@ :ref:`Use functools.partial to pass keywords to the callback `. -.. method:: BaseEventLoop.remove_signal_handler(sig) +.. method:: AbstractEventLoop.remove_signal_handler(sig) Remove a handler for a signal. @@ -650,7 +659,7 @@ pool of processes). By default, an event loop uses a thread pool executor (:class:`~concurrent.futures.ThreadPoolExecutor`). -.. coroutinemethod:: BaseEventLoop.run_in_executor(executor, func, \*args) +.. coroutinemethod:: AbstractEventLoop.run_in_executor(executor, func, \*args) Arrange for a *func* to be called in the specified executor. @@ -662,7 +671,7 @@ This method is a :ref:`coroutine `. -.. method:: BaseEventLoop.set_default_executor(executor) +.. method:: AbstractEventLoop.set_default_executor(executor) Set the default executor used by :meth:`run_in_executor`. @@ -672,7 +681,7 @@ Allows customizing how exceptions are handled in the event loop. -.. method:: BaseEventLoop.set_exception_handler(handler) +.. method:: AbstractEventLoop.set_exception_handler(handler) Set *handler* as the new event loop exception handler. @@ -685,14 +694,14 @@ will be a ``dict`` object (see :meth:`call_exception_handler` documentation for details about context). -.. method:: BaseEventLoop.get_exception_handler() +.. method:: AbstractEventLoop.get_exception_handler() Return the exception handler, or ``None`` if the default one is in use. .. versionadded:: 3.5.2 -.. method:: BaseEventLoop.default_exception_handler(context) +.. method:: AbstractEventLoop.default_exception_handler(context) Default exception handler. @@ -703,7 +712,7 @@ *context* parameter has the same meaning as in :meth:`call_exception_handler`. -.. method:: BaseEventLoop.call_exception_handler(context) +.. method:: AbstractEventLoop.call_exception_handler(context) Call the current event loop exception handler. @@ -727,7 +736,7 @@ Debug mode ---------- -.. method:: BaseEventLoop.get_debug() +.. method:: AbstractEventLoop.get_debug() Get the debug mode (:class:`bool`) of the event loop. @@ -737,7 +746,7 @@ .. versionadded:: 3.4.2 -.. method:: BaseEventLoop.set_debug(enabled: bool) +.. method:: AbstractEventLoop.set_debug(enabled: bool) Set the debug mode of the event loop. @@ -754,7 +763,7 @@ Server listening on sockets. - Object created by the :meth:`BaseEventLoop.create_server` method and the + Object created by the :meth:`AbstractEventLoop.create_server` method and the :func:`start_server` function. Don't instantiate the class directly. .. method:: close() @@ -785,9 +794,9 @@ .. class:: Handle - A callback wrapper object returned by :func:`BaseEventLoop.call_soon`, - :func:`BaseEventLoop.call_soon_threadsafe`, :func:`BaseEventLoop.call_later`, - and :func:`BaseEventLoop.call_at`. + A callback wrapper object returned by :func:`AbstractEventLoop.call_soon`, + :func:`AbstractEventLoop.call_soon_threadsafe`, :func:`AbstractEventLoop.call_later`, + and :func:`AbstractEventLoop.call_at`. .. method:: cancel() @@ -803,7 +812,7 @@ Hello World with call_soon() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Example using the :meth:`BaseEventLoop.call_soon` method to schedule a +Example using the :meth:`AbstractEventLoop.call_soon` method to schedule a callback. The callback displays ``"Hello World"`` and then stops the event loop:: @@ -834,7 +843,7 @@ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Example of callback displaying the current date every second. The callback uses -the :meth:`BaseEventLoop.call_later` method to reschedule itself during 5 +the :meth:`AbstractEventLoop.call_later` method to reschedule itself during 5 seconds, and then stops the event loop:: import asyncio @@ -870,7 +879,7 @@ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Wait until a file descriptor received some data using the -:meth:`BaseEventLoop.add_reader` method and then close the event loop:: +:meth:`AbstractEventLoop.add_reader` method and then close the event loop:: import asyncio try: @@ -908,7 +917,7 @@ The :ref:`register an open socket to wait for data using a protocol ` example uses a low-level protocol created by the - :meth:`BaseEventLoop.create_connection` method. + :meth:`AbstractEventLoop.create_connection` method. The :ref:`register an open socket to wait for data using streams ` example uses high-level streams @@ -919,7 +928,7 @@ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Register handlers for signals :py:data:`SIGINT` and :py:data:`SIGTERM` using -the :meth:`BaseEventLoop.add_signal_handler` method:: +the :meth:`AbstractEventLoop.add_signal_handler` method:: import asyncio import functools diff --git a/Doc/library/asyncio-eventloops.rst b/Doc/library/asyncio-eventloops.rst --- a/Doc/library/asyncio-eventloops.rst +++ b/Doc/library/asyncio-eventloops.rst @@ -35,7 +35,7 @@ .. class:: SelectorEventLoop Event loop based on the :mod:`selectors` module. Subclass of - :class:`BaseEventLoop`. + :class:`AbstractEventLoop`. Use the most efficient selector available on the platform. @@ -46,7 +46,7 @@ .. class:: ProactorEventLoop Proactor event loop for Windows using "I/O Completion Ports" aka IOCP. - Subclass of :class:`BaseEventLoop`. + Subclass of :class:`AbstractEventLoop`. Availability: Windows. @@ -76,11 +76,11 @@ Common limits of Windows event loops: -- :meth:`~BaseEventLoop.create_unix_connection` and - :meth:`~BaseEventLoop.create_unix_server` are not supported: the socket +- :meth:`~AbstractEventLoop.create_unix_connection` and + :meth:`~AbstractEventLoop.create_unix_server` are not supported: the socket family :data:`socket.AF_UNIX` is specific to UNIX -- :meth:`~BaseEventLoop.add_signal_handler` and - :meth:`~BaseEventLoop.remove_signal_handler` are not supported +- :meth:`~AbstractEventLoop.add_signal_handler` and + :meth:`~AbstractEventLoop.remove_signal_handler` are not supported - :meth:`EventLoopPolicy.set_child_watcher` is not supported. :class:`ProactorEventLoop` supports subprocesses. It has only one implementation to watch child processes, there is no need to configure it. @@ -89,19 +89,19 @@ - :class:`~selectors.SelectSelector` is used which only supports sockets and is limited to 512 sockets. -- :meth:`~BaseEventLoop.add_reader` and :meth:`~BaseEventLoop.add_writer` only +- :meth:`~AbstractEventLoop.add_reader` and :meth:`~AbstractEventLoop.add_writer` only accept file descriptors of sockets - Pipes are not supported - (ex: :meth:`~BaseEventLoop.connect_read_pipe`, - :meth:`~BaseEventLoop.connect_write_pipe`) + (ex: :meth:`~AbstractEventLoop.connect_read_pipe`, + :meth:`~AbstractEventLoop.connect_write_pipe`) - :ref:`Subprocesses ` are not supported - (ex: :meth:`~BaseEventLoop.subprocess_exec`, - :meth:`~BaseEventLoop.subprocess_shell`) + (ex: :meth:`~AbstractEventLoop.subprocess_exec`, + :meth:`~AbstractEventLoop.subprocess_shell`) :class:`ProactorEventLoop` specific limits: -- :meth:`~BaseEventLoop.create_datagram_endpoint` (UDP) is not supported -- :meth:`~BaseEventLoop.add_reader` and :meth:`~BaseEventLoop.add_writer` are +- :meth:`~AbstractEventLoop.create_datagram_endpoint` (UDP) is not supported +- :meth:`~AbstractEventLoop.add_reader` and :meth:`~AbstractEventLoop.add_writer` are not supported The resolution of the monotonic clock on Windows is usually around 15.6 msec. @@ -167,7 +167,7 @@ Get the event loop for the current context. - Returns an event loop object implementing the :class:`BaseEventLoop` + Returns an event loop object implementing the :class:`AbstractEventLoop` interface. Raises an exception in case no event loop has been set for the current diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -11,7 +11,7 @@ Transports are classes provided by :mod:`asyncio` in order to abstract various kinds of communication channels. You generally won't instantiate -a transport yourself; instead, you will call a :class:`BaseEventLoop` method +a transport yourself; instead, you will call a :class:`AbstractEventLoop` method which will create the transport and try to initiate the underlying communication channel, calling you back when it succeeds. @@ -475,7 +475,7 @@ TCP echo client protocol ------------------------ -TCP echo client using the :meth:`BaseEventLoop.create_connection` method, send +TCP echo client using the :meth:`AbstractEventLoop.create_connection` method, send data and wait until the connection is closed:: import asyncio @@ -506,10 +506,10 @@ loop.close() The event loop is running twice. The -:meth:`~BaseEventLoop.run_until_complete` method is preferred in this short +:meth:`~AbstractEventLoop.run_until_complete` method is preferred in this short example to raise an exception if the server is not listening, instead of having to write a short coroutine to handle the exception and stop the -running loop. At :meth:`~BaseEventLoop.run_until_complete` exit, the loop is +running loop. At :meth:`~AbstractEventLoop.run_until_complete` exit, the loop is no longer running, so there is no need to stop the loop in case of an error. .. seealso:: @@ -523,7 +523,7 @@ TCP echo server protocol ------------------------ -TCP echo server using the :meth:`BaseEventLoop.create_server` method, send back +TCP echo server using the :meth:`AbstractEventLoop.create_server` method, send back received data and close the connection:: import asyncio @@ -577,7 +577,7 @@ UDP echo client protocol ------------------------ -UDP echo client using the :meth:`BaseEventLoop.create_datagram_endpoint` +UDP echo client using the :meth:`AbstractEventLoop.create_datagram_endpoint` method, send data and close the transport when we received the answer:: import asyncio @@ -623,7 +623,7 @@ UDP echo server protocol ------------------------ -UDP echo server using the :meth:`BaseEventLoop.create_datagram_endpoint` +UDP echo server using the :meth:`AbstractEventLoop.create_datagram_endpoint` method, send back received data:: import asyncio @@ -660,7 +660,7 @@ --------------------------------------------------------- Wait until a socket receives data using the -:meth:`BaseEventLoop.create_connection` method with a protocol, and then close +:meth:`AbstractEventLoop.create_connection` method with a protocol, and then close the event loop :: import asyncio @@ -708,7 +708,7 @@ The :ref:`watch a file descriptor for read events ` example uses the low-level - :meth:`BaseEventLoop.add_reader` method to register the file descriptor of a + :meth:`AbstractEventLoop.add_reader` method to register the file descriptor of a socket. The :ref:`register an open socket to wait for data using streams diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -18,14 +18,14 @@ .. coroutinefunction:: open_connection(host=None, port=None, \*, loop=None, limit=None, \*\*kwds) - A wrapper for :meth:`~BaseEventLoop.create_connection()` returning a (reader, + A wrapper for :meth:`~AbstractEventLoop.create_connection()` returning a (reader, writer) pair. The reader returned is a :class:`StreamReader` instance; the writer is a :class:`StreamWriter` instance. The arguments are all the usual arguments to - :meth:`BaseEventLoop.create_connection` except *protocol_factory*; most + :meth:`AbstractEventLoop.create_connection` except *protocol_factory*; most common are positional host and port, with various optional keyword arguments following. @@ -38,7 +38,7 @@ .. coroutinefunction:: start_server(client_connected_cb, host=None, port=None, \*, loop=None, limit=None, \*\*kwds) Start a socket server, with a callback for each client connected. The return - value is the same as :meth:`~BaseEventLoop.create_server()`. + value is the same as :meth:`~AbstractEventLoop.create_server()`. The *client_connected_cb* parameter is called with two parameters: *client_reader*, *client_writer*. *client_reader* is a @@ -49,7 +49,7 @@ converted into a :class:`Task`. The rest of the arguments are all the usual arguments to - :meth:`~BaseEventLoop.create_server()` except *protocol_factory*; most + :meth:`~AbstractEventLoop.create_server()` except *protocol_factory*; most common are positional *host* and *port*, with various optional keyword arguments following. @@ -61,7 +61,7 @@ .. coroutinefunction:: open_unix_connection(path=None, \*, loop=None, limit=None, **kwds) - A wrapper for :meth:`~BaseEventLoop.create_unix_connection()` returning + A wrapper for :meth:`~AbstractEventLoop.create_unix_connection()` returning a (reader, writer) pair. See :func:`open_connection` for information about return value and other @@ -321,7 +321,7 @@ .. seealso:: The :ref:`TCP echo client protocol ` - example uses the :meth:`BaseEventLoop.create_connection` method. + example uses the :meth:`AbstractEventLoop.create_connection` method. .. _asyncio-tcp-echo-server-streams: @@ -366,7 +366,7 @@ .. seealso:: The :ref:`TCP echo server protocol ` - example uses the :meth:`BaseEventLoop.create_server` method. + example uses the :meth:`AbstractEventLoop.create_server` method. Get HTTP headers @@ -458,10 +458,10 @@ The :ref:`register an open socket to wait for data using a protocol ` example uses a low-level protocol created by the - :meth:`BaseEventLoop.create_connection` method. + :meth:`AbstractEventLoop.create_connection` method. The :ref:`watch a file descriptor for read events ` example uses the low-level - :meth:`BaseEventLoop.add_reader` method to register the file descriptor of a + :meth:`AbstractEventLoop.add_reader` method to register the file descriptor of a socket. diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -32,7 +32,7 @@ Create a subprocess. The *limit* parameter sets the buffer limit passed to the - :class:`StreamReader`. See :meth:`BaseEventLoop.subprocess_exec` for other + :class:`StreamReader`. See :meth:`AbstractEventLoop.subprocess_exec` for other parameters. Return a :class:`~asyncio.subprocess.Process` instance. @@ -44,7 +44,7 @@ Run the shell command *cmd*. The *limit* parameter sets the buffer limit passed to the - :class:`StreamReader`. See :meth:`BaseEventLoop.subprocess_shell` for other + :class:`StreamReader`. See :meth:`AbstractEventLoop.subprocess_shell` for other parameters. Return a :class:`~asyncio.subprocess.Process` instance. @@ -58,8 +58,8 @@ This function is a :ref:`coroutine `. -Use the :meth:`BaseEventLoop.connect_read_pipe` and -:meth:`BaseEventLoop.connect_write_pipe` methods to connect pipes. +Use the :meth:`AbstractEventLoop.connect_read_pipe` and +:meth:`AbstractEventLoop.connect_write_pipe` methods to connect pipes. Create a subprocess: low-level API using subprocess.Popen @@ -67,7 +67,7 @@ Run subprocesses asynchronously using the :mod:`subprocess` module. -.. coroutinemethod:: BaseEventLoop.subprocess_exec(protocol_factory, \*args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) +.. coroutinemethod:: AbstractEventLoop.subprocess_exec(protocol_factory, \*args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) Create a subprocess from one or more string arguments (character strings or bytes strings encoded to the :ref:`filesystem encoding @@ -87,19 +87,19 @@ * *stdin*: Either a file-like object representing the pipe to be connected to the subprocess's standard input stream using - :meth:`~BaseEventLoop.connect_write_pipe`, or the constant + :meth:`~AbstractEventLoop.connect_write_pipe`, or the constant :const:`subprocess.PIPE` (the default). By default a new pipe will be created and connected. * *stdout*: Either a file-like object representing the pipe to be connected to the subprocess's standard output stream using - :meth:`~BaseEventLoop.connect_read_pipe`, or the constant + :meth:`~AbstractEventLoop.connect_read_pipe`, or the constant :const:`subprocess.PIPE` (the default). By default a new pipe will be created and connected. * *stderr*: Either a file-like object representing the pipe to be connected to the subprocess's standard error stream using - :meth:`~BaseEventLoop.connect_read_pipe`, or one of the constants + :meth:`~AbstractEventLoop.connect_read_pipe`, or one of the constants :const:`subprocess.PIPE` (the default) or :const:`subprocess.STDOUT`. By default a new pipe will be created and connected. When :const:`subprocess.STDOUT` is specified, the subprocess's standard error @@ -116,7 +116,7 @@ See the constructor of the :class:`subprocess.Popen` class for parameters. -.. coroutinemethod:: BaseEventLoop.subprocess_shell(protocol_factory, cmd, \*, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) +.. coroutinemethod:: AbstractEventLoop.subprocess_shell(protocol_factory, cmd, \*, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) Create a subprocess from *cmd*, which is a character string or a bytes string encoded to the :ref:`filesystem encoding `, @@ -126,7 +126,7 @@ The *protocol_factory* must instanciate a subclass of the :class:`asyncio.SubprocessProtocol` class. - See :meth:`~BaseEventLoop.subprocess_exec` for more details about + See :meth:`~AbstractEventLoop.subprocess_exec` for more details about the remaining arguments. Returns a pair of ``(transport, protocol)``, where *transport* is an @@ -143,8 +143,8 @@ .. seealso:: - The :meth:`BaseEventLoop.connect_read_pipe` and - :meth:`BaseEventLoop.connect_write_pipe` methods. + The :meth:`AbstractEventLoop.connect_read_pipe` and + :meth:`AbstractEventLoop.connect_write_pipe` methods. Constants @@ -329,7 +329,7 @@ Example of a subprocess protocol using to get the output of a subprocess and to wait for the subprocess exit. The subprocess is created by the -:meth:`BaseEventLoop.subprocess_exec` method:: +:meth:`AbstractEventLoop.subprocess_exec` method:: import asyncio import sys diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -59,7 +59,7 @@ schedule its execution. There are two basic ways to start it running: call ``await coroutine`` or ``yield from coroutine`` from another coroutine (assuming the other coroutine is already running!), or schedule its execution -using the :func:`ensure_future` function or the :meth:`BaseEventLoop.create_task` +using the :func:`ensure_future` function or the :meth:`AbstractEventLoop.create_task` method. @@ -108,7 +108,7 @@ .. seealso:: The :ref:`Hello World with call_soon() ` - example uses the :meth:`BaseEventLoop.call_soon` method to schedule a + example uses the :meth:`AbstractEventLoop.call_soon` method to schedule a callback. @@ -151,7 +151,7 @@ The :ref:`display the current date with call_later() ` example uses a callback with the - :meth:`BaseEventLoop.call_later` method. + :meth:`AbstractEventLoop.call_later` method. Example: Chain coroutines @@ -182,12 +182,12 @@ .. image:: tulip_coro.png :align: center -The "Task" is created by the :meth:`BaseEventLoop.run_until_complete` method +The "Task" is created by the :meth:`AbstractEventLoop.run_until_complete` method when it gets a coroutine object instead of a task. The diagram shows the control flow, it does not describe exactly how things work internally. For example, the sleep coroutine creates an internal future -which uses :meth:`BaseEventLoop.call_later` to wake up the task in 1 second. +which uses :meth:`AbstractEventLoop.call_later` to wake up the task in 1 second. InvalidStateError @@ -223,7 +223,7 @@ raise an exception when the future isn't done yet. - Callbacks registered with :meth:`add_done_callback` are always called - via the event loop's :meth:`~BaseEventLoop.call_soon_threadsafe`. + via the event loop's :meth:`~AbstractEventLoop.call_soon_threadsafe`. - This class is not compatible with the :func:`~concurrent.futures.wait` and :func:`~concurrent.futures.as_completed` functions in the @@ -273,7 +273,7 @@ The callback is called with a single argument - the future object. If the future is already done when this is called, the callback is scheduled - with :meth:`~BaseEventLoop.call_soon`. + with :meth:`~AbstractEventLoop.call_soon`. :ref:`Use functools.partial to pass parameters to the callback `. For example, @@ -323,11 +323,11 @@ The coroutine function is responsible for the computation (which takes 1 second) and it stores the result into the future. The -:meth:`~BaseEventLoop.run_until_complete` method waits for the completion of +:meth:`~AbstractEventLoop.run_until_complete` method waits for the completion of the future. .. note:: - The :meth:`~BaseEventLoop.run_until_complete` method uses internally the + The :meth:`~AbstractEventLoop.run_until_complete` method uses internally the :meth:`~Future.add_done_callback` method to be notified when the future is done. @@ -396,7 +396,7 @@ logged: see :ref:`Pending task destroyed `. Don't directly create :class:`Task` instances: use the :func:`ensure_future` - function or the :meth:`BaseEventLoop.create_task` method. + function or the :meth:`AbstractEventLoop.create_task` method. This class is :ref:`not thread safe `. @@ -546,7 +546,7 @@ .. seealso:: - The :meth:`BaseEventLoop.create_task` method. + The :meth:`AbstractEventLoop.create_task` method. .. function:: async(coro_or_future, \*, loop=None) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 8 23:20:40 2016 From: python-checkins at python.org (steven.daprano) Date: Tue, 09 Aug 2016 03:20:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Tighten_up_test_of_harmoni?= =?utf-8?q?c_mean_on_a_single_value=2E?= Message-ID: <20160809032040.13076.71862.47B044A7@psf.io> https://hg.python.org/cpython/rev/1fab715bbd43 changeset: 102576:1fab715bbd43 user: Steven D'Aprano date: Tue Aug 09 13:19:48 2016 +1000 summary: Tighten up test of harmonic mean on a single value. files: Lib/test/test_statistics.py | 5 +---- 1 files changed, 1 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_statistics.py b/Lib/test/test_statistics.py --- a/Lib/test/test_statistics.py +++ b/Lib/test/test_statistics.py @@ -1469,10 +1469,7 @@ def test_singleton_lists(self): # Test that harmonic mean([x]) returns (approximately) x. for x in range(1, 101): - if x in (49, 93, 98, 99): - self.assertApproxEqual(self.func([x]), x, tol=2e-14) - else: - self.assertEqual(self.func([x]), x) + self.assertEqual(self.func([x]), x) def test_decimals_exact(self): # Test harmonic mean with some carefully chosen Decimals. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 8 23:20:40 2016 From: python-checkins at python.org (steven.daprano) Date: Tue, 09 Aug 2016 03:20:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Automated_merge_with_ssh=3A//hg=2Epython=2Eorg/cpython?= Message-ID: <20160809032040.30864.12656.B374DECD@psf.io> https://hg.python.org/cpython/rev/2478b49aad21 changeset: 102575:2478b49aad21 parent: 102573:d69f782d642d parent: 102574:791acb2cd025 user: Steven D'Aprano date: Tue Aug 09 12:49:20 2016 +1000 summary: Automated merge with ssh://hg.python.org/cpython files: Lib/statistics.py | 66 +++++++++- Lib/test/test_statistics.py | 159 ++++++++++++++++++++++- 2 files changed, 211 insertions(+), 14 deletions(-) diff --git a/Lib/statistics.py b/Lib/statistics.py --- a/Lib/statistics.py +++ b/Lib/statistics.py @@ -28,6 +28,7 @@ Function Description ================== ============================================= mean Arithmetic mean (average) of data. +harmonic_mean Harmonic mean of data. median Median (middle value) of data. median_low Low median of data. median_high High median of data. @@ -95,16 +96,17 @@ __all__ = [ 'StatisticsError', 'pstdev', 'pvariance', 'stdev', 'variance', 'median', 'median_low', 'median_high', 'median_grouped', - 'mean', 'mode', + 'mean', 'mode', 'harmonic_mean', ] - import collections +import decimal import math +import numbers from fractions import Fraction from decimal import Decimal -from itertools import groupby +from itertools import groupby, chain from bisect import bisect_left, bisect_right @@ -135,7 +137,8 @@ Some sources of round-off error will be avoided: - >>> _sum([1e50, 1, -1e50] * 1000) # Built-in sum returns zero. + # Built-in sum returns zero. + >>> _sum([1e50, 1, -1e50] * 1000) (, Fraction(1000, 1), 3000) Fractions and Decimals are also supported: @@ -291,6 +294,15 @@ return i-1 raise ValueError + +def _fail_neg(values, errmsg='negative value'): + """Iterate over values, failing if any are less than zero.""" + for x in values: + if x < 0: + raise StatisticsError(errmsg) + yield x + + # === Measures of central tendency (averages) === def mean(data): @@ -319,6 +331,52 @@ return _convert(total/n, T) +def harmonic_mean(data): + """Return the harmonic mean of data. + + The harmonic mean, sometimes called the subcontrary mean, is the + reciprocal of the arithmetic mean of the reciprocals of the data, + and is often appropriate when averaging quantities which are rates + or ratios, for example speeds. Example: + + Suppose an investor purchases an equal value of shares in each of + three companies, with P/E (price/earning) ratios of 2.5, 3 and 10. + What is the average P/E ratio for the investor's portfolio? + + >>> harmonic_mean([2.5, 3, 10]) # For an equal investment portfolio. + 3.6 + + Using the arithmetic mean would give an average of about 5.167, which + is too high. + + If ``data`` is empty, or any element is less than zero, + ``harmonic_mean`` will raise ``StatisticsError``. + """ + # For a justification for using harmonic mean for P/E ratios, see + # http://fixthepitch.pellucid.com/comps-analysis-the-missing-harmony-of-summary-statistics/ + # http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2621087 + if iter(data) is data: + data = list(data) + errmsg = 'harmonic mean does not support negative values' + n = len(data) + if n < 1: + raise StatisticsError('harmonic_mean requires at least one data point') + elif n == 1: + x = data[0] + if isinstance(x, (numbers.Real, Decimal)): + if x < 0: + raise StatisticsError(errmsg) + return x + else: + raise TypeError('unsupported type') + try: + T, total, count = _sum(1/x for x in _fail_neg(data, errmsg)) + except ZeroDivisionError: + return 0 + assert count == n + return _convert(n/total, T) + + # FIXME: investigate ways to calculate medians without sorting? Quickselect? def median(data): """Return the median (middle value) of numeric data. diff --git a/Lib/test/test_statistics.py b/Lib/test/test_statistics.py --- a/Lib/test/test_statistics.py +++ b/Lib/test/test_statistics.py @@ -21,6 +21,10 @@ # === Helper functions and class === +def sign(x): + """Return -1.0 for negatives, including -0.0, otherwise +1.0.""" + return math.copysign(1, x) + def _nan_equal(a, b): """Return True if a and b are both the same kind of NAN. @@ -264,6 +268,13 @@ # === Test the helpers === # ======================== +class TestSign(unittest.TestCase): + """Test that the helper function sign() works correctly.""" + def testZeroes(self): + # Test that signed zeroes report their sign correctly. + self.assertEqual(sign(0.0), +1) + self.assertEqual(sign(-0.0), -1) + # --- Tests for approx_equal --- @@ -659,7 +670,7 @@ @unittest.skipIf(sys.flags.optimize >= 2, "Docstrings are omitted with -OO and above") def test_doc_tests(self): - failed, tried = doctest.testmod(statistics) + failed, tried = doctest.testmod(statistics, optionflags=doctest.ELLIPSIS) self.assertGreater(tried, 0) self.assertEqual(failed, 0) @@ -971,6 +982,34 @@ self.assertTrue(_nan_equal(x, nan)) +class FailNegTest(unittest.TestCase): + """Test _fail_neg private function.""" + + def test_pass_through(self): + # Test that values are passed through unchanged. + values = [1, 2.0, Fraction(3), Decimal(4)] + new = list(statistics._fail_neg(values)) + self.assertEqual(values, new) + + def test_negatives_raise(self): + # Test that negatives raise an exception. + for x in [1, 2.0, Fraction(3), Decimal(4)]: + seq = [-x] + it = statistics._fail_neg(seq) + self.assertRaises(statistics.StatisticsError, next, it) + + def test_error_msg(self): + # Test that a given error message is used. + msg = "badness #%d" % random.randint(10000, 99999) + try: + next(statistics._fail_neg([-1], msg)) + except statistics.StatisticsError as e: + errmsg = e.args[0] + else: + self.fail("expected exception, but it didn't happen") + self.assertEqual(errmsg, msg) + + # === Tests for public functions === class UnivariateCommonMixin: @@ -1082,13 +1121,13 @@ Not all tests to do with types need go in this class. Only those that rely on the function returning the same type as its input data. """ - def test_types_conserved(self): - # Test that functions keeps the same type as their data points. - # (Excludes mixed data types.) This only tests the type of the return - # result, not the value. + def prepare_types_for_conservation_test(self): + """Return the types which are expected to be conserved.""" class MyFloat(float): def __truediv__(self, other): return type(self)(super().__truediv__(other)) + def __rtruediv__(self, other): + return type(self)(super().__rtruediv__(other)) def __sub__(self, other): return type(self)(super().__sub__(other)) def __rsub__(self, other): @@ -1098,9 +1137,14 @@ def __add__(self, other): return type(self)(super().__add__(other)) __radd__ = __add__ + return (float, Decimal, Fraction, MyFloat) + def test_types_conserved(self): + # Test that functions keeps the same type as their data points. + # (Excludes mixed data types.) This only tests the type of the return + # result, not the value. data = self.prepare_data() - for kind in (float, Decimal, Fraction, MyFloat): + for kind in self.prepare_types_for_conservation_test(): d = [kind(x) for x in data] result = self.func(d) self.assertIs(type(result), kind) @@ -1275,12 +1319,16 @@ for x in (23, 42.5, 1.3e15, Fraction(15, 19), Decimal('0.28')): self.assertEqual(self.func([x]), x) + def prepare_values_for_repeated_single_test(self): + return (3.5, 17, 2.5e15, Fraction(61, 67), Decimal('4.9712')) + def test_repeated_single_value(self): # The average of a single repeated value is the value itself. - for x in (3.5, 17, 2.5e15, Fraction(61, 67), Decimal('4.9712')): + for x in self.prepare_values_for_repeated_single_test(): for count in (2, 5, 10, 20): - data = [x]*count - self.assertEqual(self.func(data), x) + with self.subTest(x=x, count=count): + data = [x]*count + self.assertEqual(self.func(data), x) class TestMean(NumericTestCase, AverageMixin, UnivariateTypeMixin): @@ -1304,7 +1352,7 @@ self.assertEqual(self.func(data), 22.015625) def test_decimals(self): - # Test mean with ints. + # Test mean with Decimals. D = Decimal data = [D("1.634"), D("2.517"), D("3.912"), D("4.072"), D("5.813")] random.shuffle(data) @@ -1379,6 +1427,97 @@ self.assertEqual(statistics.mean([tiny]*n), tiny) +class TestHarmonicMean(NumericTestCase, AverageMixin, UnivariateTypeMixin): + def setUp(self): + self.func = statistics.harmonic_mean + + def prepare_data(self): + # Override mixin method. + values = super().prepare_data() + values.remove(0) + return values + + def prepare_values_for_repeated_single_test(self): + # Override mixin method. + return (3.5, 17, 2.5e15, Fraction(61, 67), Decimal('4.125')) + + def test_zero(self): + # Test that harmonic mean returns zero when given zero. + values = [1, 0, 2] + self.assertEqual(self.func(values), 0) + + def test_negative_error(self): + # Test that harmonic mean raises when given a negative value. + exc = statistics.StatisticsError + for values in ([-1], [1, -2, 3]): + with self.subTest(values=values): + self.assertRaises(exc, self.func, values) + + def test_ints(self): + # Test harmonic mean with ints. + data = [2, 4, 4, 8, 16, 16] + random.shuffle(data) + self.assertEqual(self.func(data), 6*4/5) + + def test_floats_exact(self): + # Test harmonic mean with some carefully chosen floats. + data = [1/8, 1/4, 1/4, 1/2, 1/2] + random.shuffle(data) + self.assertEqual(self.func(data), 1/4) + self.assertEqual(self.func([0.25, 0.5, 1.0, 1.0]), 0.5) + + def test_singleton_lists(self): + # Test that harmonic mean([x]) returns (approximately) x. + for x in range(1, 101): + if x in (49, 93, 98, 99): + self.assertApproxEqual(self.func([x]), x, tol=2e-14) + else: + self.assertEqual(self.func([x]), x) + + def test_decimals_exact(self): + # Test harmonic mean with some carefully chosen Decimals. + D = Decimal + self.assertEqual(self.func([D(15), D(30), D(60), D(60)]), D(30)) + data = [D("0.05"), D("0.10"), D("0.20"), D("0.20")] + random.shuffle(data) + self.assertEqual(self.func(data), D("0.10")) + data = [D("1.68"), D("0.32"), D("5.94"), D("2.75")] + random.shuffle(data) + self.assertEqual(self.func(data), D(66528)/70723) + + def test_fractions(self): + # Test harmonic mean with Fractions. + F = Fraction + data = [F(1, 2), F(2, 3), F(3, 4), F(4, 5), F(5, 6), F(6, 7), F(7, 8)] + random.shuffle(data) + self.assertEqual(self.func(data), F(7*420, 4029)) + + def test_inf(self): + # Test harmonic mean with infinity. + values = [2.0, float('inf'), 1.0] + self.assertEqual(self.func(values), 2.0) + + def test_nan(self): + # Test harmonic mean with NANs. + values = [2.0, float('nan'), 1.0] + self.assertTrue(math.isnan(self.func(values))) + + def test_multiply_data_points(self): + # Test multiplying every data point by a constant. + c = 111 + data = [3.4, 4.5, 4.9, 6.7, 6.8, 7.2, 8.0, 8.1, 9.4] + expected = self.func(data)*c + result = self.func([x*c for x in data]) + self.assertEqual(result, expected) + + def test_doubled_data(self): + # Harmonic mean of [a,b...z] should be same as for [a,a,b,b...z,z]. + data = [random.uniform(1, 5) for _ in range(1000)] + expected = self.func(data) + actual = self.func(data*2) + self.assertApproxEqual(actual, expected) + + class TestMedian(NumericTestCase, AverageMixin): # Common tests for median and all median.* functions. def setUp(self): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 8 23:20:40 2016 From: python-checkins at python.org (steven.daprano) Date: Tue, 09 Aug 2016 03:20:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_harmonic_mean_and_test?= =?utf-8?q?s=2E?= Message-ID: <20160809032040.52595.6701.E1AE8A69@psf.io> https://hg.python.org/cpython/rev/791acb2cd025 changeset: 102574:791acb2cd025 parent: 102570:535f88ad80d8 user: Steven D'Aprano date: Tue Aug 09 12:49:01 2016 +1000 summary: Add harmonic mean and tests. files: Lib/statistics.py | 66 +++++++++- Lib/test/test_statistics.py | 159 ++++++++++++++++++++++- 2 files changed, 211 insertions(+), 14 deletions(-) diff --git a/Lib/statistics.py b/Lib/statistics.py --- a/Lib/statistics.py +++ b/Lib/statistics.py @@ -28,6 +28,7 @@ Function Description ================== ============================================= mean Arithmetic mean (average) of data. +harmonic_mean Harmonic mean of data. median Median (middle value) of data. median_low Low median of data. median_high High median of data. @@ -95,16 +96,17 @@ __all__ = [ 'StatisticsError', 'pstdev', 'pvariance', 'stdev', 'variance', 'median', 'median_low', 'median_high', 'median_grouped', - 'mean', 'mode', + 'mean', 'mode', 'harmonic_mean', ] - import collections +import decimal import math +import numbers from fractions import Fraction from decimal import Decimal -from itertools import groupby +from itertools import groupby, chain from bisect import bisect_left, bisect_right @@ -135,7 +137,8 @@ Some sources of round-off error will be avoided: - >>> _sum([1e50, 1, -1e50] * 1000) # Built-in sum returns zero. + # Built-in sum returns zero. + >>> _sum([1e50, 1, -1e50] * 1000) (, Fraction(1000, 1), 3000) Fractions and Decimals are also supported: @@ -291,6 +294,15 @@ return i-1 raise ValueError + +def _fail_neg(values, errmsg='negative value'): + """Iterate over values, failing if any are less than zero.""" + for x in values: + if x < 0: + raise StatisticsError(errmsg) + yield x + + # === Measures of central tendency (averages) === def mean(data): @@ -319,6 +331,52 @@ return _convert(total/n, T) +def harmonic_mean(data): + """Return the harmonic mean of data. + + The harmonic mean, sometimes called the subcontrary mean, is the + reciprocal of the arithmetic mean of the reciprocals of the data, + and is often appropriate when averaging quantities which are rates + or ratios, for example speeds. Example: + + Suppose an investor purchases an equal value of shares in each of + three companies, with P/E (price/earning) ratios of 2.5, 3 and 10. + What is the average P/E ratio for the investor's portfolio? + + >>> harmonic_mean([2.5, 3, 10]) # For an equal investment portfolio. + 3.6 + + Using the arithmetic mean would give an average of about 5.167, which + is too high. + + If ``data`` is empty, or any element is less than zero, + ``harmonic_mean`` will raise ``StatisticsError``. + """ + # For a justification for using harmonic mean for P/E ratios, see + # http://fixthepitch.pellucid.com/comps-analysis-the-missing-harmony-of-summary-statistics/ + # http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2621087 + if iter(data) is data: + data = list(data) + errmsg = 'harmonic mean does not support negative values' + n = len(data) + if n < 1: + raise StatisticsError('harmonic_mean requires at least one data point') + elif n == 1: + x = data[0] + if isinstance(x, (numbers.Real, Decimal)): + if x < 0: + raise StatisticsError(errmsg) + return x + else: + raise TypeError('unsupported type') + try: + T, total, count = _sum(1/x for x in _fail_neg(data, errmsg)) + except ZeroDivisionError: + return 0 + assert count == n + return _convert(n/total, T) + + # FIXME: investigate ways to calculate medians without sorting? Quickselect? def median(data): """Return the median (middle value) of numeric data. diff --git a/Lib/test/test_statistics.py b/Lib/test/test_statistics.py --- a/Lib/test/test_statistics.py +++ b/Lib/test/test_statistics.py @@ -21,6 +21,10 @@ # === Helper functions and class === +def sign(x): + """Return -1.0 for negatives, including -0.0, otherwise +1.0.""" + return math.copysign(1, x) + def _nan_equal(a, b): """Return True if a and b are both the same kind of NAN. @@ -264,6 +268,13 @@ # === Test the helpers === # ======================== +class TestSign(unittest.TestCase): + """Test that the helper function sign() works correctly.""" + def testZeroes(self): + # Test that signed zeroes report their sign correctly. + self.assertEqual(sign(0.0), +1) + self.assertEqual(sign(-0.0), -1) + # --- Tests for approx_equal --- @@ -659,7 +670,7 @@ @unittest.skipIf(sys.flags.optimize >= 2, "Docstrings are omitted with -OO and above") def test_doc_tests(self): - failed, tried = doctest.testmod(statistics) + failed, tried = doctest.testmod(statistics, optionflags=doctest.ELLIPSIS) self.assertGreater(tried, 0) self.assertEqual(failed, 0) @@ -971,6 +982,34 @@ self.assertTrue(_nan_equal(x, nan)) +class FailNegTest(unittest.TestCase): + """Test _fail_neg private function.""" + + def test_pass_through(self): + # Test that values are passed through unchanged. + values = [1, 2.0, Fraction(3), Decimal(4)] + new = list(statistics._fail_neg(values)) + self.assertEqual(values, new) + + def test_negatives_raise(self): + # Test that negatives raise an exception. + for x in [1, 2.0, Fraction(3), Decimal(4)]: + seq = [-x] + it = statistics._fail_neg(seq) + self.assertRaises(statistics.StatisticsError, next, it) + + def test_error_msg(self): + # Test that a given error message is used. + msg = "badness #%d" % random.randint(10000, 99999) + try: + next(statistics._fail_neg([-1], msg)) + except statistics.StatisticsError as e: + errmsg = e.args[0] + else: + self.fail("expected exception, but it didn't happen") + self.assertEqual(errmsg, msg) + + # === Tests for public functions === class UnivariateCommonMixin: @@ -1082,13 +1121,13 @@ Not all tests to do with types need go in this class. Only those that rely on the function returning the same type as its input data. """ - def test_types_conserved(self): - # Test that functions keeps the same type as their data points. - # (Excludes mixed data types.) This only tests the type of the return - # result, not the value. + def prepare_types_for_conservation_test(self): + """Return the types which are expected to be conserved.""" class MyFloat(float): def __truediv__(self, other): return type(self)(super().__truediv__(other)) + def __rtruediv__(self, other): + return type(self)(super().__rtruediv__(other)) def __sub__(self, other): return type(self)(super().__sub__(other)) def __rsub__(self, other): @@ -1098,9 +1137,14 @@ def __add__(self, other): return type(self)(super().__add__(other)) __radd__ = __add__ + return (float, Decimal, Fraction, MyFloat) + def test_types_conserved(self): + # Test that functions keeps the same type as their data points. + # (Excludes mixed data types.) This only tests the type of the return + # result, not the value. data = self.prepare_data() - for kind in (float, Decimal, Fraction, MyFloat): + for kind in self.prepare_types_for_conservation_test(): d = [kind(x) for x in data] result = self.func(d) self.assertIs(type(result), kind) @@ -1275,12 +1319,16 @@ for x in (23, 42.5, 1.3e15, Fraction(15, 19), Decimal('0.28')): self.assertEqual(self.func([x]), x) + def prepare_values_for_repeated_single_test(self): + return (3.5, 17, 2.5e15, Fraction(61, 67), Decimal('4.9712')) + def test_repeated_single_value(self): # The average of a single repeated value is the value itself. - for x in (3.5, 17, 2.5e15, Fraction(61, 67), Decimal('4.9712')): + for x in self.prepare_values_for_repeated_single_test(): for count in (2, 5, 10, 20): - data = [x]*count - self.assertEqual(self.func(data), x) + with self.subTest(x=x, count=count): + data = [x]*count + self.assertEqual(self.func(data), x) class TestMean(NumericTestCase, AverageMixin, UnivariateTypeMixin): @@ -1304,7 +1352,7 @@ self.assertEqual(self.func(data), 22.015625) def test_decimals(self): - # Test mean with ints. + # Test mean with Decimals. D = Decimal data = [D("1.634"), D("2.517"), D("3.912"), D("4.072"), D("5.813")] random.shuffle(data) @@ -1379,6 +1427,97 @@ self.assertEqual(statistics.mean([tiny]*n), tiny) +class TestHarmonicMean(NumericTestCase, AverageMixin, UnivariateTypeMixin): + def setUp(self): + self.func = statistics.harmonic_mean + + def prepare_data(self): + # Override mixin method. + values = super().prepare_data() + values.remove(0) + return values + + def prepare_values_for_repeated_single_test(self): + # Override mixin method. + return (3.5, 17, 2.5e15, Fraction(61, 67), Decimal('4.125')) + + def test_zero(self): + # Test that harmonic mean returns zero when given zero. + values = [1, 0, 2] + self.assertEqual(self.func(values), 0) + + def test_negative_error(self): + # Test that harmonic mean raises when given a negative value. + exc = statistics.StatisticsError + for values in ([-1], [1, -2, 3]): + with self.subTest(values=values): + self.assertRaises(exc, self.func, values) + + def test_ints(self): + # Test harmonic mean with ints. + data = [2, 4, 4, 8, 16, 16] + random.shuffle(data) + self.assertEqual(self.func(data), 6*4/5) + + def test_floats_exact(self): + # Test harmonic mean with some carefully chosen floats. + data = [1/8, 1/4, 1/4, 1/2, 1/2] + random.shuffle(data) + self.assertEqual(self.func(data), 1/4) + self.assertEqual(self.func([0.25, 0.5, 1.0, 1.0]), 0.5) + + def test_singleton_lists(self): + # Test that harmonic mean([x]) returns (approximately) x. + for x in range(1, 101): + if x in (49, 93, 98, 99): + self.assertApproxEqual(self.func([x]), x, tol=2e-14) + else: + self.assertEqual(self.func([x]), x) + + def test_decimals_exact(self): + # Test harmonic mean with some carefully chosen Decimals. + D = Decimal + self.assertEqual(self.func([D(15), D(30), D(60), D(60)]), D(30)) + data = [D("0.05"), D("0.10"), D("0.20"), D("0.20")] + random.shuffle(data) + self.assertEqual(self.func(data), D("0.10")) + data = [D("1.68"), D("0.32"), D("5.94"), D("2.75")] + random.shuffle(data) + self.assertEqual(self.func(data), D(66528)/70723) + + def test_fractions(self): + # Test harmonic mean with Fractions. + F = Fraction + data = [F(1, 2), F(2, 3), F(3, 4), F(4, 5), F(5, 6), F(6, 7), F(7, 8)] + random.shuffle(data) + self.assertEqual(self.func(data), F(7*420, 4029)) + + def test_inf(self): + # Test harmonic mean with infinity. + values = [2.0, float('inf'), 1.0] + self.assertEqual(self.func(values), 2.0) + + def test_nan(self): + # Test harmonic mean with NANs. + values = [2.0, float('nan'), 1.0] + self.assertTrue(math.isnan(self.func(values))) + + def test_multiply_data_points(self): + # Test multiplying every data point by a constant. + c = 111 + data = [3.4, 4.5, 4.9, 6.7, 6.8, 7.2, 8.0, 8.1, 9.4] + expected = self.func(data)*c + result = self.func([x*c for x in data]) + self.assertEqual(result, expected) + + def test_doubled_data(self): + # Harmonic mean of [a,b...z] should be same as for [a,a,b,b...z,z]. + data = [random.uniform(1, 5) for _ in range(1000)] + expected = self.func(data) + actual = self.func(data*2) + self.assertApproxEqual(actual, expected) + + class TestMedian(NumericTestCase, AverageMixin): # Common tests for median and all median.* functions. def setUp(self): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 9 00:18:49 2016 From: python-checkins at python.org (steven.daprano) Date: Tue, 09 Aug 2016 04:18:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue27181_add_geometric_m?= =?utf-8?q?ean=2E?= Message-ID: <20160809041848.130563.31157.D0BE4998@psf.io> https://hg.python.org/cpython/rev/9eb5edfcf604 changeset: 102577:9eb5edfcf604 user: Steven D'Aprano date: Tue Aug 09 13:58:10 2016 +1000 summary: Issue27181 add geometric mean. files: Lib/statistics.py | 267 ++++++++++++++++++++++ Lib/test/test_statistics.py | 285 ++++++++++++++++++++++++ 2 files changed, 552 insertions(+), 0 deletions(-) diff --git a/Lib/statistics.py b/Lib/statistics.py --- a/Lib/statistics.py +++ b/Lib/statistics.py @@ -303,6 +303,230 @@ yield x +class _nroot_NS: + """Hands off! Don't touch! + + Everything inside this namespace (class) is an even-more-private + implementation detail of the private _nth_root function. + """ + # This class exists only to be used as a namespace, for convenience + # of being able to keep the related functions together, and to + # collapse the group in an editor. If this were C# or C++, I would + # use a Namespace, but the closest Python has is a class. + # + # FIXME possibly move this out into a separate module? + # That feels like overkill, and may encourage people to treat it as + # a public feature. + def __init__(self): + raise TypeError('namespace only, do not instantiate') + + def nth_root(x, n): + """Return the positive nth root of numeric x. + + This may be more accurate than ** or pow(): + + >>> math.pow(1000, 1.0/3) #doctest:+SKIP + 9.999999999999998 + + >>> _nth_root(1000, 3) + 10.0 + >>> _nth_root(11**5, 5) + 11.0 + >>> _nth_root(2, 12) + 1.0594630943592953 + + """ + if not isinstance(n, int): + raise TypeError('degree n must be an int') + if n < 2: + raise ValueError('degree n must be 2 or more') + if isinstance(x, decimal.Decimal): + return _nroot_NS.decimal_nroot(x, n) + elif isinstance(x, numbers.Real): + return _nroot_NS.float_nroot(x, n) + else: + raise TypeError('expected a number, got %s') % type(x).__name__ + + def float_nroot(x, n): + """Handle nth root of Reals, treated as a float.""" + assert isinstance(n, int) and n > 1 + if x < 0: + if n%2 == 0: + raise ValueError('domain error: even root of negative number') + else: + return -_nroot_NS.nroot(-x, n) + elif x == 0: + return math.copysign(0.0, x) + elif x > 0: + try: + isinfinity = math.isinf(x) + except OverflowError: + return _nroot_NS.bignum_nroot(x, n) + else: + if isinfinity: + return float('inf') + else: + return _nroot_NS.nroot(x, n) + else: + assert math.isnan(x) + return float('nan') + + def nroot(x, n): + """Calculate x**(1/n), then improve the answer.""" + # This uses math.pow() to calculate an initial guess for the root, + # then uses the iterated nroot algorithm to improve it. + # + # By my testing, about 8% of the time the iterated algorithm ends + # up converging to a result which is less accurate than the initial + # guess. [FIXME: is this still true?] In that case, we use the + # guess instead of the "improved" value. This way, we're never + # less accurate than math.pow(). + r1 = math.pow(x, 1.0/n) + eps1 = abs(r1**n - x) + if eps1 == 0.0: + # r1 is the exact root, so we're done. By my testing, this + # occurs about 80% of the time for x < 1 and 30% of the + # time for x > 1. + return r1 + else: + try: + r2 = _nroot_NS.iterated_nroot(x, n, r1) + except RuntimeError: + return r1 + else: + eps2 = abs(r2**n - x) + if eps1 < eps2: + return r1 + return r2 + + def iterated_nroot(a, n, g): + """Return the nth root of a, starting with guess g. + + This is a special case of Newton's Method. + https://en.wikipedia.org/wiki/Nth_root_algorithm + """ + np = n - 1 + def iterate(r): + try: + return (np*r + a/math.pow(r, np))/n + except OverflowError: + # If r is large enough, r**np may overflow. If that + # happens, r**-np will be small, but not necessarily zero. + return (np*r + a*math.pow(r, -np))/n + # With a good guess, such as g = a**(1/n), this will converge in + # only a few iterations. However a poor guess can take thousands + # of iterations to converge, if at all. We guard against poor + # guesses by setting an upper limit to the number of iterations. + r1 = g + r2 = iterate(g) + for i in range(1000): + if r1 == r2: + break + # Use Floyd's cycle-finding algorithm to avoid being trapped + # in a cycle. + # https://en.wikipedia.org/wiki/Cycle_detection#Tortoise_and_hare + r1 = iterate(r1) + r2 = iterate(iterate(r2)) + else: + # If the guess is particularly bad, the above may fail to + # converge in any reasonable time. + raise RuntimeError('nth-root failed to converge') + return r2 + + def decimal_nroot(x, n): + """Handle nth root of Decimals.""" + assert isinstance(x, decimal.Decimal) + assert isinstance(n, int) + if x.is_snan(): + # Signalling NANs always raise. + raise decimal.InvalidOperation('nth-root of snan') + if x.is_qnan(): + # Quiet NANs only raise if the context is set to raise, + # otherwise return a NAN. + ctx = decimal.getcontext() + if ctx.traps[decimal.InvalidOperation]: + raise decimal.InvalidOperation('nth-root of nan') + else: + # Preserve the input NAN. + return x + if x.is_infinite(): + return x + # FIXME this hasn't had the extensive testing of the float + # version _iterated_nroot so there's possibly some buggy + # corner cases buried in here. Can it overflow? Fail to + # converge or get trapped in a cycle? Converge to a less + # accurate root? + np = n - 1 + def iterate(r): + return (np*r + x/r**np)/n + r0 = x**(decimal.Decimal(1)/n) + assert isinstance(r0, decimal.Decimal) + r1 = iterate(r0) + while True: + if r1 == r0: + return r1 + r0, r1 = r1, iterate(r1) + + def bignum_nroot(x, n): + """Return the nth root of a positive huge number.""" + assert x > 0 + # I state without proof that ??x ? ??2???(x//2) + # and that for sufficiently big x the error is acceptible. + # We now halve x until it is small enough to get the root. + m = 0 + while True: + x //= 2 + m += 1 + try: + y = float(x) + except OverflowError: + continue + break + a = _nroot_NS.nroot(y, n) + # At this point, we want the nth-root of 2**m, or 2**(m/n). + # We can write that as 2**(q + r/n) = 2**q * ??2**r where q = m//n. + q, r = divmod(m, n) + b = 2**q * _nroot_NS.nroot(2**r, n) + return a * b + + +# This is the (private) function for calculating nth roots: +_nth_root = _nroot_NS.nth_root +assert type(_nth_root) is type(lambda: None) + + +def _product(values): + """Return product of values as (exponent, mantissa).""" + errmsg = 'mixed Decimal and float is not supported' + prod = 1 + for x in values: + if isinstance(x, float): + break + prod *= x + else: + return (0, prod) + if isinstance(prod, Decimal): + raise TypeError(errmsg) + # Since floats can overflow easily, we calculate the product as a + # sort of poor-man's BigFloat. Given that: + # + # x = 2**p * m # p == power or exponent (scale), m = mantissa + # + # we can calculate the product of two (or more) x values as: + # + # x1*x2 = 2**p1*m1 * 2**p2*m2 = 2**(p1+p2)*(m1*m2) + # + mant, scale = 1, 0 #math.frexp(prod) # FIXME + for y in chain([x], values): + if isinstance(y, Decimal): + raise TypeError(errmsg) + m1, e1 = math.frexp(y) + m2, e2 = math.frexp(mant) + scale += (e1 + e2) + mant = m1*m2 + return (scale, mant) + + # === Measures of central tendency (averages) === def mean(data): @@ -331,6 +555,49 @@ return _convert(total/n, T) +def geometric_mean(data): + """Return the geometric mean of data. + + The geometric mean is appropriate when averaging quantities which + are multiplied together rather than added, for example growth rates. + Suppose an investment grows by 10% in the first year, falls by 5% in + the second, then grows by 12% in the third, what is the average rate + of growth over the three years? + + >>> geometric_mean([1.10, 0.95, 1.12]) + 1.0538483123382172 + + giving an average growth of 5.385%. Using the arithmetic mean will + give approximately 5.667%, which is too high. + + ``StatisticsError`` will be raised if ``data`` is empty, or any + element is less than zero. + """ + if iter(data) is data: + data = list(data) + errmsg = 'geometric mean does not support negative values' + n = len(data) + if n < 1: + raise StatisticsError('geometric_mean requires at least one data point') + elif n == 1: + x = data[0] + if isinstance(g, (numbers.Real, Decimal)): + if x < 0: + raise StatisticsError(errmsg) + return x + else: + raise TypeError('unsupported type') + else: + scale, prod = _product(_fail_neg(data, errmsg)) + r = _nth_root(prod, n) + if scale: + p, q = divmod(scale, n) + s = 2**p * _nth_root(2**q, n) + else: + s = 1 + return s*r + + def harmonic_mean(data): """Return the harmonic mean of data. diff --git a/Lib/test/test_statistics.py b/Lib/test/test_statistics.py --- a/Lib/test/test_statistics.py +++ b/Lib/test/test_statistics.py @@ -1010,6 +1010,291 @@ self.assertEqual(errmsg, msg) +class Test_Product(NumericTestCase): + """Test the private _product function.""" + + def test_ints(self): + data = [1, 2, 5, 7, 9] + self.assertEqual(statistics._product(data), (0, 630)) + self.assertEqual(statistics._product(data*100), (0, 630**100)) + + def test_floats(self): + data = [1.0, 2.0, 4.0, 8.0] + self.assertEqual(statistics._product(data), (8, 0.25)) + + def test_overflow(self): + # Test with floats that overflow. + data = [1e300]*5 + self.assertEqual(statistics._product(data), (5980, 0.6928287951283193)) + + def test_fractions(self): + F = Fraction + data = [F(14, 23), F(69, 1), F(665, 529), F(299, 105), F(1683, 39)] + exp, mant = statistics._product(data) + self.assertEqual(exp, 0) + self.assertEqual(mant, F(2*3*7*11*17*19, 23)) + self.assertTrue(isinstance(mant, F)) + # Mixed Fraction and int. + data = [3, 25, F(2, 15)] + exp, mant = statistics._product(data) + self.assertEqual(exp, 0) + self.assertEqual(mant, F(10)) + self.assertTrue(isinstance(mant, F)) + + @unittest.expectedFailure + def test_decimal(self): + D = Decimal + data = [D('24.5'), D('17.6'), D('0.025'), D('1.3')] + assert False + + def test_mixed_decimal_float(self): + # Test that mixed Decimal and float raises. + self.assertRaises(TypeError, statistics._product, [1.0, Decimal(1)]) + self.assertRaises(TypeError, statistics._product, [Decimal(1), 1.0]) + + +class Test_Nth_Root(NumericTestCase): + """Test the functionality of the private _nth_root function.""" + + def setUp(self): + self.nroot = statistics._nth_root + + # --- Special values (infinities, NANs, zeroes) --- + + def test_float_NAN(self): + # Test that the root of a float NAN is a float NAN. + NAN = float('nan') + for n in range(2, 9): + with self.subTest(n=n): + result = self.nroot(NAN, n) + self.assertTrue(math.isnan(result)) + + def test_decimal_QNAN(self): + # Test the behaviour when taking the root of a Decimal quiet NAN. + NAN = decimal.Decimal('nan') + with decimal.localcontext() as ctx: + ctx.traps[decimal.InvalidOperation] = 1 + self.assertRaises(decimal.InvalidOperation, self.nroot, NAN, 5) + ctx.traps[decimal.InvalidOperation] = 0 + self.assertTrue(self.nroot(NAN, 5).is_qnan()) + + def test_decimal_SNAN(self): + # Test that taking the root of a Decimal sNAN always raises. + sNAN = decimal.Decimal('snan') + with decimal.localcontext() as ctx: + ctx.traps[decimal.InvalidOperation] = 1 + self.assertRaises(decimal.InvalidOperation, self.nroot, sNAN, 5) + ctx.traps[decimal.InvalidOperation] = 0 + self.assertRaises(decimal.InvalidOperation, self.nroot, sNAN, 5) + + def test_inf(self): + # Test that the root of infinity is infinity. + for INF in (float('inf'), decimal.Decimal('inf')): + for n in range(2, 9): + with self.subTest(n=n, inf=INF): + self.assertEqual(self.nroot(INF, n), INF) + + def testNInf(self): + # Test that the root of -inf is -inf for odd n. + for NINF in (float('-inf'), decimal.Decimal('-inf')): + for n in range(3, 11, 2): + with self.subTest(n=n, inf=NINF): + self.assertEqual(self.nroot(NINF, n), NINF) + + # FIXME: need to check Decimal zeroes too. + def test_zero(self): + # Test that the root of +0.0 is +0.0. + for n in range(2, 11): + with self.subTest(n=n): + result = self.nroot(+0.0, n) + self.assertEqual(result, 0.0) + self.assertEqual(sign(result), +1) + + # FIXME: need to check Decimal zeroes too. + def test_neg_zero(self): + # Test that the root of -0.0 is -0.0. + for n in range(2, 11): + with self.subTest(n=n): + result = self.nroot(-0.0, n) + self.assertEqual(result, 0.0) + self.assertEqual(sign(result), -1) + + # --- Test return types --- + + def check_result_type(self, x, n, outtype): + self.assertIsInstance(self.nroot(x, n), outtype) + class MySubclass(type(x)): + pass + self.assertIsInstance(self.nroot(MySubclass(x), n), outtype) + + def testDecimal(self): + # Test that Decimal arguments return Decimal results. + self.check_result_type(decimal.Decimal('33.3'), 3, decimal.Decimal) + + def testFloat(self): + # Test that other arguments return float results. + for x in (0.2, Fraction(11, 7), 91): + self.check_result_type(x, 6, float) + + # --- Test bad input --- + + def testBadOrderTypes(self): + # Test that nroot raises correctly when n has the wrong type. + for n in (5.0, 2j, None, 'x', b'x', [], {}, set(), sign): + with self.subTest(n=n): + self.assertRaises(TypeError, self.nroot, 2.5, n) + + def testBadOrderValues(self): + # Test that nroot raises correctly when n has a wrong value. + for n in (1, 0, -1, -2, -87): + with self.subTest(n=n): + self.assertRaises(ValueError, self.nroot, 2.5, n) + + def testBadTypes(self): + # Test that nroot raises correctly when x has the wrong type. + for x in (None, 'x', b'x', [], {}, set(), sign): + with self.subTest(x=x): + self.assertRaises(TypeError, self.nroot, x, 3) + + def testNegativeEvenPower(self): + # Test negative x with even n raises correctly. + x = random.uniform(-20.0, -0.1) + assert x < 0 + for n in range(2, 9, 2): + with self.subTest(x=x, n=n): + self.assertRaises(ValueError, self.nroot, x, n) + + # --- Test that nroot is never worse than calling math.pow() --- + + def check_error_is_no_worse(self, x, n): + y = math.pow(x, n) + with self.subTest(x=x, n=n, y=y): + err1 = abs(self.nroot(y, n) - x) + err2 = abs(math.pow(y, 1.0/n) - x) + self.assertLessEqual(err1, err2) + + def testCompareWithPowSmall(self): + # Compare nroot with pow for small values of x. + for i in range(200): + x = random.uniform(1e-9, 1.0-1e-9) + n = random.choice(range(2, 16)) + self.check_error_is_no_worse(x, n) + + def testCompareWithPowMedium(self): + # Compare nroot with pow for medium-sized values of x. + for i in range(200): + x = random.uniform(1.0, 100.0) + n = random.choice(range(2, 16)) + self.check_error_is_no_worse(x, n) + + def testCompareWithPowLarge(self): + # Compare nroot with pow for largish values of x. + for i in range(200): + x = random.uniform(100.0, 10000.0) + n = random.choice(range(2, 16)) + self.check_error_is_no_worse(x, n) + + def testCompareWithPowHuge(self): + # Compare nroot with pow for huge values of x. + for i in range(200): + x = random.uniform(1e20, 1e50) + # We restrict the order here to avoid an Overflow error. + n = random.choice(range(2, 7)) + self.check_error_is_no_worse(x, n) + + # --- Test for numerically correct answers --- + + def testExactPowers(self): + # Test that small integer powers are calculated exactly. + for i in range(1, 51): + for n in range(2, 16): + if (i, n) == (35, 13): + # See testExpectedFailure35p13 + continue + with self.subTest(i=i, n=n): + x = i**n + self.assertEqual(self.nroot(x, n), i) + + def testExactPowersNegatives(self): + # Test that small negative integer powers are calculated exactly. + for i in range(-1, -51, -1): + for n in range(3, 16, 2): + if (i, n) == (-35, 13): + # See testExpectedFailure35p13 + continue + with self.subTest(i=i, n=n): + x = i**n + assert sign(x) == -1 + self.assertEqual(self.nroot(x, n), i) + + def testExpectedFailure35p13(self): + # Test the expected failure 35**13 is almost exact. + x = 35**13 + err = abs(self.nroot(x, 13) - 35) + self.assertLessEqual(err, 0.000000001) + err = abs(self.nroot(-x, 13) + 35) + self.assertLessEqual(err, 0.000000001) + + def testOne(self): + # Test that the root of 1.0 is 1.0. + for n in range(2, 11): + with self.subTest(n=n): + self.assertEqual(self.nroot(1.0, n), 1.0) + + def testFraction(self): + # Test Fraction results. + x = Fraction(89, 75) + self.assertEqual(self.nroot(x**12, 12), float(x)) + + def testInt(self): + # Test int results. + x = 276 + self.assertEqual(self.nroot(x**24, 24), x) + + def testBigInt(self): + # Test that ints too big to convert to floats work. + bignum = 10**20 # That's not that big... + self.assertEqual(self.nroot(bignum**280, 280), bignum) + # Can we make it bigger? + hugenum = bignum**50 + # Make sure that it is too big to convert to a float. + try: + y = float(hugenum) + except OverflowError: + pass + else: + raise AssertionError('hugenum is not big enough') + self.assertEqual(self.nroot(hugenum, 50), float(bignum)) + + def testDecimal(self): + # Test Decimal results. + for s in '3.759 64.027 5234.338'.split(): + x = decimal.Decimal(s) + with self.subTest(x=x): + a = self.nroot(x**5, 5) + self.assertEqual(a, x) + a = self.nroot(x**17, 17) + self.assertEqual(a, x) + + def testFloat(self): + # Test float results. + for x in (3.04e-16, 18.25, 461.3, 1.9e17): + with self.subTest(x=x): + self.assertEqual(self.nroot(x**3, 3), x) + self.assertEqual(self.nroot(x**8, 8), x) + self.assertEqual(self.nroot(x**11, 11), x) + + +class Test_NthRoot_NS(unittest.TestCase): + """Test internals of the nth_root function, hidden in _nroot_NS.""" + + def test_class_cannot_be_instantiated(self): + # Test that _nroot_NS cannot be instantiated. + # It should be a namespace, like in C++ or C#, but Python + # lacks that feature and so we have to make do with a class. + self.assertRaises(TypeError, statistics._nroot_NS) + + # === Tests for public functions === class UnivariateCommonMixin: -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Tue Aug 9 08:20:51 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 9 Aug 2016 13:20:51 +0100 Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-08-09 Message-ID: <7eab8e1b-2e63-42c3-945e-15d3258abc74@irsmsx106.ger.corp.intel.com> No new revisions. Here are the previous results: Results for project Python 2.7, build date 2016-08-09 02:47:34 +0000 commit: 09475e6135d0 previous commit: 5ab52b5ea4a7 revision date: 2016-08-05 20:10:26 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.14% -2.28% 3.79% 8.49% :-) pybench 0.20% -0.05% 5.66% 5.57% :-( regex_v8 0.62% -0.06% -2.37% 11.11% :-) nbody 0.29% -0.13% 8.50% 2.01% :-| json_dump_v2 0.29% -0.15% 1.92% 11.93% :-( normal_startup 1.88% -0.06% -5.97% 2.17% :-) ssbench 0.17% 0.30% 2.68% 1.69% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-08-09/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Tue Aug 9 08:27:35 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 9 Aug 2016 13:27:35 +0100 Subject: [Python-checkins] NEUTRAL Benchmark Results for Python Default 2016-08-09 Message-ID: Results for project Python default, build date 2016-08-09 02:01:56 +0000 commit: d69f782d642d previous commit: d86eabe2c457 revision date: 2016-08-08 21:43:46 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.32% -0.24% 10.65% 14.39% :-) pybench 0.07% 0.04% 5.97% 7.64% :-( regex_v8 2.69% -0.04% -2.84% 7.73% :-| nbody 1.08% -0.09% 1.12% 5.50% :-| json_dump_v2 0.29% -0.09% -0.84% 11.67% :-( normal_startup 1.00% -0.31% -2.18% 5.94% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-default-2016-08-09/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Tue Aug 9 09:58:34 2016 From: python-checkins at python.org (vinay.sajip) Date: Tue, 09 Aug 2016 13:58:34 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIxOTk5?= =?utf-8?q?=3A_Handled_empty_strings_correctly_when_in_POSIX_mode=2E?= Message-ID: <20160809135834.29328.10304.28BBCE46@psf.io> https://hg.python.org/cpython/rev/cf04243e8d7d changeset: 102578:cf04243e8d7d branch: 2.7 parent: 102543:09475e6135d0 user: Vinay Sajip date: Tue Aug 09 14:57:03 2016 +0100 summary: Issue #21999: Handled empty strings correctly when in POSIX mode. files: Lib/shlex.py | 2 +- Lib/test/test_shlex.py | 13 +++++++++++++ 2 files changed, 14 insertions(+), 1 deletions(-) diff --git a/Lib/shlex.py b/Lib/shlex.py --- a/Lib/shlex.py +++ b/Lib/shlex.py @@ -230,7 +230,7 @@ if self.debug >= 2: print "shlex: I see punctuation in word state" self.state = ' ' - if self.token: + if self.token or (self.posix and quoted): break # emit current token else: continue diff --git a/Lib/test/test_shlex.py b/Lib/test/test_shlex.py --- a/Lib/test/test_shlex.py +++ b/Lib/test/test_shlex.py @@ -178,6 +178,19 @@ "%s: %s != %s" % (self.data[i][0], l, self.data[i][1:])) + def testEmptyStringHandling(self): + """Test that parsing of empty strings is correctly handled.""" + # see Issue #21999 + expected = ['', ')', 'abc'] + + s = shlex.shlex("'')abc", posix=True) + slist = list(s) + self.assertEqual(slist, expected) + expected = ["''", ')', 'abc'] + s = shlex.shlex("'')abc") + self.assertEqual(list(s), expected) + + # Allow this test to be used with old shlex.py if not getattr(shlex, "split", None): for methname in dir(ShlexTest): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 9 10:07:16 2016 From: python-checkins at python.org (vinay.sajip) Date: Tue, 09 Aug 2016 14:07:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogQ2xvc2VzICMyMTk5OTogKEVtcHR5KSBtZXJnZSBmcm9tIDMuNS4=?= Message-ID: <20160809140715.13377.71152.F51B08D6@psf.io> https://hg.python.org/cpython/rev/a8e74448678c changeset: 102580:a8e74448678c parent: 102577:9eb5edfcf604 parent: 102579:4f02ad46a0a8 user: Vinay Sajip date: Tue Aug 09 15:07:06 2016 +0100 summary: Closes #21999: (Empty) merge from 3.5. files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 9 10:07:16 2016 From: python-checkins at python.org (vinay.sajip) Date: Tue, 09 Aug 2016 14:07:16 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogQ2xvc2VzICMyMTk5?= =?utf-8?q?9=3A_Handled_empty_strings_correctly_when_in_POSIX_mode=2E?= Message-ID: <20160809140715.100087.56872.7FF988B5@psf.io> https://hg.python.org/cpython/rev/4f02ad46a0a8 changeset: 102579:4f02ad46a0a8 branch: 3.5 parent: 102572:87e3a58ed3c3 user: Vinay Sajip date: Tue Aug 09 15:04:49 2016 +0100 summary: Closes #21999: Handled empty strings correctly when in POSIX mode. files: Lib/shlex.py | 2 +- Lib/test/test_shlex.py | 12 ++++++++++++ 2 files changed, 13 insertions(+), 1 deletions(-) diff --git a/Lib/shlex.py b/Lib/shlex.py --- a/Lib/shlex.py +++ b/Lib/shlex.py @@ -224,7 +224,7 @@ if self.debug >= 2: print("shlex: I see punctuation in word state") self.state = ' ' - if self.token: + if self.token or (self.posix and quoted): break # emit current token else: continue diff --git a/Lib/test/test_shlex.py b/Lib/test/test_shlex.py --- a/Lib/test/test_shlex.py +++ b/Lib/test/test_shlex.py @@ -173,6 +173,18 @@ "%s: %s != %s" % (self.data[i][0], l, self.data[i][1:])) + def testEmptyStringHandling(self): + """Test that parsing of empty strings is correctly handled.""" + # see Issue #21999 + expected = ['', ')', 'abc'] + + s = shlex.shlex("'')abc", posix=True) + slist = list(s) + self.assertEqual(slist, expected) + expected = ["''", ')', 'abc'] + s = shlex.shlex("'')abc") + self.assertEqual(list(s), expected) + def testQuote(self): safeunquoted = string.ascii_letters + string.digits + '@%_-+=:,./' unicode_sample = '\xe9\xe0\xdf' # e + acute accent, a + grave, sharp s -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 9 17:23:20 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 09 Aug 2016 21:23:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Correct_signat?= =?utf-8?q?ures_for_a_couple_of_logging_handlers?= Message-ID: <20160809212320.7012.12696.29F8334D@psf.io> https://hg.python.org/cpython/rev/7bb08f363db1 changeset: 102581:7bb08f363db1 branch: 3.5 parent: 102579:4f02ad46a0a8 user: Zachary Ware date: Tue Aug 09 16:20:41 2016 -0500 summary: Correct signatures for a couple of logging handlers Reported by Oleg Gromyak on docs@ files: Doc/library/logging.handlers.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -152,7 +152,7 @@ for this value. -.. class:: WatchedFileHandler(filename[,mode[, encoding[, delay]]]) +.. class:: WatchedFileHandler(filename, mode='a', encoding=None, delay=False) Returns a new instance of the :class:`WatchedFileHandler` class. The specified file is opened and used as the stream for logging. If *mode* is not specified, @@ -257,7 +257,7 @@ module, supports rotation of disk log files. -.. class:: RotatingFileHandler(filename, mode='a', maxBytes=0, backupCount=0, encoding=None, delay=0) +.. class:: RotatingFileHandler(filename, mode='a', maxBytes=0, backupCount=0, encoding=None, delay=False) Returns a new instance of the :class:`RotatingFileHandler` class. The specified file is opened and used as the stream for logging. If *mode* is not specified, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 9 17:23:20 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 09 Aug 2016 21:23:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20160809212320.130420.43480.E129EDA1@psf.io> https://hg.python.org/cpython/rev/769355e031e8 changeset: 102582:769355e031e8 parent: 102580:a8e74448678c parent: 102581:7bb08f363db1 user: Zachary Ware date: Tue Aug 09 16:21:17 2016 -0500 summary: Merge with 3.5 files: Doc/library/logging.handlers.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -155,7 +155,7 @@ for this value. -.. class:: WatchedFileHandler(filename[,mode[, encoding[, delay]]]) +.. class:: WatchedFileHandler(filename, mode='a', encoding=None, delay=False) Returns a new instance of the :class:`WatchedFileHandler` class. The specified file is opened and used as the stream for logging. If *mode* is not specified, @@ -271,7 +271,7 @@ module, supports rotation of disk log files. -.. class:: RotatingFileHandler(filename, mode='a', maxBytes=0, backupCount=0, encoding=None, delay=0) +.. class:: RotatingFileHandler(filename, mode='a', maxBytes=0, backupCount=0, encoding=None, delay=False) Returns a new instance of the :class:`RotatingFileHandler` class. The specified file is opened and used as the stream for logging. If *mode* is not specified, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 9 18:05:51 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 09 Aug 2016 22:05:51 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3MjA0?= =?utf-8?q?=3A_Fix_doctests_in_Doc/howto?= Message-ID: <20160809220551.108623.15577.6489639A@psf.io> https://hg.python.org/cpython/rev/0eaa05a3db02 changeset: 102583:0eaa05a3db02 branch: 2.7 parent: 102578:cf04243e8d7d user: Zachary Ware date: Tue Aug 09 17:03:32 2016 -0500 summary: Issue #27204: Fix doctests in Doc/howto Initial patch by Jelle Zijlstra. files: Doc/howto/sorting.rst | 43 ++++++++++++++++++------------ 1 files changed, 25 insertions(+), 18 deletions(-) diff --git a/Doc/howto/sorting.rst b/Doc/howto/sorting.rst --- a/Doc/howto/sorting.rst +++ b/Doc/howto/sorting.rst @@ -59,28 +59,28 @@ as keys. For example: >>> student_tuples = [ - ('john', 'A', 15), - ('jane', 'B', 12), - ('dave', 'B', 10), - ] + ... ('john', 'A', 15), + ... ('jane', 'B', 12), + ... ('dave', 'B', 10), + ... ] >>> sorted(student_tuples, key=lambda student: student[2]) # sort by age [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] The same technique works for objects with named attributes. For example: >>> class Student: - def __init__(self, name, grade, age): - self.name = name - self.grade = grade - self.age = age - def __repr__(self): - return repr((self.name, self.grade, self.age)) + ... def __init__(self, name, grade, age): + ... self.name = name + ... self.grade = grade + ... self.age = age + ... def __repr__(self): + ... return repr((self.name, self.grade, self.age)) >>> student_objects = [ - Student('john', 'A', 15), - Student('jane', 'B', 12), - Student('dave', 'B', 10), - ] + ... Student('john', 'A', 15), + ... Student('jane', 'B', 12), + ... Student('dave', 'B', 10), + ... ] >>> sorted(student_objects, key=lambda student: student.age) # sort by age [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] @@ -116,6 +116,7 @@ method could be used to compute message priority by counting the number of exclamation marks in a message: + >>> from operator import methodcaller >>> messages = ['critical!!!', 'hurry!', 'standby', 'immediate!!'] >>> sorted(messages, key=methodcaller('count', '!')) ['standby', 'hurry!', 'immediate!!', 'critical!!!'] @@ -220,15 +221,15 @@ a positive value for greater-than. For example, we can do: >>> def numeric_compare(x, y): - return x - y - >>> sorted([5, 2, 4, 1, 3], cmp=numeric_compare) + ... return x - y + >>> sorted([5, 2, 4, 1, 3], cmp=numeric_compare) # doctest: +SKIP [1, 2, 3, 4, 5] Or you can reverse the order of comparison with: >>> def reverse_numeric(x, y): - return y - x - >>> sorted([5, 2, 4, 1, 3], cmp=reverse_numeric) + ... return y - x + >>> sorted([5, 2, 4, 1, 3], cmp=reverse_numeric) # doctest: +SKIP [5, 4, 3, 2, 1] When porting code from Python 2.x to 3.x, the situation can arise when you have @@ -256,6 +257,12 @@ To convert to a key function, just wrap the old comparison function: +.. testsetup:: + + from functools import cmp_to_key + +.. doctest:: + >>> sorted([5, 2, 4, 1, 3], key=cmp_to_key(reverse_numeric)) [5, 4, 3, 2, 1] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 9 18:05:52 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 09 Aug 2016 22:05:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2327204=3A_Merge_with_3=2E5?= Message-ID: <20160809220552.8232.4026.EDD35F71@psf.io> https://hg.python.org/cpython/rev/e31597cbf52c changeset: 102585:e31597cbf52c parent: 102582:769355e031e8 parent: 102584:df92f9856b59 user: Zachary Ware date: Tue Aug 09 17:05:23 2016 -0500 summary: Closes #27204: Merge with 3.5 files: Doc/howto/functional.rst | 2 +- Doc/howto/ipaddress.rst | 9 ++--- Doc/howto/sorting.rst | 42 ++++++++++++++++------------ 3 files changed, 29 insertions(+), 24 deletions(-) diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst --- a/Doc/howto/functional.rst +++ b/Doc/howto/functional.rst @@ -1040,7 +1040,7 @@ elements of the iterable. This case is so common that there's a special built-in called :func:`sum` to compute it: - >>> import functools + >>> import functools, operator >>> functools.reduce(operator.add, [1,2,3,4], 0) 10 >>> sum([1,2,3,4]) diff --git a/Doc/howto/ipaddress.rst b/Doc/howto/ipaddress.rst --- a/Doc/howto/ipaddress.rst +++ b/Doc/howto/ipaddress.rst @@ -1,3 +1,7 @@ +.. testsetup:: + + import ipaddress + .. _ipaddress-howto: *************************************** @@ -49,11 +53,6 @@ determines whether to create an IPv4 or IPv6 address based on the passed in value: -.. testsetup:: - >>> import ipaddress - -:: - >>> ipaddress.ip_address('192.0.2.1') IPv4Address('192.0.2.1') >>> ipaddress.ip_address('2001:DB8::1') diff --git a/Doc/howto/sorting.rst b/Doc/howto/sorting.rst --- a/Doc/howto/sorting.rst +++ b/Doc/howto/sorting.rst @@ -58,28 +58,28 @@ as keys. For example: >>> student_tuples = [ - ('john', 'A', 15), - ('jane', 'B', 12), - ('dave', 'B', 10), - ] + ... ('john', 'A', 15), + ... ('jane', 'B', 12), + ... ('dave', 'B', 10), + ... ] >>> sorted(student_tuples, key=lambda student: student[2]) # sort by age [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] The same technique works for objects with named attributes. For example: >>> class Student: - def __init__(self, name, grade, age): - self.name = name - self.grade = grade - self.age = age - def __repr__(self): - return repr((self.name, self.grade, self.age)) + ... def __init__(self, name, grade, age): + ... self.name = name + ... self.grade = grade + ... self.age = age + ... def __repr__(self): + ... return repr((self.name, self.grade, self.age)) >>> student_objects = [ - Student('john', 'A', 15), - Student('jane', 'B', 12), - Student('dave', 'B', 10), - ] + ... Student('john', 'A', 15), + ... Student('jane', 'B', 12), + ... Student('dave', 'B', 10), + ... ] >>> sorted(student_objects, key=lambda student: student.age) # sort by age [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] @@ -208,15 +208,15 @@ a positive value for greater-than. For example, we can do: >>> def numeric_compare(x, y): - return x - y - >>> sorted([5, 2, 4, 1, 3], cmp=numeric_compare) + ... return x - y + >>> sorted([5, 2, 4, 1, 3], cmp=numeric_compare) # doctest: +SKIP [1, 2, 3, 4, 5] Or you can reverse the order of comparison with: >>> def reverse_numeric(x, y): - return y - x - >>> sorted([5, 2, 4, 1, 3], cmp=reverse_numeric) + ... return y - x + >>> sorted([5, 2, 4, 1, 3], cmp=reverse_numeric) # doctest: +SKIP [5, 4, 3, 2, 1] When porting code from Python 2.x to 3.x, the situation can arise when you have @@ -244,6 +244,12 @@ To convert to a key function, just wrap the old comparison function: +.. testsetup:: + + from functools import cmp_to_key + +.. doctest:: + >>> sorted([5, 2, 4, 1, 3], key=cmp_to_key(reverse_numeric)) [5, 4, 3, 2, 1] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 9 18:05:52 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 09 Aug 2016 22:05:52 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3MjA0?= =?utf-8?q?=3A_Fix_doctests_in_Doc/howto?= Message-ID: <20160809220552.30739.92850.F1A7096A@psf.io> https://hg.python.org/cpython/rev/df92f9856b59 changeset: 102584:df92f9856b59 branch: 3.5 parent: 102581:7bb08f363db1 user: Zachary Ware date: Tue Aug 09 16:47:04 2016 -0500 summary: Issue #27204: Fix doctests in Doc/howto Patch by Jelle Zijlstra. files: Doc/howto/functional.rst | 2 +- Doc/howto/ipaddress.rst | 9 ++--- Doc/howto/sorting.rst | 42 ++++++++++++++++------------ 3 files changed, 29 insertions(+), 24 deletions(-) diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst --- a/Doc/howto/functional.rst +++ b/Doc/howto/functional.rst @@ -1040,7 +1040,7 @@ elements of the iterable. This case is so common that there's a special built-in called :func:`sum` to compute it: - >>> import functools + >>> import functools, operator >>> functools.reduce(operator.add, [1,2,3,4], 0) 10 >>> sum([1,2,3,4]) diff --git a/Doc/howto/ipaddress.rst b/Doc/howto/ipaddress.rst --- a/Doc/howto/ipaddress.rst +++ b/Doc/howto/ipaddress.rst @@ -1,3 +1,7 @@ +.. testsetup:: + + import ipaddress + .. _ipaddress-howto: *************************************** @@ -49,11 +53,6 @@ determines whether to create an IPv4 or IPv6 address based on the passed in value: -.. testsetup:: - >>> import ipaddress - -:: - >>> ipaddress.ip_address('192.0.2.1') IPv4Address('192.0.2.1') >>> ipaddress.ip_address('2001:DB8::1') diff --git a/Doc/howto/sorting.rst b/Doc/howto/sorting.rst --- a/Doc/howto/sorting.rst +++ b/Doc/howto/sorting.rst @@ -58,28 +58,28 @@ as keys. For example: >>> student_tuples = [ - ('john', 'A', 15), - ('jane', 'B', 12), - ('dave', 'B', 10), - ] + ... ('john', 'A', 15), + ... ('jane', 'B', 12), + ... ('dave', 'B', 10), + ... ] >>> sorted(student_tuples, key=lambda student: student[2]) # sort by age [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] The same technique works for objects with named attributes. For example: >>> class Student: - def __init__(self, name, grade, age): - self.name = name - self.grade = grade - self.age = age - def __repr__(self): - return repr((self.name, self.grade, self.age)) + ... def __init__(self, name, grade, age): + ... self.name = name + ... self.grade = grade + ... self.age = age + ... def __repr__(self): + ... return repr((self.name, self.grade, self.age)) >>> student_objects = [ - Student('john', 'A', 15), - Student('jane', 'B', 12), - Student('dave', 'B', 10), - ] + ... Student('john', 'A', 15), + ... Student('jane', 'B', 12), + ... Student('dave', 'B', 10), + ... ] >>> sorted(student_objects, key=lambda student: student.age) # sort by age [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] @@ -208,15 +208,15 @@ a positive value for greater-than. For example, we can do: >>> def numeric_compare(x, y): - return x - y - >>> sorted([5, 2, 4, 1, 3], cmp=numeric_compare) + ... return x - y + >>> sorted([5, 2, 4, 1, 3], cmp=numeric_compare) # doctest: +SKIP [1, 2, 3, 4, 5] Or you can reverse the order of comparison with: >>> def reverse_numeric(x, y): - return y - x - >>> sorted([5, 2, 4, 1, 3], cmp=reverse_numeric) + ... return y - x + >>> sorted([5, 2, 4, 1, 3], cmp=reverse_numeric) # doctest: +SKIP [5, 4, 3, 2, 1] When porting code from Python 2.x to 3.x, the situation can arise when you have @@ -244,6 +244,12 @@ To convert to a key function, just wrap the old comparison function: +.. testsetup:: + + from functools import cmp_to_key + +.. doctest:: + >>> sorted([5, 2, 4, 1, 3], key=cmp_to_key(reverse_numeric)) [5, 4, 3, 2, 1] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 9 18:11:08 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 09 Aug 2016 22:11:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NzIy?= =?utf-8?q?=3A_Fix_default_for_touch_method=27s_=27mode=27_argument?= Message-ID: <20160809221108.6797.22566.19F5D723@psf.io> https://hg.python.org/cpython/rev/773bc9a3172d changeset: 102586:773bc9a3172d branch: 3.5 parent: 102584:df92f9856b59 user: Zachary Ware date: Tue Aug 09 17:10:39 2016 -0500 summary: Issue #27722: Fix default for touch method's 'mode' argument files: Doc/library/pathlib.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -989,7 +989,7 @@ of :func:`os.symlink`'s. -.. method:: Path.touch(mode=0o777, exist_ok=True) +.. method:: Path.touch(mode=0o666, exist_ok=True) Create a file at this given path. If *mode* is given, it is combined with the process' ``umask`` value to determine the file mode and access -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 9 18:11:09 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 09 Aug 2016 22:11:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2327722=3A_Merge_with_3=2E5?= Message-ID: <20160809221108.3240.67295.582282B1@psf.io> https://hg.python.org/cpython/rev/763d98f0a105 changeset: 102587:763d98f0a105 parent: 102585:e31597cbf52c parent: 102586:773bc9a3172d user: Zachary Ware date: Tue Aug 09 17:10:53 2016 -0500 summary: Closes #27722: Merge with 3.5 files: Doc/library/pathlib.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -998,7 +998,7 @@ of :func:`os.symlink`'s. -.. method:: Path.touch(mode=0o777, exist_ok=True) +.. method:: Path.touch(mode=0o666, exist_ok=True) Create a file at this given path. If *mode* is given, it is combined with the process' ``umask`` value to determine the file mode and access -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 9 18:40:10 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 09 Aug 2016 22:40:10 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3MjA1?= =?utf-8?q?=3A_Fix_doctests_in_Doc/library/collections=2Erst?= Message-ID: <20160809224010.11584.94474.1D34A025@psf.io> https://hg.python.org/cpython/rev/e399930d14f0 changeset: 102588:e399930d14f0 branch: 2.7 parent: 102583:0eaa05a3db02 user: Zachary Ware date: Tue Aug 09 17:28:33 2016 -0500 summary: Issue #27205: Fix doctests in Doc/library/collections.rst files: Doc/library/collections.rst | 25 +++++++++++++------------ 1 files changed, 13 insertions(+), 12 deletions(-) diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -608,7 +608,7 @@ _fields = ('x', 'y') def __new__(_cls, x, y): - 'Create a new instance of Point(x, y)' + 'Create new instance of Point(x, y)' return _tuple.__new__(_cls, (x, y)) @classmethod @@ -635,7 +635,7 @@ return result def __getnewargs__(self): - 'Return self as a plain tuple. Used by copy and pickle.' + 'Return self as a plain tuple. Used by copy and pickle.' return tuple(self) __dict__ = _property(_asdict) @@ -648,6 +648,7 @@ y = _property(_itemgetter(1), doc='Alias for field number 1') + >>> p = Point(11, y=22) # instantiate with positional or keyword arguments >>> p[0] + p[1] # indexable like the plain tuple (11, 22) @@ -712,7 +713,7 @@ Point(x=33, y=22) >>> for partnum, record in inventory.items(): - inventory[partnum] = record._replace(price=newprices[partnum], timestamp=time.now()) + ... inventory[partnum] = record._replace(price=newprices[partnum], timestamp=time.now()) .. attribute:: somenamedtuple._fields @@ -747,15 +748,15 @@ a fixed-width print format: >>> class Point(namedtuple('Point', 'x y')): - __slots__ = () - @property - def hypot(self): - return (self.x ** 2 + self.y ** 2) ** 0.5 - def __str__(self): - return 'Point: x=%6.3f y=%6.3f hypot=%6.3f' % (self.x, self.y, self.hypot) - + ... __slots__ = () + ... @property + ... def hypot(self): + ... return (self.x ** 2 + self.y ** 2) ** 0.5 + ... def __str__(self): + ... return 'Point: x=%6.3f y=%6.3f hypot=%6.3f' % (self.x, self.y, self.hypot) + ... >>> for p in Point(3, 4), Point(14, 5/7.): - print p + ... print p Point: x= 3.000 y= 4.000 hypot= 5.000 Point: x=14.000 y= 0.714 hypot=14.018 @@ -781,7 +782,7 @@ >>> Status.open, Status.pending, Status.closed (0, 1, 2) >>> class Status: - open, pending, closed = range(3) + ... open, pending, closed = range(3) .. seealso:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 9 18:40:11 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 09 Aug 2016 22:40:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2327205=3A_Merge_with_3=2E5?= Message-ID: <20160809224010.7814.14098.15E8F017@psf.io> https://hg.python.org/cpython/rev/6817b8112f6e changeset: 102590:6817b8112f6e parent: 102587:763d98f0a105 parent: 102589:0d95e47cfe3e user: Zachary Ware date: Tue Aug 09 17:38:44 2016 -0500 summary: Closes #27205: Merge with 3.5 files: Doc/library/collections.rst | 30 ++++++++++++------------ 1 files changed, 15 insertions(+), 15 deletions(-) diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -263,7 +263,7 @@ is less than one, :meth:`elements` will ignore it. >>> c = Counter(a=4, b=2, c=0, d=-2) - >>> list(c.elements()) + >>> sorted(c.elements()) ['a', 'a', 'a', 'a', 'b', 'b'] .. method:: most_common([n]) @@ -273,7 +273,7 @@ :func:`most_common` returns *all* elements in the counter. Elements with equal counts are ordered arbitrarily: - >>> Counter('abracadabra').most_common(3) + >>> Counter('abracadabra').most_common(3) # doctest: +SKIP [('a', 5), ('r', 2), ('b', 2)] .. method:: subtract([iterable-or-mapping]) @@ -329,7 +329,7 @@ Counter({'a': 4, 'b': 3}) >>> c - d # subtract (keeping only positive counts) Counter({'a': 2}) - >>> c & d # intersection: min(c[x], d[x]) + >>> c & d # intersection: min(c[x], d[x]) # doctest: +SKIP Counter({'a': 1, 'b': 1}) >>> c | d # union: max(c[x], d[x]) Counter({'a': 3, 'b': 2}) @@ -698,7 +698,7 @@ >>> for k, v in s: ... d[k].append(v) ... - >>> list(d.items()) + >>> sorted(d.items()) [('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])] When each key is encountered for the first time, it is not already in the @@ -713,7 +713,7 @@ >>> for k, v in s: ... d.setdefault(k, []).append(v) ... - >>> list(d.items()) + >>> sorted(d.items()) [('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])] Setting the :attr:`default_factory` to :class:`int` makes the @@ -725,8 +725,8 @@ >>> for k in s: ... d[k] += 1 ... - >>> list(d.items()) - [('i', 4), ('p', 2), ('s', 4), ('m', 1)] + >>> sorted(d.items()) + [('i', 4), ('m', 1), ('p', 2), ('s', 4)] When a letter is first encountered, it is missing from the mapping, so the :attr:`default_factory` function calls :func:`int` to supply a default count of @@ -752,7 +752,7 @@ >>> for k, v in s: ... d[k].add(v) ... - >>> list(d.items()) + >>> sorted(d.items()) [('blue', {2, 4}), ('red', {1, 3})] @@ -914,15 +914,15 @@ a fixed-width print format: >>> class Point(namedtuple('Point', 'x y')): - __slots__ = () - @property - def hypot(self): - return (self.x ** 2 + self.y ** 2) ** 0.5 - def __str__(self): - return 'Point: x=%6.3f y=%6.3f hypot=%6.3f' % (self.x, self.y, self.hypot) + ... __slots__ = () + ... @property + ... def hypot(self): + ... return (self.x ** 2 + self.y ** 2) ** 0.5 + ... def __str__(self): + ... return 'Point: x=%6.3f y=%6.3f hypot=%6.3f' % (self.x, self.y, self.hypot) >>> for p in Point(3, 4), Point(14, 5/7): - print(p) + ... print(p) Point: x= 3.000 y= 4.000 hypot= 5.000 Point: x=14.000 y= 0.714 hypot=14.018 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 9 18:40:16 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 09 Aug 2016 22:40:16 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3MjA1?= =?utf-8?q?=3A_Fix_doctests_in_Doc/library/collections=2Erst=2E?= Message-ID: <20160809224010.3293.15304.AC6E4F01@psf.io> https://hg.python.org/cpython/rev/0d95e47cfe3e changeset: 102589:0d95e47cfe3e branch: 3.5 parent: 102586:773bc9a3172d user: Zachary Ware date: Tue Aug 09 17:38:22 2016 -0500 summary: Issue #27205: Fix doctests in Doc/library/collections.rst. Initial patch by Jelle Zijlstra. files: Doc/library/collections.rst | 30 ++++++++++++------------ 1 files changed, 15 insertions(+), 15 deletions(-) diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -263,7 +263,7 @@ is less than one, :meth:`elements` will ignore it. >>> c = Counter(a=4, b=2, c=0, d=-2) - >>> list(c.elements()) + >>> sorted(c.elements()) ['a', 'a', 'a', 'a', 'b', 'b'] .. method:: most_common([n]) @@ -273,7 +273,7 @@ :func:`most_common` returns *all* elements in the counter. Elements with equal counts are ordered arbitrarily: - >>> Counter('abracadabra').most_common(3) + >>> Counter('abracadabra').most_common(3) # doctest: +SKIP [('a', 5), ('r', 2), ('b', 2)] .. method:: subtract([iterable-or-mapping]) @@ -329,7 +329,7 @@ Counter({'a': 4, 'b': 3}) >>> c - d # subtract (keeping only positive counts) Counter({'a': 2}) - >>> c & d # intersection: min(c[x], d[x]) + >>> c & d # intersection: min(c[x], d[x]) # doctest: +SKIP Counter({'a': 1, 'b': 1}) >>> c | d # union: max(c[x], d[x]) Counter({'a': 3, 'b': 2}) @@ -698,7 +698,7 @@ >>> for k, v in s: ... d[k].append(v) ... - >>> list(d.items()) + >>> sorted(d.items()) [('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])] When each key is encountered for the first time, it is not already in the @@ -713,7 +713,7 @@ >>> for k, v in s: ... d.setdefault(k, []).append(v) ... - >>> list(d.items()) + >>> sorted(d.items()) [('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])] Setting the :attr:`default_factory` to :class:`int` makes the @@ -725,8 +725,8 @@ >>> for k in s: ... d[k] += 1 ... - >>> list(d.items()) - [('i', 4), ('p', 2), ('s', 4), ('m', 1)] + >>> sorted(d.items()) + [('i', 4), ('m', 1), ('p', 2), ('s', 4)] When a letter is first encountered, it is missing from the mapping, so the :attr:`default_factory` function calls :func:`int` to supply a default count of @@ -752,7 +752,7 @@ >>> for k, v in s: ... d[k].add(v) ... - >>> list(d.items()) + >>> sorted(d.items()) [('blue', {2, 4}), ('red', {1, 3})] @@ -914,15 +914,15 @@ a fixed-width print format: >>> class Point(namedtuple('Point', 'x y')): - __slots__ = () - @property - def hypot(self): - return (self.x ** 2 + self.y ** 2) ** 0.5 - def __str__(self): - return 'Point: x=%6.3f y=%6.3f hypot=%6.3f' % (self.x, self.y, self.hypot) + ... __slots__ = () + ... @property + ... def hypot(self): + ... return (self.x ** 2 + self.y ** 2) ** 0.5 + ... def __str__(self): + ... return 'Point: x=%6.3f y=%6.3f hypot=%6.3f' % (self.x, self.y, self.hypot) >>> for p in Point(3, 4), Point(14, 5/7): - print(p) + ... print(p) Point: x= 3.000 y= 4.000 hypot= 5.000 Point: x=14.000 y= 0.714 hypot=14.018 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 9 18:45:52 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 09 Aug 2016 22:45:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2327206=3A_Merge_with_3=2E5?= Message-ID: <20160809224552.29419.43397.52528493@psf.io> https://hg.python.org/cpython/rev/61d7aeb6aa1b changeset: 102592:61d7aeb6aa1b parent: 102590:6817b8112f6e parent: 102591:8b7efeeefa50 user: Zachary Ware date: Tue Aug 09 17:45:15 2016 -0500 summary: Closes #27206: Merge with 3.5 files: Doc/tutorial/floatingpoint.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/tutorial/floatingpoint.rst b/Doc/tutorial/floatingpoint.rst --- a/Doc/tutorial/floatingpoint.rst +++ b/Doc/tutorial/floatingpoint.rst @@ -1,3 +1,7 @@ +.. testsetup:: + + import math + .. _tut-fp-issues: ************************************************** -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 9 18:45:52 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 09 Aug 2016 22:45:52 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3MjA2?= =?utf-8?q?=3A_Fix_doctests_in_Doc/tutorial=2E?= Message-ID: <20160809224552.29553.22232.712D44B3@psf.io> https://hg.python.org/cpython/rev/8b7efeeefa50 changeset: 102591:8b7efeeefa50 branch: 3.5 parent: 102589:0d95e47cfe3e user: Zachary Ware date: Tue Aug 09 17:44:52 2016 -0500 summary: Issue #27206: Fix doctests in Doc/tutorial. Patch by Jelle Zijlstra. files: Doc/tutorial/floatingpoint.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/tutorial/floatingpoint.rst b/Doc/tutorial/floatingpoint.rst --- a/Doc/tutorial/floatingpoint.rst +++ b/Doc/tutorial/floatingpoint.rst @@ -1,3 +1,7 @@ +.. testsetup:: + + import math + .. _tut-fp-issues: ************************************************** -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 01:31:43 2016 From: python-checkins at python.org (zach.ware) Date: Wed, 10 Aug 2016 05:31:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2327207=3A_Merge_with_3=2E5?= Message-ID: <20160810053143.3664.28449.BAF734BA@psf.io> https://hg.python.org/cpython/rev/ea43895ebfd9 changeset: 102594:ea43895ebfd9 parent: 102592:61d7aeb6aa1b parent: 102593:cee3074233e6 user: Zachary Ware date: Wed Aug 10 00:31:31 2016 -0500 summary: Closes #27207: Merge with 3.5 files: Doc/whatsnew/3.2.rst | 51 ++++++++++++++++++++----------- 1 files changed, 32 insertions(+), 19 deletions(-) diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst --- a/Doc/whatsnew/3.2.rst +++ b/Doc/whatsnew/3.2.rst @@ -313,14 +313,14 @@ of the actual file that was imported: >>> import collections - >>> collections.__cached__ + >>> collections.__cached__ # doctest: +SKIP 'c:/py32/lib/__pycache__/collections.cpython-32.pyc' * The tag that is unique to each interpreter is accessible from the :mod:`imp` module: >>> import imp - >>> imp.get_tag() + >>> imp.get_tag() # doctest: +SKIP 'cpython-32' * Scripts that try to deduce source filename from the imported file now need to @@ -329,7 +329,7 @@ >>> imp.source_from_cache('c:/py32/lib/__pycache__/collections.cpython-32.pyc') 'c:/py32/lib/collections.py' - >>> imp.cache_from_source('c:/py32/lib/collections.py') + >>> imp.cache_from_source('c:/py32/lib/collections.py') # doctest: +SKIP 'c:/py32/lib/__pycache__/collections.cpython-32.pyc' * The :mod:`py_compile` and :mod:`compileall` modules have been updated to @@ -532,7 +532,7 @@ original object. >>> with memoryview(b'abcdefgh') as v: - print(v.tolist()) + ... print(v.tolist()) [97, 98, 99, 100, 101, 102, 103, 104] (Added by Antoine Pitrou; :issue:`9757`.) @@ -568,9 +568,10 @@ expect a tuple as an argument. This is a big step forward in making the C structures as flexible as their pure Python counterparts: + >>> import sys >>> isinstance(sys.version_info, tuple) True - >>> 'Version %d.%d.%d %s(%d)' % sys.version_info + >>> 'Version %d.%d.%d %s(%d)' % sys.version_info # doctest: +SKIP 'Version 3.2.0 final(0)' (Suggested by Arfrever Frehtes Taifersar Arahesis and implemented @@ -757,18 +758,18 @@ >>> import functools >>> @functools.lru_cache(maxsize=300) - >>> def get_phone_number(name): - c = conn.cursor() - c.execute('SELECT phonenumber FROM phonelist WHERE name=?', (name,)) - return c.fetchone()[0] - - >>> for name in user_requests: - get_phone_number(name) # cached lookup + ... def get_phone_number(name): + ... c = conn.cursor() + ... c.execute('SELECT phonenumber FROM phonelist WHERE name=?', (name,)) + ... return c.fetchone()[0] + + >>> for name in user_requests: # doctest: +SKIP + ... get_phone_number(name) # cached lookup To help with choosing an effective cache size, the wrapped function is instrumented for tracking cache statistics: - >>> get_phone_number.cache_info() + >>> get_phone_number.cache_info() # doctest: +SKIP CacheInfo(hits=4805, misses=980, maxsize=300, currsize=300) If the phonelist table gets updated, the outdated contents of the cache can be @@ -823,7 +824,7 @@ modern :term:`key function`: >>> # locale-aware sort order - >>> sorted(iterable, key=cmp_to_key(locale.strcoll)) + >>> sorted(iterable, key=cmp_to_key(locale.strcoll)) # doctest: +SKIP For sorting examples and a brief sorting tutorial, see the `Sorting HowTo `_ tutorial. @@ -861,7 +862,8 @@ which only have positive counts, and the latter is more suitable for use cases that allow negative counts: - >>> tally = Counter(dogs=5, cat=3) + >>> from collections import Counter + >>> tally = Counter(dogs=5, cats=3) >>> tally -= Counter(dogs=2, cats=8) # saturating subtraction >>> tally Counter({'dogs': 3}) @@ -884,6 +886,7 @@ an ordered dictionary can be used to track order of access by aging entries from the oldest to the most recently accessed. + >>> from collections import OrderedDict >>> d = OrderedDict.fromkeys(['a', 'b', 'X', 'd', 'e']) >>> list(d) ['a', 'b', 'X', 'd', 'e'] @@ -897,6 +900,7 @@ :meth:`~collections.deque.count` and :meth:`~collections.deque.reverse` that make them more substitutable for :class:`list` objects: + >>> from collections import deque >>> d = deque('simsalabim') >>> d.count('s') 2 @@ -1042,6 +1046,7 @@ special values. It returns *True* for regular numbers and *False* for *Nan* or *Infinity*: +>>> from math import isfinite >>> [isfinite(x) for x in (123, 4.56, float('Nan'), float('Inf'))] [True, True, False, False] @@ -1049,6 +1054,7 @@ without incurring the loss of precision that usually accompanies the subtraction of nearly equal quantities: +>>> from math import expm1 >>> expm1(0.013671875) # more accurate way to compute e**x-1 for a small x 0.013765762467652909 @@ -1056,6 +1062,7 @@ error function `_. The complementary error function, :func:`~math.erfc`, is ``1 - erf(x)``: +>>> from math import erf, erfc, sqrt >>> erf(1.0/sqrt(2.0)) # portion of normal distribution within 1 standard deviation 0.682689492137086 >>> erfc(1.0/sqrt(2.0)) # portion of normal distribution outside 1 standard deviation @@ -1069,6 +1076,7 @@ *x*, so there is also a :func:`~math.lgamma` function for computing the natural logarithm of the gamma function: +>>> from math import gamma, lgamma >>> gamma(7.0) # six factorial 720.0 >>> lgamma(801.0) # log(800 factorial) @@ -1287,7 +1295,7 @@ prime modulus, the hash values for *infinity* and *nan*, and the multiplier used for the imaginary part of a number: ->>> sys.hash_info +>>> sys.hash_info # doctest: +SKIP sys.hash_info(width=64, modulus=2305843009213693951, inf=314159, nan=0, imag=1000003) An early decision to limit the inter-operability of various numeric types has @@ -1310,6 +1318,8 @@ :meth:`~fractions.Fraction.from_float()` and :meth:`~fractions.Fraction.from_decimal` methods are no longer needed (:issue:`8294`): +>>> from decimal import Decimal +>>> from fractions import Fraction >>> Decimal(1.1) Decimal('1.100000000000000088817841970012523233890533447265625') >>> Fraction(1.1) @@ -1392,6 +1402,7 @@ decompression. Keep in mind that text needs to be encoded as :class:`bytes` before compressing and decompressing: +>>> import gzip >>> s = 'Three shall be the number thou shalt count, ' >>> s += 'and the number of the counting shall be three' >>> b = s.encode() # convert to utf-8 @@ -1401,7 +1412,7 @@ >>> len(c) 77 >>> gzip.decompress(c).decode()[:42] # decompress and convert to text -'Three shall be the number thou shalt count,' +'Three shall be the number thou shalt count' (Contributed by Anand B. Pillai in :issue:`3488`; and by Antoine Pitrou, Nir Aides and Brian Curtin in :issue:`9962`, :issue:`1675951`, :issue:`7471` and @@ -1503,6 +1514,7 @@ :func:`~os.fsencode` and :func:`~os.fsdecode`, for encoding and decoding filenames: +>>> import os >>> filename = 'Sehensw?rdigkeiten' >>> os.fsencode(filename) b'Sehensw\xc3\xbcrdigkeiten' @@ -1740,6 +1752,7 @@ :class:`unittest.case.TestCase` class can now be instantiated without arguments: + >>> from unittest import TestCase >>> TestCase().assertEqual(pow(2, 3), 8) (Contributed by Michael Foord.) @@ -2201,7 +2214,7 @@ `_ addresses as described in :rfc:`2732`: >>> import urllib.parse - >>> urllib.parse.urlparse('http://[dead:beef:cafe:5417:affe:8FA3:deaf:feed]/foo/') + >>> urllib.parse.urlparse('http://[dead:beef:cafe:5417:affe:8FA3:deaf:feed]/foo/') # doctest: +NORMALIZE_WHITESPACE ParseResult(scheme='http', netloc='[dead:beef:cafe:5417:affe:8FA3:deaf:feed]', path='/foo/', @@ -2235,7 +2248,7 @@ not mixed with regular strings. If ASCII-encoded byte strings are given as parameters, the return types will also be an ASCII-encoded byte strings: - >>> urllib.parse.urlparse(b'http://www.python.org:80/about/') + >>> urllib.parse.urlparse(b'http://www.python.org:80/about/') # doctest: +NORMALIZE_WHITESPACE ParseResultBytes(scheme=b'http', netloc=b'www.python.org:80', path=b'/about/', params=b'', query=b'', fragment=b'') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 01:31:43 2016 From: python-checkins at python.org (zach.ware) Date: Wed, 10 Aug 2016 05:31:43 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3MjA3?= =?utf-8?q?=3A_Fix_doctests_in_Doc/whatsnew/3=2E2=2Erst?= Message-ID: <20160810053142.3622.11018.51F2A459@psf.io> https://hg.python.org/cpython/rev/cee3074233e6 changeset: 102593:cee3074233e6 branch: 3.5 parent: 102591:8b7efeeefa50 user: Zachary Ware date: Wed Aug 10 00:30:41 2016 -0500 summary: Issue #27207: Fix doctests in Doc/whatsnew/3.2.rst Initial patch by Jelle Zijlstra. files: Doc/whatsnew/3.2.rst | 51 ++++++++++++++++++++----------- 1 files changed, 32 insertions(+), 19 deletions(-) diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst --- a/Doc/whatsnew/3.2.rst +++ b/Doc/whatsnew/3.2.rst @@ -313,14 +313,14 @@ of the actual file that was imported: >>> import collections - >>> collections.__cached__ + >>> collections.__cached__ # doctest: +SKIP 'c:/py32/lib/__pycache__/collections.cpython-32.pyc' * The tag that is unique to each interpreter is accessible from the :mod:`imp` module: >>> import imp - >>> imp.get_tag() + >>> imp.get_tag() # doctest: +SKIP 'cpython-32' * Scripts that try to deduce source filename from the imported file now need to @@ -329,7 +329,7 @@ >>> imp.source_from_cache('c:/py32/lib/__pycache__/collections.cpython-32.pyc') 'c:/py32/lib/collections.py' - >>> imp.cache_from_source('c:/py32/lib/collections.py') + >>> imp.cache_from_source('c:/py32/lib/collections.py') # doctest: +SKIP 'c:/py32/lib/__pycache__/collections.cpython-32.pyc' * The :mod:`py_compile` and :mod:`compileall` modules have been updated to @@ -532,7 +532,7 @@ original object. >>> with memoryview(b'abcdefgh') as v: - print(v.tolist()) + ... print(v.tolist()) [97, 98, 99, 100, 101, 102, 103, 104] (Added by Antoine Pitrou; :issue:`9757`.) @@ -568,9 +568,10 @@ expect a tuple as an argument. This is a big step forward in making the C structures as flexible as their pure Python counterparts: + >>> import sys >>> isinstance(sys.version_info, tuple) True - >>> 'Version %d.%d.%d %s(%d)' % sys.version_info + >>> 'Version %d.%d.%d %s(%d)' % sys.version_info # doctest: +SKIP 'Version 3.2.0 final(0)' (Suggested by Arfrever Frehtes Taifersar Arahesis and implemented @@ -757,18 +758,18 @@ >>> import functools >>> @functools.lru_cache(maxsize=300) - >>> def get_phone_number(name): - c = conn.cursor() - c.execute('SELECT phonenumber FROM phonelist WHERE name=?', (name,)) - return c.fetchone()[0] - - >>> for name in user_requests: - get_phone_number(name) # cached lookup + ... def get_phone_number(name): + ... c = conn.cursor() + ... c.execute('SELECT phonenumber FROM phonelist WHERE name=?', (name,)) + ... return c.fetchone()[0] + + >>> for name in user_requests: # doctest: +SKIP + ... get_phone_number(name) # cached lookup To help with choosing an effective cache size, the wrapped function is instrumented for tracking cache statistics: - >>> get_phone_number.cache_info() + >>> get_phone_number.cache_info() # doctest: +SKIP CacheInfo(hits=4805, misses=980, maxsize=300, currsize=300) If the phonelist table gets updated, the outdated contents of the cache can be @@ -823,7 +824,7 @@ modern :term:`key function`: >>> # locale-aware sort order - >>> sorted(iterable, key=cmp_to_key(locale.strcoll)) + >>> sorted(iterable, key=cmp_to_key(locale.strcoll)) # doctest: +SKIP For sorting examples and a brief sorting tutorial, see the `Sorting HowTo `_ tutorial. @@ -861,7 +862,8 @@ which only have positive counts, and the latter is more suitable for use cases that allow negative counts: - >>> tally = Counter(dogs=5, cat=3) + >>> from collections import Counter + >>> tally = Counter(dogs=5, cats=3) >>> tally -= Counter(dogs=2, cats=8) # saturating subtraction >>> tally Counter({'dogs': 3}) @@ -884,6 +886,7 @@ an ordered dictionary can be used to track order of access by aging entries from the oldest to the most recently accessed. + >>> from collections import OrderedDict >>> d = OrderedDict.fromkeys(['a', 'b', 'X', 'd', 'e']) >>> list(d) ['a', 'b', 'X', 'd', 'e'] @@ -897,6 +900,7 @@ :meth:`~collections.deque.count` and :meth:`~collections.deque.reverse` that make them more substitutable for :class:`list` objects: + >>> from collections import deque >>> d = deque('simsalabim') >>> d.count('s') 2 @@ -1042,6 +1046,7 @@ special values. It returns *True* for regular numbers and *False* for *Nan* or *Infinity*: +>>> from math import isfinite >>> [isfinite(x) for x in (123, 4.56, float('Nan'), float('Inf'))] [True, True, False, False] @@ -1049,6 +1054,7 @@ without incurring the loss of precision that usually accompanies the subtraction of nearly equal quantities: +>>> from math import expm1 >>> expm1(0.013671875) # more accurate way to compute e**x-1 for a small x 0.013765762467652909 @@ -1056,6 +1062,7 @@ error function `_. The complementary error function, :func:`~math.erfc`, is ``1 - erf(x)``: +>>> from math import erf, erfc, sqrt >>> erf(1.0/sqrt(2.0)) # portion of normal distribution within 1 standard deviation 0.682689492137086 >>> erfc(1.0/sqrt(2.0)) # portion of normal distribution outside 1 standard deviation @@ -1069,6 +1076,7 @@ *x*, so there is also a :func:`~math.lgamma` function for computing the natural logarithm of the gamma function: +>>> from math import gamma, lgamma >>> gamma(7.0) # six factorial 720.0 >>> lgamma(801.0) # log(800 factorial) @@ -1287,7 +1295,7 @@ prime modulus, the hash values for *infinity* and *nan*, and the multiplier used for the imaginary part of a number: ->>> sys.hash_info +>>> sys.hash_info # doctest: +SKIP sys.hash_info(width=64, modulus=2305843009213693951, inf=314159, nan=0, imag=1000003) An early decision to limit the inter-operability of various numeric types has @@ -1310,6 +1318,8 @@ :meth:`~fractions.Fraction.from_float()` and :meth:`~fractions.Fraction.from_decimal` methods are no longer needed (:issue:`8294`): +>>> from decimal import Decimal +>>> from fractions import Fraction >>> Decimal(1.1) Decimal('1.100000000000000088817841970012523233890533447265625') >>> Fraction(1.1) @@ -1392,6 +1402,7 @@ decompression. Keep in mind that text needs to be encoded as :class:`bytes` before compressing and decompressing: +>>> import gzip >>> s = 'Three shall be the number thou shalt count, ' >>> s += 'and the number of the counting shall be three' >>> b = s.encode() # convert to utf-8 @@ -1401,7 +1412,7 @@ >>> len(c) 77 >>> gzip.decompress(c).decode()[:42] # decompress and convert to text -'Three shall be the number thou shalt count,' +'Three shall be the number thou shalt count' (Contributed by Anand B. Pillai in :issue:`3488`; and by Antoine Pitrou, Nir Aides and Brian Curtin in :issue:`9962`, :issue:`1675951`, :issue:`7471` and @@ -1503,6 +1514,7 @@ :func:`~os.fsencode` and :func:`~os.fsdecode`, for encoding and decoding filenames: +>>> import os >>> filename = 'Sehensw?rdigkeiten' >>> os.fsencode(filename) b'Sehensw\xc3\xbcrdigkeiten' @@ -1740,6 +1752,7 @@ :class:`unittest.case.TestCase` class can now be instantiated without arguments: + >>> from unittest import TestCase >>> TestCase().assertEqual(pow(2, 3), 8) (Contributed by Michael Foord.) @@ -2201,7 +2214,7 @@ `_ addresses as described in :rfc:`2732`: >>> import urllib.parse - >>> urllib.parse.urlparse('http://[dead:beef:cafe:5417:affe:8FA3:deaf:feed]/foo/') + >>> urllib.parse.urlparse('http://[dead:beef:cafe:5417:affe:8FA3:deaf:feed]/foo/') # doctest: +NORMALIZE_WHITESPACE ParseResult(scheme='http', netloc='[dead:beef:cafe:5417:affe:8FA3:deaf:feed]', path='/foo/', @@ -2235,7 +2248,7 @@ not mixed with regular strings. If ASCII-encoded byte strings are given as parameters, the return types will also be an ASCII-encoded byte strings: - >>> urllib.parse.urlparse(b'http://www.python.org:80/about/') + >>> urllib.parse.urlparse(b'http://www.python.org:80/about/') # doctest: +NORMALIZE_WHITESPACE ParseResultBytes(scheme=b'http', netloc=b'www.python.org:80', path=b'/about/', params=b'', query=b'', fragment=b'') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 01:36:00 2016 From: python-checkins at python.org (zach.ware) Date: Wed, 10 Aug 2016 05:36:00 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3MjA4?= =?utf-8?q?=3A_Fix_doctest_in_Doc/library/traceback=2Erst?= Message-ID: <20160810053600.8100.63705.A24EED75@psf.io> https://hg.python.org/cpython/rev/eadf2905f424 changeset: 102595:eadf2905f424 branch: 3.5 parent: 102593:cee3074233e6 user: Zachary Ware date: Wed Aug 10 00:35:27 2016 -0500 summary: Issue #27208: Fix doctest in Doc/library/traceback.rst Patch by Jelle Zijlstra. files: Doc/library/traceback.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/traceback.rst b/Doc/library/traceback.rst --- a/Doc/library/traceback.rst +++ b/Doc/library/traceback.rst @@ -361,7 +361,7 @@ traceback.print_exception(exc_type, exc_value, exc_traceback, limit=2, file=sys.stdout) print("*** print_exc:") - traceback.print_exc() + traceback.print_exc(limit=2, file=sys.stdout) print("*** format_exc, first and last line:") formatted_lines = traceback.format_exc().splitlines() print(formatted_lines[0]) @@ -407,9 +407,9 @@ ' File "", line 7, in bright_side_of_death\n return tuple()[0]\n', 'IndexError: tuple index out of range\n'] *** extract_tb: - [('', 10, '', 'lumberjack()'), - ('', 4, 'lumberjack', 'bright_side_of_death()'), - ('', 7, 'bright_side_of_death', 'return tuple()[0]')] + [, line 10 in >, + , line 4 in lumberjack>, + , line 7 in bright_side_of_death>] *** format_tb: [' File "", line 10, in \n lumberjack()\n', ' File "", line 4, in lumberjack\n bright_side_of_death()\n', -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 01:36:00 2016 From: python-checkins at python.org (zach.ware) Date: Wed, 10 Aug 2016 05:36:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2327208=3A_Merge_with_3=2E5?= Message-ID: <20160810053600.7814.72807.73908D74@psf.io> https://hg.python.org/cpython/rev/76c04e780cd5 changeset: 102596:76c04e780cd5 parent: 102594:ea43895ebfd9 parent: 102595:eadf2905f424 user: Zachary Ware date: Wed Aug 10 00:35:51 2016 -0500 summary: Closes #27208: Merge with 3.5 files: Doc/library/traceback.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/traceback.rst b/Doc/library/traceback.rst --- a/Doc/library/traceback.rst +++ b/Doc/library/traceback.rst @@ -361,7 +361,7 @@ traceback.print_exception(exc_type, exc_value, exc_traceback, limit=2, file=sys.stdout) print("*** print_exc:") - traceback.print_exc() + traceback.print_exc(limit=2, file=sys.stdout) print("*** format_exc, first and last line:") formatted_lines = traceback.format_exc().splitlines() print(formatted_lines[0]) @@ -407,9 +407,9 @@ ' File "", line 7, in bright_side_of_death\n return tuple()[0]\n', 'IndexError: tuple index out of range\n'] *** extract_tb: - [('', 10, '', 'lumberjack()'), - ('', 4, 'lumberjack', 'bright_side_of_death()'), - ('', 7, 'bright_side_of_death', 'return tuple()[0]')] + [, line 10 in >, + , line 4 in lumberjack>, + , line 7 in bright_side_of_death>] *** format_tb: [' File "", line 10, in \n lumberjack()\n', ' File "", line 4, in lumberjack\n bright_side_of_death()\n', -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 01:40:31 2016 From: python-checkins at python.org (zach.ware) Date: Wed, 10 Aug 2016 05:40:31 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3MjA5?= =?utf-8?q?=3A_Fix_doctests_in_Doc/library/email*=2Erst?= Message-ID: <20160810054031.11733.63275.C8685C4F@psf.io> https://hg.python.org/cpython/rev/bb443518141e changeset: 102597:bb443518141e branch: 3.5 parent: 102595:eadf2905f424 user: Zachary Ware date: Wed Aug 10 00:39:41 2016 -0500 summary: Issue #27209: Fix doctests in Doc/library/email*.rst Patch by Jelle Zijlstra. files: Doc/library/email.iterators.rst | 10 +++--- Doc/library/email.message.rst | 20 +++++++------- Doc/library/email.policy.rst | 29 ++++++++++---------- 3 files changed, 29 insertions(+), 30 deletions(-) diff --git a/Doc/library/email.iterators.rst b/Doc/library/email.iterators.rst --- a/Doc/library/email.iterators.rst +++ b/Doc/library/email.iterators.rst @@ -50,9 +50,9 @@ .. testsetup:: - >>> import email - >>> from email.iterators import _structure - >>> somefile = open('Lib/test/test_email/data/msg_02.txt') + import email + from email.iterators import _structure + somefile = open('../Lib/test/test_email/data/msg_02.txt') .. doctest:: @@ -74,9 +74,9 @@ text/plain text/plain - .. testsetup:: + .. testcleanup:: - >>> somefile.close() + somefile.close() Optional *fp* is a file-like object to print the output to. It must be suitable for Python's :func:`print` function. *level* is used internally. diff --git a/Doc/library/email.message.rst b/Doc/library/email.message.rst --- a/Doc/library/email.message.rst +++ b/Doc/library/email.message.rst @@ -602,10 +602,10 @@ .. testsetup:: - >>> from email import message_from_binary_file - >>> with open('Lib/test/test_email/data/msg_16.txt', 'rb') as f: - ... msg = message_from_binary_file(f) - >>> from email.iterators import _structure + from email import message_from_binary_file + with open('../Lib/test/test_email/data/msg_16.txt', 'rb') as f: + msg = message_from_binary_file(f) + from email.iterators import _structure .. doctest:: @@ -628,7 +628,7 @@ .. doctest:: >>> for part in msg.walk(): - ... print(part.get_content_maintype() == 'multipart'), + ... print(part.get_content_maintype() == 'multipart', ... part.is_multipart()) True True False False @@ -640,11 +640,11 @@ >>> _structure(msg) multipart/report text/plain - message/delivery-status - text/plain - text/plain - message/rfc822 - text/plain + message/delivery-status + text/plain + text/plain + message/rfc822 + text/plain Here the ``message`` parts are not ``multiparts``, but they do contain subparts. ``is_multipart()`` returns ``True`` and ``walk`` descends diff --git a/Doc/library/email.policy.rst b/Doc/library/email.policy.rst --- a/Doc/library/email.policy.rst +++ b/Doc/library/email.policy.rst @@ -63,16 +63,15 @@ .. testsetup:: - >>> from unittest import mock - >>> mocker = mock.patch('subprocess.Popen') - >>> m = mocker.start() - >>> proc = mock.MagicMock() - >>> m.return_value = proc - >>> proc.stdin.close.return_value = None - >>> mymsg = open('mymsg.txt', 'w') - >>> mymsg.write('To: abc at xyz.com\n\n') - 17 - >>> mymsg.flush() + from unittest import mock + mocker = mock.patch('subprocess.Popen') + m = mocker.start() + proc = mock.MagicMock() + m.return_value = proc + proc.stdin.close.return_value = None + mymsg = open('mymsg.txt', 'w') + mymsg.write('To: abc at xyz.com\n\n') + mymsg.flush() .. doctest:: @@ -88,12 +87,12 @@ >>> p.stdin.close() >>> rc = p.wait() -.. testsetup:: +.. testcleanup:: - >>> mymsg.close() - >>> mocker.stop() - >>> import os - >>> os.remove('mymsg.txt') + mymsg.close() + mocker.stop() + import os + os.remove('mymsg.txt') Here we are telling :class:`~email.generator.BytesGenerator` to use the RFC correct line separator characters when creating the binary string to feed into -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 01:40:32 2016 From: python-checkins at python.org (zach.ware) Date: Wed, 10 Aug 2016 05:40:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2327209=3A_Merge_with_3=2E5?= Message-ID: <20160810054031.3277.46809.E0E074BA@psf.io> https://hg.python.org/cpython/rev/81dc2d250209 changeset: 102598:81dc2d250209 parent: 102596:76c04e780cd5 parent: 102597:bb443518141e user: Zachary Ware date: Wed Aug 10 00:39:59 2016 -0500 summary: Closes #27209: Merge with 3.5 files: Doc/library/email.iterators.rst | 10 +++--- Doc/library/email.message.rst | 20 +++++++------- Doc/library/email.policy.rst | 29 ++++++++++---------- 3 files changed, 29 insertions(+), 30 deletions(-) diff --git a/Doc/library/email.iterators.rst b/Doc/library/email.iterators.rst --- a/Doc/library/email.iterators.rst +++ b/Doc/library/email.iterators.rst @@ -50,9 +50,9 @@ .. testsetup:: - >>> import email - >>> from email.iterators import _structure - >>> somefile = open('Lib/test/test_email/data/msg_02.txt') + import email + from email.iterators import _structure + somefile = open('../Lib/test/test_email/data/msg_02.txt') .. doctest:: @@ -74,9 +74,9 @@ text/plain text/plain - .. testsetup:: + .. testcleanup:: - >>> somefile.close() + somefile.close() Optional *fp* is a file-like object to print the output to. It must be suitable for Python's :func:`print` function. *level* is used internally. diff --git a/Doc/library/email.message.rst b/Doc/library/email.message.rst --- a/Doc/library/email.message.rst +++ b/Doc/library/email.message.rst @@ -602,10 +602,10 @@ .. testsetup:: - >>> from email import message_from_binary_file - >>> with open('Lib/test/test_email/data/msg_16.txt', 'rb') as f: - ... msg = message_from_binary_file(f) - >>> from email.iterators import _structure + from email import message_from_binary_file + with open('../Lib/test/test_email/data/msg_16.txt', 'rb') as f: + msg = message_from_binary_file(f) + from email.iterators import _structure .. doctest:: @@ -628,7 +628,7 @@ .. doctest:: >>> for part in msg.walk(): - ... print(part.get_content_maintype() == 'multipart'), + ... print(part.get_content_maintype() == 'multipart', ... part.is_multipart()) True True False False @@ -640,11 +640,11 @@ >>> _structure(msg) multipart/report text/plain - message/delivery-status - text/plain - text/plain - message/rfc822 - text/plain + message/delivery-status + text/plain + text/plain + message/rfc822 + text/plain Here the ``message`` parts are not ``multiparts``, but they do contain subparts. ``is_multipart()`` returns ``True`` and ``walk`` descends diff --git a/Doc/library/email.policy.rst b/Doc/library/email.policy.rst --- a/Doc/library/email.policy.rst +++ b/Doc/library/email.policy.rst @@ -63,16 +63,15 @@ .. testsetup:: - >>> from unittest import mock - >>> mocker = mock.patch('subprocess.Popen') - >>> m = mocker.start() - >>> proc = mock.MagicMock() - >>> m.return_value = proc - >>> proc.stdin.close.return_value = None - >>> mymsg = open('mymsg.txt', 'w') - >>> mymsg.write('To: abc at xyz.com\n\n') - 17 - >>> mymsg.flush() + from unittest import mock + mocker = mock.patch('subprocess.Popen') + m = mocker.start() + proc = mock.MagicMock() + m.return_value = proc + proc.stdin.close.return_value = None + mymsg = open('mymsg.txt', 'w') + mymsg.write('To: abc at xyz.com\n\n') + mymsg.flush() .. doctest:: @@ -88,12 +87,12 @@ >>> p.stdin.close() >>> rc = p.wait() -.. testsetup:: +.. testcleanup:: - >>> mymsg.close() - >>> mocker.stop() - >>> import os - >>> os.remove('mymsg.txt') + mymsg.close() + mocker.stop() + import os + os.remove('mymsg.txt') Here we are telling :class:`~email.generator.BytesGenerator` to use the RFC correct line separator characters when creating the binary string to feed into -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 02:01:33 2016 From: python-checkins at python.org (zach.ware) Date: Wed, 10 Aug 2016 06:01:33 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3MjEw?= =?utf-8?q?=3A_Fix_doctests_for_a_few_modules=2E?= Message-ID: <20160810060132.8232.69035.6A8E3824@psf.io> https://hg.python.org/cpython/rev/6b00ccf5e435 changeset: 102599:6b00ccf5e435 branch: 3.5 parent: 102597:bb443518141e user: Zachary Ware date: Wed Aug 10 00:59:59 2016 -0500 summary: Issue #27210: Fix doctests for a few modules. Initial patch by Jelle Zijlstra. files: Doc/library/difflib.rst | 8 +++----- Doc/library/ftplib.rst | 2 +- Doc/library/nntplib.rst | 10 +++++----- 3 files changed, 9 insertions(+), 11 deletions(-) diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst --- a/Doc/library/difflib.rst +++ b/Doc/library/difflib.rst @@ -170,8 +170,7 @@ >>> s1 = ['bacon\n', 'eggs\n', 'ham\n', 'guido\n'] >>> s2 = ['python\n', 'eggy\n', 'hamster\n', 'guido\n'] - >>> for line in context_diff(s1, s2, fromfile='before.py', tofile='after.py'): - ... sys.stdout.write(line) # doctest: +NORMALIZE_WHITESPACE + >>> sys.stdout.writelines(context_diff(s1, s2, fromfile='before.py', tofile='after.py')) *** before.py --- after.py *************** @@ -209,7 +208,7 @@ >>> import keyword >>> get_close_matches('wheel', keyword.kwlist) ['while'] - >>> get_close_matches('apple', keyword.kwlist) + >>> get_close_matches('pineapple', keyword.kwlist) [] >>> get_close_matches('accept', keyword.kwlist) ['except'] @@ -303,8 +302,7 @@ >>> s1 = ['bacon\n', 'eggs\n', 'ham\n', 'guido\n'] >>> s2 = ['python\n', 'eggy\n', 'hamster\n', 'guido\n'] - >>> for line in unified_diff(s1, s2, fromfile='before.py', tofile='after.py'): - ... sys.stdout.write(line) # doctest: +NORMALIZE_WHITESPACE + >>> sys.stdout.writelines(unified_diff(s1, s2, fromfile='before.py', tofile='after.py')) --- before.py +++ after.py @@ -1,4 +1,4 @@ diff --git a/Doc/library/ftplib.rst b/Doc/library/ftplib.rst --- a/Doc/library/ftplib.rst +++ b/Doc/library/ftplib.rst @@ -57,7 +57,7 @@ >>> with FTP("ftp1.at.proftpd.org") as ftp: ... ftp.login() ... ftp.dir() - ... + ... # doctest: +SKIP '230 Anonymous login ok, restrictions apply.' dr-xr-xr-x 9 ftp ftp 154 May 6 10:43 . dr-xr-xr-x 9 ftp ftp 154 May 6 10:43 .. diff --git a/Doc/library/nntplib.rst b/Doc/library/nntplib.rst --- a/Doc/library/nntplib.rst +++ b/Doc/library/nntplib.rst @@ -75,7 +75,7 @@ >>> from nntplib import NNTP >>> with NNTP('news.gmane.org') as n: ... n.group('gmane.comp.python.committers') - ... + ... # doctest: +SKIP ('211 1755 1 1755 gmane.comp.python.committers', 1755, 1, 1755, 'gmane.comp.python.committers') >>> @@ -261,9 +261,9 @@ >>> from datetime import date, timedelta >>> resp, groups = s.newgroups(date.today() - timedelta(days=3)) - >>> len(groups) + >>> len(groups) # doctest: +SKIP 85 - >>> groups[0] + >>> groups[0] # doctest: +SKIP GroupInfo(group='gmane.network.tor.devel', last='4', first='1', flag='m') @@ -312,9 +312,9 @@ is a dictionary mapping group names to textual descriptions. >>> resp, descs = s.descriptions('gmane.comp.python.*') - >>> len(descs) + >>> len(descs) # doctest: +SKIP 295 - >>> descs.popitem() + >>> descs.popitem() # doctest: +SKIP ('gmane.comp.python.bio.general', 'BioPython discussion list (Moderated)') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 02:01:33 2016 From: python-checkins at python.org (zach.ware) Date: Wed, 10 Aug 2016 06:01:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2327210=3A_Merge_with_3=2E5?= Message-ID: <20160810060133.29705.72603.07CE3D92@psf.io> https://hg.python.org/cpython/rev/0a569e4b1d57 changeset: 102600:0a569e4b1d57 parent: 102598:81dc2d250209 parent: 102599:6b00ccf5e435 user: Zachary Ware date: Wed Aug 10 01:01:05 2016 -0500 summary: Closes #27210: Merge with 3.5 files: Doc/library/difflib.rst | 8 +++----- Doc/library/ftplib.rst | 2 +- Doc/library/nntplib.rst | 10 +++++----- 3 files changed, 9 insertions(+), 11 deletions(-) diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst --- a/Doc/library/difflib.rst +++ b/Doc/library/difflib.rst @@ -170,8 +170,7 @@ >>> s1 = ['bacon\n', 'eggs\n', 'ham\n', 'guido\n'] >>> s2 = ['python\n', 'eggy\n', 'hamster\n', 'guido\n'] - >>> for line in context_diff(s1, s2, fromfile='before.py', tofile='after.py'): - ... sys.stdout.write(line) # doctest: +NORMALIZE_WHITESPACE + >>> sys.stdout.writelines(context_diff(s1, s2, fromfile='before.py', tofile='after.py')) *** before.py --- after.py *************** @@ -209,7 +208,7 @@ >>> import keyword >>> get_close_matches('wheel', keyword.kwlist) ['while'] - >>> get_close_matches('apple', keyword.kwlist) + >>> get_close_matches('pineapple', keyword.kwlist) [] >>> get_close_matches('accept', keyword.kwlist) ['except'] @@ -303,8 +302,7 @@ >>> s1 = ['bacon\n', 'eggs\n', 'ham\n', 'guido\n'] >>> s2 = ['python\n', 'eggy\n', 'hamster\n', 'guido\n'] - >>> for line in unified_diff(s1, s2, fromfile='before.py', tofile='after.py'): - ... sys.stdout.write(line) # doctest: +NORMALIZE_WHITESPACE + >>> sys.stdout.writelines(unified_diff(s1, s2, fromfile='before.py', tofile='after.py')) --- before.py +++ after.py @@ -1,4 +1,4 @@ diff --git a/Doc/library/ftplib.rst b/Doc/library/ftplib.rst --- a/Doc/library/ftplib.rst +++ b/Doc/library/ftplib.rst @@ -57,7 +57,7 @@ >>> with FTP("ftp1.at.proftpd.org") as ftp: ... ftp.login() ... ftp.dir() - ... + ... # doctest: +SKIP '230 Anonymous login ok, restrictions apply.' dr-xr-xr-x 9 ftp ftp 154 May 6 10:43 . dr-xr-xr-x 9 ftp ftp 154 May 6 10:43 .. diff --git a/Doc/library/nntplib.rst b/Doc/library/nntplib.rst --- a/Doc/library/nntplib.rst +++ b/Doc/library/nntplib.rst @@ -75,7 +75,7 @@ >>> from nntplib import NNTP >>> with NNTP('news.gmane.org') as n: ... n.group('gmane.comp.python.committers') - ... + ... # doctest: +SKIP ('211 1755 1 1755 gmane.comp.python.committers', 1755, 1, 1755, 'gmane.comp.python.committers') >>> @@ -261,9 +261,9 @@ >>> from datetime import date, timedelta >>> resp, groups = s.newgroups(date.today() - timedelta(days=3)) - >>> len(groups) + >>> len(groups) # doctest: +SKIP 85 - >>> groups[0] + >>> groups[0] # doctest: +SKIP GroupInfo(group='gmane.network.tor.devel', last='4', first='1', flag='m') @@ -312,9 +312,9 @@ is a dictionary mapping group names to textual descriptions. >>> resp, descs = s.descriptions('gmane.comp.python.*') - >>> len(descs) + >>> len(descs) # doctest: +SKIP 295 - >>> descs.popitem() + >>> descs.popitem() # doctest: +SKIP ('gmane.comp.python.bio.general', 'BioPython discussion list (Moderated)') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 02:06:06 2016 From: python-checkins at python.org (zach.ware) Date: Wed, 10 Aug 2016 06:06:06 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3MjAw?= =?utf-8?q?=3A_Fix_doctests_in_Doc/library/hashlib=2Erst?= Message-ID: <20160810060605.29534.85076.34808C35@psf.io> https://hg.python.org/cpython/rev/f3a04d19e5cb changeset: 102601:f3a04d19e5cb branch: 3.5 parent: 102599:6b00ccf5e435 user: Zachary Ware date: Wed Aug 10 01:05:19 2016 -0500 summary: Issue #27200: Fix doctests in Doc/library/hashlib.rst files: Doc/library/hashlib.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -13,6 +13,11 @@ single: message digest, MD5 single: secure hash algorithm, SHA1, SHA224, SHA256, SHA384, SHA512 +.. testsetup:: + + import hashlib + + -------------- This module implements a common interface to many different secure hash and -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 02:06:06 2016 From: python-checkins at python.org (zach.ware) Date: Wed, 10 Aug 2016 06:06:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327200=3A_Merge_with_3=2E5?= Message-ID: <20160810060606.3622.16132.8B8E04D6@psf.io> https://hg.python.org/cpython/rev/d0302d8ecbc1 changeset: 102602:d0302d8ecbc1 parent: 102600:0a569e4b1d57 parent: 102601:f3a04d19e5cb user: Zachary Ware date: Wed Aug 10 01:05:56 2016 -0500 summary: Issue #27200: Merge with 3.5 files: Doc/library/hashlib.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -13,6 +13,11 @@ single: message digest, MD5 single: secure hash algorithm, SHA1, SHA224, SHA256, SHA384, SHA512 +.. testsetup:: + + import hashlib + + -------------- This module implements a common interface to many different secure hash and -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 03:05:24 2016 From: python-checkins at python.org (martin.panter) Date: Wed, 10 Aug 2016 07:05:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_ISO-8859_fixes_from_3=2E5?= Message-ID: <20160810070459.3238.62692.2C885E6E@psf.io> https://hg.python.org/cpython/rev/9c082472d0a2 changeset: 102605:9c082472d0a2 parent: 102602:d0302d8ecbc1 parent: 102604:d692a116bc43 user: Martin Panter date: Wed Aug 10 07:00:33 2016 +0000 summary: Merge ISO-8859 fixes from 3.5 files: Doc/library/http.client.rst | 4 ++-- Lib/email/base64mime.py | 2 +- Lib/email/quoprimime.py | 2 +- Lib/test/test_email/test_email.py | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -227,12 +227,12 @@ If *body* is specified, the specified data is sent after the headers are finished. It may be a string, a :term:`bytes-like object`, an open :term:`file object`, or an iterable of :term:`bytes-like object`\s. If - *body* is a string, it is encoded as ISO-8851-1, the default for HTTP. If + *body* is a string, it is encoded as ISO-8859-1, the default for HTTP. If it is a bytes-like object the bytes are sent as is. If it is a :term:`file object`, the contents of the file is sent; this file object should support at least the ``read()`` method. If the file object has a ``mode`` attribute, the data returned by the ``read()`` method will be encoded as - ISO-8851-1 unless the ``mode`` attribute contains the substring ``b``, + ISO-8859-1 unless the ``mode`` attribute contains the substring ``b``, otherwise the data returned by ``read()`` is sent as is. If *body* is an iterable, the elements of the iterable are sent as is until the iterable is exhausted. diff --git a/Lib/email/base64mime.py b/Lib/email/base64mime.py --- a/Lib/email/base64mime.py +++ b/Lib/email/base64mime.py @@ -103,7 +103,7 @@ """Decode a raw base64 string, returning a bytes object. This function does not parse a full MIME header value encoded with - base64 (like =?iso-8895-1?b?bmloISBuaWgh?=) -- please use the high + base64 (like =?iso-8859-1?b?bmloISBuaWgh?=) -- please use the high level email.header class for that functionality. """ if not string: diff --git a/Lib/email/quoprimime.py b/Lib/email/quoprimime.py --- a/Lib/email/quoprimime.py +++ b/Lib/email/quoprimime.py @@ -292,7 +292,7 @@ """Decode a string encoded with RFC 2045 MIME header `Q' encoding. This function does not parse a full MIME header value encoded with - quoted-printable (like =?iso-8895-1?q?Hello_World?=) -- please use + quoted-printable (like =?iso-8859-1?q?Hello_World?=) -- please use the high level email.header class for that functionality. """ s = s.replace('_', ' ') diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -2302,9 +2302,9 @@ def test_rfc2047_Q_invalid_digits(self): # issue 10004. - s = '=?iso-8659-1?Q?andr=e9=zz?=' + s = '=?iso-8859-1?Q?andr=e9=zz?=' self.assertEqual(decode_header(s), - [(b'andr\xe9=zz', 'iso-8659-1')]) + [(b'andr\xe9=zz', 'iso-8859-1')]) def test_rfc2047_rfc2047_1(self): # 1st testcase at end of rfc2047 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 03:07:08 2016 From: python-checkins at python.org (martin.panter) Date: Wed, 10 Aug 2016 07:07:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Correct_misspe?= =?utf-8?q?llings_of_ISO-8859?= Message-ID: <20160810070459.29460.57576.9A5ECA14@psf.io> https://hg.python.org/cpython/rev/d692a116bc43 changeset: 102604:d692a116bc43 branch: 3.5 parent: 102601:f3a04d19e5cb user: Martin Panter date: Wed Aug 10 05:25:16 2016 +0000 summary: Correct misspellings of ISO-8859 files: Doc/library/http.client.rst | 4 ++-- Lib/email/base64mime.py | 2 +- Lib/email/quoprimime.py | 2 +- Lib/test/test_email/test_email.py | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -227,12 +227,12 @@ If *body* is specified, the specified data is sent after the headers are finished. It may be a string, a :term:`bytes-like object`, an open :term:`file object`, or an iterable of :term:`bytes-like object`\s. If - *body* is a string, it is encoded as ISO-8851-1, the default for HTTP. If + *body* is a string, it is encoded as ISO-8859-1, the default for HTTP. If it is a bytes-like object the bytes are sent as is. If it is a :term:`file object`, the contents of the file is sent; this file object should support at least the ``read()`` method. If the file object has a ``mode`` attribute, the data returned by the ``read()`` method will be encoded as - ISO-8851-1 unless the ``mode`` attribute contains the substring ``b``, + ISO-8859-1 unless the ``mode`` attribute contains the substring ``b``, otherwise the data returned by ``read()`` is sent as is. If *body* is an iterable, the elements of the iterable are sent as is until the iterable is exhausted. diff --git a/Lib/email/base64mime.py b/Lib/email/base64mime.py --- a/Lib/email/base64mime.py +++ b/Lib/email/base64mime.py @@ -103,7 +103,7 @@ """Decode a raw base64 string, returning a bytes object. This function does not parse a full MIME header value encoded with - base64 (like =?iso-8895-1?b?bmloISBuaWgh?=) -- please use the high + base64 (like =?iso-8859-1?b?bmloISBuaWgh?=) -- please use the high level email.header class for that functionality. """ if not string: diff --git a/Lib/email/quoprimime.py b/Lib/email/quoprimime.py --- a/Lib/email/quoprimime.py +++ b/Lib/email/quoprimime.py @@ -292,7 +292,7 @@ """Decode a string encoded with RFC 2045 MIME header `Q' encoding. This function does not parse a full MIME header value encoded with - quoted-printable (like =?iso-8895-1?q?Hello_World?=) -- please use + quoted-printable (like =?iso-8859-1?q?Hello_World?=) -- please use the high level email.header class for that functionality. """ s = s.replace('_', ' ') diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -2302,9 +2302,9 @@ def test_rfc2047_Q_invalid_digits(self): # issue 10004. - s = '=?iso-8659-1?Q?andr=e9=zz?=' + s = '=?iso-8859-1?Q?andr=e9=zz?=' self.assertEqual(decode_header(s), - [(b'andr\xe9=zz', 'iso-8659-1')]) + [(b'andr\xe9=zz', 'iso-8859-1')]) def test_rfc2047_rfc2047_1(self): # 1st testcase at end of rfc2047 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 03:07:08 2016 From: python-checkins at python.org (martin.panter) Date: Wed, 10 Aug 2016 07:07:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Correct_misspe?= =?utf-8?q?llings_of_ISO-8859?= Message-ID: <20160810070459.6895.38386.FFE8630A@psf.io> https://hg.python.org/cpython/rev/5efdef26c821 changeset: 102603:5efdef26c821 branch: 2.7 parent: 102588:e399930d14f0 user: Martin Panter date: Wed Aug 10 05:25:16 2016 +0000 summary: Correct misspellings of ISO-8859 files: Lib/email/base64mime.py | 2 +- Lib/email/quoprimime.py | 2 +- Lib/email/test/test_email.py | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Lib/email/base64mime.py b/Lib/email/base64mime.py --- a/Lib/email/base64mime.py +++ b/Lib/email/base64mime.py @@ -166,7 +166,7 @@ decoding a text attachment. This function does not parse a full MIME header value encoded with - base64 (like =?iso-8895-1?b?bmloISBuaWgh?=) -- please use the high + base64 (like =?iso-8859-1?b?bmloISBuaWgh?=) -- please use the high level email.header class for that functionality. """ if not s: diff --git a/Lib/email/quoprimime.py b/Lib/email/quoprimime.py --- a/Lib/email/quoprimime.py +++ b/Lib/email/quoprimime.py @@ -329,7 +329,7 @@ """Decode a string encoded with RFC 2045 MIME header `Q' encoding. This function does not parse a full MIME header value encoded with - quoted-printable (like =?iso-8895-1?q?Hello_World?=) -- please use + quoted-printable (like =?iso-8859-1?q?Hello_World?=) -- please use the high level email.header class for that functionality. """ s = s.replace('_', ' ') diff --git a/Lib/email/test/test_email.py b/Lib/email/test/test_email.py --- a/Lib/email/test/test_email.py +++ b/Lib/email/test/test_email.py @@ -1673,9 +1673,9 @@ def test_rfc2047_Q_invalid_digits(self): # issue 10004. - s = '=?iso-8659-1?Q?andr=e9=zz?=' + s = '=?iso-8859-1?Q?andr=e9=zz?=' self.assertEqual(decode_header(s), - [(b'andr\xe9=zz', 'iso-8659-1')]) + [(b'andr\xe9=zz', 'iso-8859-1')]) # Test the MIMEMessage class -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Wed Aug 10 12:22:42 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 10 Aug 2016 17:22:42 +0100 Subject: [Python-checkins] GOOD Benchmark Results for Python 2.7 2016-08-10 Message-ID: Results for project Python 2.7, build date 2016-08-10 02:47:53 +0000 commit: e399930d14f0 previous commit: 09475e6135d0 revision date: 2016-08-09 22:28:33 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.10% 1.58% 5.31% 5.17% :-) pybench 0.22% 0.02% 5.68% 5.75% :-( regex_v8 0.64% 0.04% -2.34% 11.06% :-) nbody 0.29% 0.15% 8.64% 2.76% :-| json_dump_v2 0.30% -0.05% 1.87% 10.45% :-( normal_startup 1.71% 0.38% -5.57% 1.41% :-) ssbench 0.20% -0.31% 2.36% 1.80% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-2-7-2016-08-10/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Wed Aug 10 12:24:36 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 10 Aug 2016 17:24:36 +0100 Subject: [Python-checkins] GOOD Benchmark Results for Python Default 2016-08-10 Message-ID: <88b143c3-e6c8-4074-8f72-c4f89a0ca517@irsmsx103.ger.corp.intel.com> Results for project Python default, build date 2016-08-10 02:02:08 +0000 commit: 61d7aeb6aa1b previous commit: d69f782d642d revision date: 2016-08-09 22:45:15 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.25% 1.36% 11.87% 14.10% :-) pybench 0.18% -0.07% 5.90% 7.40% :-( regex_v8 2.69% -0.02% -2.86% 8.25% :-| nbody 0.26% 0.35% 1.46% 9.06% :-| json_dump_v2 0.31% 0.75% -0.08% 10.59% :-| normal_startup 0.80% 0.41% -1.24% 6.16% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-08-10/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Wed Aug 10 12:50:29 2016 From: python-checkins at python.org (terry.reedy) Date: Wed, 10 Aug 2016 16:50:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327621=3A_Put_quer?= =?utf-8?q?y_response_validation_error_messages_in_query_box?= Message-ID: <20160810165029.8245.10645.1FA58D02@psf.io> https://hg.python.org/cpython/rev/f0e86b60de5f changeset: 102606:f0e86b60de5f user: Terry Jan Reedy date: Wed Aug 10 12:50:16 2016 -0400 summary: Issue #27621: Put query response validation error messages in query box instead of in separate massagebox. Redo tests to match. Add Mac OSX refinements. Original patch by Mark Roseman. files: Lib/idlelib/idle_test/test_query.py | 235 ++++++--------- Lib/idlelib/query.py | 122 ++++--- 2 files changed, 164 insertions(+), 193 deletions(-) diff --git a/Lib/idlelib/idle_test/test_query.py b/Lib/idlelib/idle_test/test_query.py --- a/Lib/idlelib/idle_test/test_query.py +++ b/Lib/idlelib/idle_test/test_query.py @@ -16,21 +16,9 @@ from tkinter import Tk import unittest from unittest import mock -from idlelib.idle_test.mock_tk import Var, Mbox_func +from idlelib.idle_test.mock_tk import Var from idlelib import query -# Mock entry.showerror messagebox so don't need click to continue -# when entry_ok and path_ok methods call it to display errors. - -orig_showerror = query.showerror -showerror = Mbox_func() # Instance has __call__ method. - -def setUpModule(): - query.showerror = showerror - -def tearDownModule(): - query.showerror = orig_showerror - # NON-GUI TESTS @@ -42,59 +30,49 @@ entry_ok = query.Query.entry_ok ok = query.Query.ok cancel = query.Query.cancel - # Add attributes needed for the tests. + # Add attributes and initialization needed for tests. entry = Var() - result = None - destroyed = False + entry_error = {} + def __init__(self, dummy_entry): + self.entry.set(dummy_entry) + self.entry_error['text'] = '' + self.result = None + self.destroyed = False + def showerror(self, message): + self.entry_error['text'] = message def destroy(self): self.destroyed = True - dialog = Dummy_Query() - - def setUp(self): - showerror.title = None - self.dialog.result = None - self.dialog.destroyed = False - def test_entry_ok_blank(self): - dialog = self.dialog - Equal = self.assertEqual - dialog.entry.set(' ') - Equal(dialog.entry_ok(), None) - Equal((dialog.result, dialog.destroyed), (None, False)) - Equal(showerror.title, 'Entry Error') - self.assertIn('Blank', showerror.message) + dialog = self.Dummy_Query(' ') + self.assertEqual(dialog.entry_ok(), None) + self.assertEqual((dialog.result, dialog.destroyed), (None, False)) + self.assertIn('blank line', dialog.entry_error['text']) def test_entry_ok_good(self): - dialog = self.dialog + dialog = self.Dummy_Query(' good ') Equal = self.assertEqual - dialog.entry.set(' good ') Equal(dialog.entry_ok(), 'good') Equal((dialog.result, dialog.destroyed), (None, False)) - Equal(showerror.title, None) + Equal(dialog.entry_error['text'], '') def test_ok_blank(self): - dialog = self.dialog - Equal = self.assertEqual - dialog.entry.set('') + dialog = self.Dummy_Query('') dialog.entry.focus_set = mock.Mock() - Equal(dialog.ok(), None) + self.assertEqual(dialog.ok(), None) self.assertTrue(dialog.entry.focus_set.called) del dialog.entry.focus_set - Equal((dialog.result, dialog.destroyed), (None, False)) + self.assertEqual((dialog.result, dialog.destroyed), (None, False)) def test_ok_good(self): - dialog = self.dialog - Equal = self.assertEqual - dialog.entry.set('good') - Equal(dialog.ok(), None) - Equal((dialog.result, dialog.destroyed), ('good', True)) + dialog = self.Dummy_Query('good') + self.assertEqual(dialog.ok(), None) + self.assertEqual((dialog.result, dialog.destroyed), ('good', True)) def test_cancel(self): - dialog = self.dialog - Equal = self.assertEqual - Equal(self.dialog.cancel(), None) - Equal((dialog.result, dialog.destroyed), (None, True)) + dialog = self.Dummy_Query('does not matter') + self.assertEqual(dialog.cancel(), None) + self.assertEqual((dialog.result, dialog.destroyed), (None, True)) class SectionNameTest(unittest.TestCase): @@ -104,42 +82,32 @@ entry_ok = query.SectionName.entry_ok # Function being tested. used_names = ['used'] entry = Var() - - dialog = Dummy_SectionName() - - def setUp(self): - showerror.title = None + entry_error = {} + def __init__(self, dummy_entry): + self.entry.set(dummy_entry) + self.entry_error['text'] = '' + def showerror(self, message): + self.entry_error['text'] = message def test_blank_section_name(self): - dialog = self.dialog - Equal = self.assertEqual - dialog.entry.set(' ') - Equal(dialog.entry_ok(), None) - Equal(showerror.title, 'Name Error') - self.assertIn('No', showerror.message) + dialog = self.Dummy_SectionName(' ') + self.assertEqual(dialog.entry_ok(), None) + self.assertIn('no name', dialog.entry_error['text']) def test_used_section_name(self): - dialog = self.dialog - Equal = self.assertEqual - dialog.entry.set('used') - Equal(self.dialog.entry_ok(), None) - Equal(showerror.title, 'Name Error') - self.assertIn('use', showerror.message) + dialog = self.Dummy_SectionName('used') + self.assertEqual(dialog.entry_ok(), None) + self.assertIn('use', dialog.entry_error['text']) def test_long_section_name(self): - dialog = self.dialog - Equal = self.assertEqual - dialog.entry.set('good'*8) - Equal(self.dialog.entry_ok(), None) - Equal(showerror.title, 'Name Error') - self.assertIn('too long', showerror.message) + dialog = self.Dummy_SectionName('good'*8) + self.assertEqual(dialog.entry_ok(), None) + self.assertIn('longer than 30', dialog.entry_error['text']) def test_good_section_name(self): - dialog = self.dialog - Equal = self.assertEqual - dialog.entry.set(' good ') - Equal(dialog.entry_ok(), 'good') - Equal(showerror.title, None) + dialog = self.Dummy_SectionName(' good ') + self.assertEqual(dialog.entry_ok(), 'good') + self.assertEqual(dialog.entry_error['text'], '') class ModuleNameTest(unittest.TestCase): @@ -149,42 +117,32 @@ entry_ok = query.ModuleName.entry_ok # Function being tested. text0 = '' entry = Var() - - dialog = Dummy_ModuleName() - - def setUp(self): - showerror.title = None + entry_error = {} + def __init__(self, dummy_entry): + self.entry.set(dummy_entry) + self.entry_error['text'] = '' + def showerror(self, message): + self.entry_error['text'] = message def test_blank_module_name(self): - dialog = self.dialog - Equal = self.assertEqual - dialog.entry.set(' ') - Equal(dialog.entry_ok(), None) - Equal(showerror.title, 'Name Error') - self.assertIn('No', showerror.message) + dialog = self.Dummy_ModuleName(' ') + self.assertEqual(dialog.entry_ok(), None) + self.assertIn('no name', dialog.entry_error['text']) def test_bogus_module_name(self): - dialog = self.dialog - Equal = self.assertEqual - dialog.entry.set('__name_xyz123_should_not_exist__') - Equal(self.dialog.entry_ok(), None) - Equal(showerror.title, 'Import Error') - self.assertIn('not found', showerror.message) + dialog = self.Dummy_ModuleName('__name_xyz123_should_not_exist__') + self.assertEqual(dialog.entry_ok(), None) + self.assertIn('not found', dialog.entry_error['text']) def test_c_source_name(self): - dialog = self.dialog - Equal = self.assertEqual - dialog.entry.set('itertools') - Equal(self.dialog.entry_ok(), None) - Equal(showerror.title, 'Import Error') - self.assertIn('source-based', showerror.message) + dialog = self.Dummy_ModuleName('itertools') + self.assertEqual(dialog.entry_ok(), None) + self.assertIn('source-based', dialog.entry_error['text']) def test_good_module_name(self): - dialog = self.dialog - Equal = self.assertEqual - dialog.entry.set('idlelib') + dialog = self.Dummy_ModuleName('idlelib') self.assertTrue(dialog.entry_ok().endswith('__init__.py')) - Equal(showerror.title, None) + self.assertEqual(dialog.entry_error['text'], '') # 3 HelpSource test classes each test one function. @@ -198,13 +156,13 @@ browse_file = query.HelpSource.browse_file pathvar = Var() - dialog = Dummy_HelpSource() - def test_file_replaces_path(self): - # Path is widget entry, file is file dialog return. - dialog = self.dialog + dialog = self.Dummy_HelpSource() + # Path is widget entry, either '' or something. + # Func return is file dialog return, either '' or something. + # Func return should override widget entry. + # We need all 4 combination to test all (most) code paths. for path, func, result in ( - # We need all combination to test all (most) code paths. ('', lambda a,b,c:'', ''), ('', lambda a,b,c: __file__, __file__), ('htest', lambda a,b,c:'', 'htest'), @@ -217,78 +175,72 @@ class HelpsourcePathokTest(unittest.TestCase): - "Test path_ok method of ModuleName subclass of Query." + "Test path_ok method of HelpSource subclass of Query." class Dummy_HelpSource: path_ok = query.HelpSource.path_ok path = Var() - - dialog = Dummy_HelpSource() + path_error = {} + def __init__(self, dummy_path): + self.path.set(dummy_path) + self.path_error['text'] = '' + def showerror(self, message, widget=None): + self.path_error['text'] = message @classmethod def tearDownClass(cls): query.platform = orig_platform - def setUp(self): - showerror.title = None - def test_path_ok_blank(self): - dialog = self.dialog - Equal = self.assertEqual - dialog.path.set(' ') - Equal(dialog.path_ok(), None) - Equal(showerror.title, 'File Path Error') - self.assertIn('No help', showerror.message) + dialog = self.Dummy_HelpSource(' ') + self.assertEqual(dialog.path_ok(), None) + self.assertIn('no help file', dialog.path_error['text']) def test_path_ok_bad(self): - dialog = self.dialog - Equal = self.assertEqual - dialog.path.set(__file__ + 'bad-bad-bad') - Equal(dialog.path_ok(), None) - Equal(showerror.title, 'File Path Error') - self.assertIn('not exist', showerror.message) + dialog = self.Dummy_HelpSource(__file__ + 'bad-bad-bad') + self.assertEqual(dialog.path_ok(), None) + self.assertIn('not exist', dialog.path_error['text']) def test_path_ok_web(self): - dialog = self.dialog + dialog = self.Dummy_HelpSource('') Equal = self.assertEqual for url in 'www.py.org', 'http://py.org': with self.subTest(): dialog.path.set(url) - Equal(dialog.path_ok(), url) - Equal(showerror.title, None) + self.assertEqual(dialog.path_ok(), url) + self.assertEqual(dialog.path_error['text'], '') def test_path_ok_file(self): - dialog = self.dialog - Equal = self.assertEqual + dialog = self.Dummy_HelpSource('') for platform, prefix in ('darwin', 'file://'), ('other', ''): with self.subTest(): query.platform = platform dialog.path.set(__file__) - Equal(dialog.path_ok(), prefix + __file__) - Equal(showerror.title, None) + self.assertEqual(dialog.path_ok(), prefix + __file__) + self.assertEqual(dialog.path_error['text'], '') class HelpsourceEntryokTest(unittest.TestCase): - "Test entry_ok method of ModuleName subclass of Query." + "Test entry_ok method of HelpSource subclass of Query." class Dummy_HelpSource: entry_ok = query.HelpSource.entry_ok + entry_error = {} + path_error = {} def item_ok(self): return self.name def path_ok(self): return self.path - dialog = Dummy_HelpSource() - def test_entry_ok_helpsource(self): - dialog = self.dialog + dialog = self.Dummy_HelpSource() for name, path, result in ((None, None, None), (None, 'doc.txt', None), ('doc', None, None), ('doc', 'doc.txt', ('doc', 'doc.txt'))): with self.subTest(): dialog.name, dialog.path = name, path - self.assertEqual(self.dialog.entry_ok(), result) + self.assertEqual(dialog.entry_ok(), result) # GUI TESTS @@ -344,10 +296,10 @@ root = Tk() dialog = query.SectionName(root, 'T', 't', {'abc'}, _utest=True) Equal = self.assertEqual - Equal(dialog.used_names, {'abc'}) + self.assertEqual(dialog.used_names, {'abc'}) dialog.entry.insert(0, 'okay') dialog.button_ok.invoke() - Equal(dialog.result, 'okay') + self.assertEqual(dialog.result, 'okay') del dialog root.destroy() del root @@ -362,9 +314,8 @@ def test_click_module_name(self): root = Tk() dialog = query.ModuleName(root, 'T', 't', 'idlelib', _utest=True) - Equal = self.assertEqual - Equal(dialog.text0, 'idlelib') - Equal(dialog.entry.get(), 'idlelib') + self.assertEqual(dialog.text0, 'idlelib') + self.assertEqual(dialog.entry.get(), 'idlelib') dialog.button_ok.invoke() self.assertTrue(dialog.result.endswith('__init__.py')) del dialog diff --git a/Lib/idlelib/query.py b/Lib/idlelib/query.py --- a/Lib/idlelib/query.py +++ b/Lib/idlelib/query.py @@ -10,6 +10,8 @@ Subclass SectionName gets a name for a new config file section. Configdialog uses it for new highlight theme and keybinding set names. +Subclass ModuleName gets a name for File => Open Module. +Subclass HelpSource gets menu item and path for additions to Help menu. """ # Query and Section name result from splitting GetCfgSectionNameDialog # of configSectionNameDialog.py (temporarily config_sec.py) into @@ -21,10 +23,10 @@ import importlib import os from sys import executable, platform # Platform is set for one test. -from tkinter import Toplevel, StringVar +from tkinter import Toplevel, StringVar, W, E, N, S from tkinter import filedialog -from tkinter.messagebox import showerror from tkinter.ttk import Frame, Button, Entry, Label +from tkinter.font import Font class Query(Toplevel): """Base class for getting verified answer from a user. @@ -47,18 +49,26 @@ """ Toplevel.__init__(self, parent) self.withdraw() # Hide while configuring, especially geometry. - self.configure(borderwidth=5) - self.resizable(height=False, width=False) + self.parent = parent self.title(title) - self.transient(parent) - self.grab_set() - self.bind('', self.ok) - self.bind('', self.cancel) - self.protocol("WM_DELETE_WINDOW", self.cancel) - self.parent = parent self.message = message self.text0 = text0 self.used_names = used_names + self.transient(parent) + self.grab_set() + windowingsystem = self.tk.call('tk', 'windowingsystem') + if windowingsystem == 'aqua': + try: + self.tk.call('::tk::unsupported::MacWindowStyle', 'style', + self._w, 'moveableModal', '') + except: + pass + self.bind("", self.cancel) + self.bind('', self.cancel) + self.protocol("WM_DELETE_WINDOW", self.cancel) + self.bind('', self.ok) + self.bind("", self.ok) + self.resizable(height=False, width=False) self.create_widgets() self.update_idletasks() # Needed here for winfo_reqwidth below. self.geometry( # Center dialog over parent (or below htest box). @@ -75,32 +85,42 @@ def create_widgets(self): # Call from override, if any. # Bind to self widgets needed for entry_ok or unittest. - self.frame = frame = Frame(self, borderwidth=2, relief='sunken', ) + self.frame = frame = Frame(self, padding=10) + frame.grid(column=0, row=0, sticky='news') + frame.grid_columnconfigure(0, weight=1) + entrylabel = Label(frame, anchor='w', justify='left', text=self.message) self.entryvar = StringVar(self, self.text0) self.entry = Entry(frame, width=30, textvariable=self.entryvar) self.entry.focus_set() + self.error_font = Font(name='TkCaptionFont', + exists=True, root=self.parent) + self.entry_error = Label(frame, text=' ', foreground='red', + font=self.error_font) + self.button_ok = Button( + frame, text='OK', default='active', command=self.ok) + self.button_cancel = Button( + frame, text='Cancel', command=self.cancel) - buttons = Frame(self) - self.button_ok = Button(buttons, text='Ok', default='active', - width=8, command=self.ok) - self.button_cancel = Button(buttons, text='Cancel', - width=8, command=self.cancel) + entrylabel.grid(column=0, row=0, columnspan=3, padx=5, sticky=W) + self.entry.grid(column=0, row=1, columnspan=3, padx=5, sticky=W+E, + pady=[10,0]) + self.entry_error.grid(column=0, row=2, columnspan=3, padx=5, + sticky=W+E) + self.button_ok.grid(column=1, row=99, padx=5) + self.button_cancel.grid(column=2, row=99, padx=5) - frame.pack(side='top', expand=True, fill='both') - entrylabel.pack(padx=5, pady=5) - self.entry.pack(padx=5, pady=5) - buttons.pack(side='bottom') - self.button_ok.pack(side='left', padx=5) - self.button_cancel.pack(side='right', padx=5) + def showerror(self, message, widget=None): + #self.bell(displayof=self) + (widget or self.entry_error)['text'] = 'ERROR: ' + message def entry_ok(self): # Example: usually replace. "Return non-blank entry or None." + self.entry_error['text'] = '' entry = self.entry.get().strip() if not entry: - showerror(title='Entry Error', - message='Blank line.', parent=self) + self.showerror('blank line.') return None return entry @@ -134,19 +154,16 @@ def entry_ok(self): "Return sensible ConfigParser section name or None." + self.entry_error['text'] = '' name = self.entry.get().strip() if not name: - showerror(title='Name Error', - message='No name specified.', parent=self) + self.showerror('no name specified.') return None elif len(name)>30: - showerror(title='Name Error', - message='Name too long. It should be no more than '+ - '30 characters.', parent=self) + self.showerror('name is longer than 30 characters.') return None elif name in self.used_names: - showerror(title='Name Error', - message='This name is already in use.', parent=self) + self.showerror('name is already in use.') return None return name @@ -162,30 +179,27 @@ def entry_ok(self): "Return entered module name as file path or None." + self.entry_error['text'] = '' name = self.entry.get().strip() if not name: - showerror(title='Name Error', - message='No name specified.', parent=self) + self.showerror('no name specified.') return None # XXX Ought to insert current file's directory in front of path. try: spec = importlib.util.find_spec(name) except (ValueError, ImportError) as msg: - showerror("Import Error", str(msg), parent=self) + self.showerror(str(msg)) return None if spec is None: - showerror("Import Error", "module not found", - parent=self) + self.showerror("module not found") return None if not isinstance(spec.loader, importlib.abc.SourceLoader): - showerror("Import Error", "not a source-based module", - parent=self) + self.showerror("not a source-based module") return None try: file_path = spec.loader.get_filename(name) except AttributeError: - showerror("Import Error", - "loader does not support get_filename", + self.showerror("loader does not support get_filename", parent=self) return None return file_path @@ -204,8 +218,9 @@ """ self.filepath = filepath message = 'Name for item on Help menu:' - super().__init__(parent, title, message, text0=menuitem, - used_names=used_names, _htest=_htest, _utest=_utest) + super().__init__( + parent, title, message, text0=menuitem, + used_names=used_names, _htest=_htest, _utest=_utest) def create_widgets(self): super().create_widgets() @@ -216,10 +231,16 @@ self.path = Entry(frame, textvariable=self.pathvar, width=40) browse = Button(frame, text='Browse', width=8, command=self.browse_file) + self.path_error = Label(frame, text=' ', foreground='red', + font=self.error_font) - pathlabel.pack(anchor='w', padx=5, pady=3) - self.path.pack(anchor='w', padx=5, pady=3) - browse.pack(pady=3) + pathlabel.grid(column=0, row=10, columnspan=3, padx=5, pady=[10,0], + sticky=W) + self.path.grid(column=0, row=11, columnspan=2, padx=5, sticky=W+E, + pady=[10,0]) + browse.grid(column=2, row=11, padx=5, sticky=W+S) + self.path_error.grid(column=0, row=12, columnspan=3, padx=5, + sticky=W+E) def askfilename(self, filetypes, initdir, initfile): # htest # # Extracted from browse_file so can mock for unittests. @@ -256,17 +277,14 @@ "Simple validity check for menu file path" path = self.path.get().strip() if not path: #no path specified - showerror(title='File Path Error', - message='No help file path specified.', - parent=self) + self.showerror('no help file path specified.', self.path_error) return None elif not path.startswith(('www.', 'http')): if path[:5] == 'file:': path = path[5:] if not os.path.exists(path): - showerror(title='File Path Error', - message='Help file path does not exist.', - parent=self) + self.showerror('help file path does not exist.', + self.path_error) return None if platform == 'darwin': # for Mac Safari path = "file://" + path @@ -274,6 +292,8 @@ def entry_ok(self): "Return apparently valid (name, path) or None" + self.entry_error['text'] = '' + self.path_error['text'] = '' name = self.item_ok() path = self.path_ok() return None if name is None or path is None else (name, path) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 13:16:41 2016 From: python-checkins at python.org (terry.reedy) Date: Wed, 10 Aug 2016 17:16:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327380=3A_For_test?= =?utf-8?q?=5Fquery_on_Mac=2C_adjust_one_expected_result=2E?= Message-ID: <20160810171640.3708.72769.BDD9F2BA@psf.io> https://hg.python.org/cpython/rev/8f37d772f71f changeset: 102607:8f37d772f71f user: Terry Jan Reedy date: Wed Aug 10 13:16:26 2016 -0400 summary: Issue #27380: For test_query on Mac, adjust one expected result. files: Lib/idlelib/idle_test/test_query.py | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/idle_test/test_query.py b/Lib/idlelib/idle_test/test_query.py --- a/Lib/idlelib/idle_test/test_query.py +++ b/Lib/idlelib/idle_test/test_query.py @@ -13,6 +13,7 @@ 6 of 8 missing are ModuleName exceptions I don't know how to trigger. """ from test.support import requires +import sys from tkinter import Tk import unittest from unittest import mock @@ -337,7 +338,8 @@ Equal(dialog.entry.get(), '__test__') Equal(dialog.path.get(), __file__) dialog.button_ok.invoke() - Equal(dialog.result, ('__test__', __file__)) + prefix = "file://" if sys.platform == 'darwin' else '' + Equal(dialog.result, ('__test__', prefix + __file__)) del dialog root.destroy() del root -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 15:15:39 2016 From: python-checkins at python.org (terry.reedy) Date: Wed, 10 Aug 2016 19:15:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327714=3A_text=5Ft?= =?utf-8?q?extview_now_passes_when_re-run_in_the_same_process?= Message-ID: <20160810191539.29307.77122.76BCC24A@psf.io> https://hg.python.org/cpython/rev/b10a312f6d00 changeset: 102608:b10a312f6d00 user: Terry Jan Reedy date: Wed Aug 10 15:15:25 2016 -0400 summary: Issue #27714: text_textview now passes when re-run in the same process because test_idle failed while running with test -w (and no -jn). Prevent a non-fatal warning from test_config_key. files: Lib/idlelib/idle_test/test_config_key.py | 3 ++- Lib/idlelib/idle_test/test_textview.py | 3 +-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Lib/idlelib/idle_test/test_config_key.py b/Lib/idlelib/idle_test/test_config_key.py --- a/Lib/idlelib/idle_test/test_config_key.py +++ b/Lib/idlelib/idle_test/test_config_key.py @@ -1,6 +1,6 @@ ''' Test idlelib.config_key. -Coverage: 56% +Coverage: 56% from creating and closing dialog. ''' from idlelib import config_key from test.support import requires @@ -17,6 +17,7 @@ @classmethod def tearDownClass(cls): + cls.root.update() # Stop "can't run event command" warning. cls.root.destroy() del cls.root diff --git a/Lib/idlelib/idle_test/test_textview.py b/Lib/idlelib/idle_test/test_textview.py --- a/Lib/idlelib/idle_test/test_textview.py +++ b/Lib/idlelib/idle_test/test_textview.py @@ -22,8 +22,7 @@ root = Tk() def tearDownModule(): - global root, TV - del TV + global root root.update_idletasks() root.destroy() # pyflakes falsely sees root as undefined del root -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 16:54:49 2016 From: python-checkins at python.org (terry.reedy) Date: Wed, 10 Aug 2016 20:54:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NzE0?= =?utf-8?q?=3A_text=5Ftextview_now_passes_when_re-run_in_the_same_process?= Message-ID: <20160810205448.16407.86372.09978CE9@psf.io> https://hg.python.org/cpython/rev/dc31fa499359 changeset: 102609:dc31fa499359 branch: 3.5 parent: 102604:d692a116bc43 user: Terry Jan Reedy date: Wed Aug 10 16:52:24 2016 -0400 summary: Issue #27714: text_textview now passes when re-run in the same process because test_idle failed while running with test -w (and no -jn). files: Lib/idlelib/idle_test/test_textview.py | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/idle_test/test_textview.py b/Lib/idlelib/idle_test/test_textview.py --- a/Lib/idlelib/idle_test/test_textview.py +++ b/Lib/idlelib/idle_test/test_textview.py @@ -22,8 +22,7 @@ root = Tk() def tearDownModule(): - global root, TV - del TV + global root root.update_idletasks() root.destroy() # pyflakes falsely sees root as undefined del root -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 16:54:51 2016 From: python-checkins at python.org (terry.reedy) Date: Wed, 10 Aug 2016 20:54:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_null_merge_from3=2E5?= Message-ID: <20160810205448.3230.86232.262500BB@psf.io> https://hg.python.org/cpython/rev/ba8d87a27335 changeset: 102610:ba8d87a27335 parent: 102608:b10a312f6d00 parent: 102609:dc31fa499359 user: Terry Jan Reedy date: Wed Aug 10 16:54:31 2016 -0400 summary: null merge from3.5 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 19:41:53 2016 From: python-checkins at python.org (terry.reedy) Date: Wed, 10 Aug 2016 23:41:53 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3NzE0?= =?utf-8?q?=3A_For_IDLE=27s_test=5Ftextview=2C_backport_3=2Ex_subclass_wit?= =?utf-8?q?h_mocks?= Message-ID: <20160810234153.3292.37695.C14C53C7@psf.io> https://hg.python.org/cpython/rev/0de509a79181 changeset: 102611:0de509a79181 branch: 2.7 parent: 102603:5efdef26c821 user: Terry Jan Reedy date: Wed Aug 10 19:41:39 2016 -0400 summary: Issue #27714: For IDLE's test_textview, backport 3.x subclass with mocks instead of overriding methods with mocks in original class and module. This makes the 2.7 test_textview nearly identical to the 3.5/.6 test. files: Lib/idlelib/idle_test/test_textview.py | 24 ++++++------- 1 files changed, 11 insertions(+), 13 deletions(-) diff --git a/Lib/idlelib/idle_test/test_textview.py b/Lib/idlelib/idle_test/test_textview.py --- a/Lib/idlelib/idle_test/test_textview.py +++ b/Lib/idlelib/idle_test/test_textview.py @@ -8,7 +8,11 @@ from idlelib.idle_test.mock_idle import Func from idlelib.idle_test.mock_tk import Mbox -orig_mbox = tv.tkMessageBox + +class TV(tv.TextViewer): # used by TextViewTest + transient = Func() + grab_set = Func() + wait_window = Func() class textviewClassTest(unittest.TestCase): @@ -16,26 +20,18 @@ def setUpClass(cls): requires('gui') cls.root = Tk() - cls.TV = TV = tv.TextViewer - TV.transient = Func() - TV.grab_set = Func() - TV.wait_window = Func() @classmethod def tearDownClass(cls): - del cls.TV cls.root.destroy() del cls.root def setUp(self): - TV = self.TV TV.transient.__init__() TV.grab_set.__init__() TV.wait_window.__init__() - def test_init_modal(self): - TV = self.TV view = TV(self.root, 'Title', 'test text') self.assertTrue(TV.transient.called) self.assertTrue(TV.grab_set.called) @@ -43,7 +39,6 @@ view.Ok() def test_init_nonmodal(self): - TV = self.TV view = TV(self.root, 'Title', 'test text', modal=False) self.assertFalse(TV.transient.called) self.assertFalse(TV.grab_set.called) @@ -51,7 +46,7 @@ view.Ok() def test_ok(self): - view = self.TV(self.root, 'Title', 'test text', modal=False) + view = TV(self.root, 'Title', 'test text', modal=False) view.destroy = Func() view.Ok() self.assertTrue(view.destroy.called) @@ -59,19 +54,21 @@ view.destroy -class textviewTest(unittest.TestCase): +class ViewFunctionTest(unittest.TestCase): @classmethod def setUpClass(cls): requires('gui') cls.root = Tk() + cls.orig_mbox = tv.tkMessageBox tv.tkMessageBox = Mbox @classmethod def tearDownClass(cls): cls.root.destroy() del cls.root - tv.tkMessageBox = orig_mbox + tv.tkMessageBox = cls.orig_mbox + del cls.orig_mbox def test_view_text(self): # If modal True, tkinter will error with 'can't invoke "event" command' @@ -91,5 +88,6 @@ view = tv.view_file(self.root, 'Title', testfile, modal=False) self.assertIsNone(view) + if __name__ == '__main__': unittest.main(verbosity=2) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 10 23:45:09 2016 From: python-checkins at python.org (terry.reedy) Date: Thu, 11 Aug 2016 03:45:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327732=3A_Silence_?= =?utf-8?q?test=5Fidle_with_dummy_bell_functions=2E?= Message-ID: <20160811034509.3161.39447.1AB04A48@psf.io> https://hg.python.org/cpython/rev/2eb84fe85889 changeset: 102612:2eb84fe85889 parent: 102610:ba8d87a27335 user: Terry Jan Reedy date: Wed Aug 10 23:44:54 2016 -0400 summary: Issue #27732: Silence test_idle with dummy bell functions. files: Lib/idlelib/autoexpand.py | 6 +++- Lib/idlelib/idle_test/test_autoexpand.py | 2 + Lib/idlelib/idle_test/test_parenmatch.py | 13 ++++++++--- Lib/idlelib/idle_test/test_replace.py | 6 ++-- Lib/idlelib/idle_test/test_search.py | 2 + Lib/idlelib/idle_test/test_undo.py | 2 +- Lib/idlelib/parenmatch.py | 9 ++----- Lib/idlelib/replace.py | 8 +++--- Lib/idlelib/search.py | 4 +- Lib/idlelib/searchbase.py | 3 +- 10 files changed, 32 insertions(+), 23 deletions(-) diff --git a/Lib/idlelib/autoexpand.py b/Lib/idlelib/autoexpand.py --- a/Lib/idlelib/autoexpand.py +++ b/Lib/idlelib/autoexpand.py @@ -31,6 +31,7 @@ def __init__(self, editwin): self.text = editwin.text + self.bell = self.text.bell self.state = None def expand_word_event(self, event): @@ -46,14 +47,14 @@ words = self.getwords() index = 0 if not words: - self.text.bell() + self.bell() return "break" word = self.getprevword() self.text.delete("insert - %d chars" % len(word), "insert") newword = words[index] index = (index + 1) % len(words) if index == 0: - self.text.bell() # Warn we cycled around + self.bell() # Warn we cycled around self.text.insert("insert", newword) curinsert = self.text.index("insert") curline = self.text.get("insert linestart", "insert lineend") @@ -99,6 +100,7 @@ i = i-1 return line[i:] + if __name__ == '__main__': import unittest unittest.main('idlelib.idle_test.test_autoexpand', verbosity=2) diff --git a/Lib/idlelib/idle_test/test_autoexpand.py b/Lib/idlelib/idle_test/test_autoexpand.py --- a/Lib/idlelib/idle_test/test_autoexpand.py +++ b/Lib/idlelib/idle_test/test_autoexpand.py @@ -22,6 +22,7 @@ else: cls.text = Text() cls.auto_expand = AutoExpand(Dummy_Editwin(cls.text)) + cls.auto_expand.bell = lambda: None @classmethod def tearDownClass(cls): @@ -137,5 +138,6 @@ new_state = self.auto_expand.state self.assertNotEqual(initial_state, new_state) + if __name__ == '__main__': unittest.main(verbosity=2) diff --git a/Lib/idlelib/idle_test/test_parenmatch.py b/Lib/idlelib/idle_test/test_parenmatch.py --- a/Lib/idlelib/idle_test/test_parenmatch.py +++ b/Lib/idlelib/idle_test/test_parenmatch.py @@ -38,12 +38,17 @@ def tearDown(self): self.text.delete('1.0', 'end') + def get_parenmatch(self): + pm = ParenMatch(self.editwin) + pm.bell = lambda: None + return pm + def test_paren_expression(self): """ Test ParenMatch with 'expression' style. """ text = self.text - pm = ParenMatch(self.editwin) + pm = self.get_parenmatch() pm.set_style('expression') text.insert('insert', 'def foobar(a, b') @@ -66,7 +71,7 @@ Test ParenMatch with 'default' style. """ text = self.text - pm = ParenMatch(self.editwin) + pm = self.get_parenmatch() pm.set_style('default') text.insert('insert', 'def foobar(a, b') @@ -86,7 +91,7 @@ These cases force conditional expression and alternate paths. """ text = self.text - pm = ParenMatch(self.editwin) + pm = self.get_parenmatch() text.insert('insert', '# this is a commen)') self.assertIsNone(pm.paren_closed_event('event')) @@ -99,7 +104,7 @@ self.assertIsNone(pm.paren_closed_event('event')) def test_handle_restore_timer(self): - pm = ParenMatch(self.editwin) + pm = self.get_parenmatch() pm.restore_event = Mock() pm.handle_restore_timer(0) self.assertTrue(pm.restore_event.called) diff --git a/Lib/idlelib/idle_test/test_replace.py b/Lib/idlelib/idle_test/test_replace.py --- a/Lib/idlelib/idle_test/test_replace.py +++ b/Lib/idlelib/idle_test/test_replace.py @@ -7,7 +7,7 @@ from tkinter import Tk, Text from idlelib.idle_test.mock_tk import Mbox import idlelib.searchengine as se -import idlelib.replace as rd +from idlelib.replace import ReplaceDialog orig_mbox = se.tkMessageBox showerror = Mbox.showerror @@ -21,7 +21,8 @@ cls.root.withdraw() se.tkMessageBox = Mbox cls.engine = se.SearchEngine(cls.root) - cls.dialog = rd.ReplaceDialog(cls.root, cls.engine) + cls.dialog = ReplaceDialog(cls.root, cls.engine) + cls.dialog.bell = lambda: None cls.dialog.ok = Mock() cls.text = Text(cls.root) cls.text.undo_block_start = Mock() @@ -70,7 +71,6 @@ # text found and replaced pv.set('a') rv.set('asdf') - self.dialog.open(self.text) replace() equal(text.get('1.8', '1.12'), 'asdf') diff --git a/Lib/idlelib/idle_test/test_search.py b/Lib/idlelib/idle_test/test_search.py --- a/Lib/idlelib/idle_test/test_search.py +++ b/Lib/idlelib/idle_test/test_search.py @@ -29,6 +29,7 @@ def setUp(self): self.engine = se.SearchEngine(self.root) self.dialog = sd.SearchDialog(self.root, self.engine) + self.dialog.bell = lambda: None self.text = tk.Text(self.root) self.text.insert('1.0', 'Hello World!') @@ -38,6 +39,7 @@ self.engine.setpat('') self.assertFalse(self.dialog.find_again(text)) + self.dialog.bell = lambda: None self.engine.setpat('Hello') self.assertTrue(self.dialog.find_again(text)) diff --git a/Lib/idlelib/idle_test/test_undo.py b/Lib/idlelib/idle_test/test_undo.py --- a/Lib/idlelib/idle_test/test_undo.py +++ b/Lib/idlelib/idle_test/test_undo.py @@ -29,8 +29,8 @@ def setUp(self): self.delegator = UndoDelegator() + self.delegator.bell = Mock() self.percolator.insertfilter(self.delegator) - self.delegator.bell = Mock(wraps=self.delegator.bell) def tearDown(self): self.percolator.removefilter(self.delegator) diff --git a/Lib/idlelib/parenmatch.py b/Lib/idlelib/parenmatch.py --- a/Lib/idlelib/parenmatch.py +++ b/Lib/idlelib/parenmatch.py @@ -64,6 +64,7 @@ # and deactivate_restore (which calls event_delete). editwin.text.bind(self.RESTORE_VIRTUAL_EVENT_NAME, self.restore_event) + self.bell = self.text.bell if self.BELL else lambda: None self.counter = 0 self.is_restore_active = 0 self.set_style(self.STYLE) @@ -93,7 +94,7 @@ indices = (HyperParser(self.editwin, "insert") .get_surrounding_brackets()) if indices is None: - self.warn_mismatched() + self.bell() return self.activate_restore() self.create_tag(indices) @@ -109,7 +110,7 @@ return indices = hp.get_surrounding_brackets(_openers[closer], True) if indices is None: - self.warn_mismatched() + self.bell() return self.activate_restore() self.create_tag(indices) @@ -124,10 +125,6 @@ if timer_count == self.counter: self.restore_event() - def warn_mismatched(self): - if self.BELL: - self.text.bell() - # any one of the create_tag_XXX methods can be used depending on # the style diff --git a/Lib/idlelib/replace.py b/Lib/idlelib/replace.py --- a/Lib/idlelib/replace.py +++ b/Lib/idlelib/replace.py @@ -95,7 +95,7 @@ text = self.text res = self.engine.search_text(text, prog) if not res: - text.bell() + self.bell() return text.tag_remove("sel", "1.0", "end") text.tag_remove("hit", "1.0", "end") @@ -142,7 +142,7 @@ text = self.text res = self.engine.search_text(text, None, ok) if not res: - text.bell() + self.bell() return False line, m = res i, j = m.span() @@ -204,8 +204,8 @@ def _replace_dialog(parent): # htest # - from tkinter import Toplevel, Text - from tkiter.ttk import Button + from tkinter import Toplevel, Text, END, SEL + from tkinter.ttk import Button box = Toplevel(parent) box.title("Test ReplaceDialog") diff --git a/Lib/idlelib/search.py b/Lib/idlelib/search.py --- a/Lib/idlelib/search.py +++ b/Lib/idlelib/search.py @@ -51,7 +51,7 @@ selfirst = text.index("sel.first") sellast = text.index("sel.last") if selfirst == first and sellast == last: - text.bell() + self.bell() return False except TclError: pass @@ -61,7 +61,7 @@ text.see("insert") return True else: - text.bell() + self.bell() return False def find_selection(self, text): diff --git a/Lib/idlelib/searchbase.py b/Lib/idlelib/searchbase.py --- a/Lib/idlelib/searchbase.py +++ b/Lib/idlelib/searchbase.py @@ -79,6 +79,7 @@ top.wm_title(self.title) top.wm_iconname(self.icon) self.top = top + self.bell = top.bell self.row = 0 self.top.grid_columnconfigure(0, pad=2, weight=0) @@ -188,7 +189,7 @@ width,height, x,y = list(map(int, re.split('[x+]', parent.geometry()))) self.top.geometry("+%d+%d" % (x + 40, y + 175)) - def default_command(self): pass + def default_command(self, dummy): pass if __name__ == '__main__': import unittest -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 11 11:01:47 2016 From: python-checkins at python.org (alexander.belopolsky) Date: Thu, 11 Aug 2016 15:01:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2324773=3A_Fix_and_?= =?utf-8?q?speed-up_ZoneInfoCompleteTest=2E?= Message-ID: <20160811150146.3211.90651.9DD8184D@psf.io> https://hg.python.org/cpython/rev/05120447f2c6 changeset: 102613:05120447f2c6 user: Alexander Belopolsky date: Thu Aug 11 11:01:45 2016 -0400 summary: Issue #24773: Fix and speed-up ZoneInfoCompleteTest. * Read the zone.tab file for the list of zones to exclude the aliases. * Skip Casablanca and El_Aaiun October 2037 transitions. files: Lib/test/datetimetester.py | 26 +++++++++++++------------- 1 files changed, 13 insertions(+), 13 deletions(-) diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -15,7 +15,6 @@ import random import struct import unittest -import sysconfig from array import array @@ -4591,13 +4590,16 @@ def zonenames(cls, zonedir=None): if zonedir is None: zonedir = cls.zoneroot - for root, _, files in os.walk(zonedir): - for f in files: - p = os.path.join(root, f) - with open(p, 'rb') as o: - magic = o.read(4) - if magic == b'TZif': - yield p[len(zonedir) + 1:] + zone_tab = os.path.join(zonedir, 'zone.tab') + try: + f = open(zone_tab) + except OSError: + return + with f: + for line in f: + line = line.strip() + if line and not line.startswith('#'): + yield line.split()[2] @classmethod def stats(cls, start_year=1): @@ -4692,7 +4694,6 @@ zonename = 'America/New_York' def setUp(self): - self.sizeof_time_t = sysconfig.get_config_var('SIZEOF_TIME_T') if sys.platform == "win32": self.skipTest("Skipping zoneinfo tests on Windows") try: @@ -4765,12 +4766,11 @@ try: _time.tzset() for udt, shift in tz.transitions(): - if self.zonename == 'Europe/Tallinn' and udt.date() == date(1999, 10, 31): + if (self.zonename == 'Europe/Tallinn' and udt.date() == date(1999, 10, 31) or + self.zonename.endswith(('Casablanca', 'El_Aaiun')) and + udt.date() == date(2037, 10, 4)): print("Skip %s %s transition" % (self.zonename, udt)) continue - if self.sizeof_time_t == 4 and udt.year >= 2037: - print("Skip %s %s transition for 32-bit time_t" % (self.zonename, udt)) - continue s0 = (udt - datetime(1970, 1, 1)) // SEC ss = shift // SEC # shift seconds for x in [-40 * 3600, -20*3600, -1, 0, -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Thu Aug 11 11:12:56 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 11 Aug 2016 16:12:56 +0100 Subject: [Python-checkins] GOOD Benchmark Results for Python 2.7 2016-08-11 Message-ID: <8906bbed-d580-4795-8aa3-431e1e6b247e@irsmsx104.ger.corp.intel.com> Results for project Python 2.7, build date 2016-08-11 02:46:01 +0000 commit: 0de509a79181 previous commit: e399930d14f0 revision date: 2016-08-10 23:41:39 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.10% 0.32% 5.61% 4.39% :-) pybench 0.22% -0.60% 5.11% 5.05% :-( regex_v8 0.61% 0.09% -2.25% 11.04% :-) nbody 0.13% 0.16% 8.78% 3.78% :-| json_dump_v2 0.29% -0.47% 1.41% 12.41% :-) normal_startup 0.97% 5.37% 0.10% 1.27% :-) ssbench 0.05% -0.13% 2.22% 2.33% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-2-7-2016-08-11/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Thu Aug 11 11:20:07 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 11 Aug 2016 16:20:07 +0100 Subject: [Python-checkins] UGLY Benchmark Results for Python Default 2016-08-11 Message-ID: Results for project Python default, build date 2016-08-11 02:00:25 +0000 commit: ba8d87a27335 previous commit: 61d7aeb6aa1b revision date: 2016-08-10 20:54:31 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.25% -1.62% 10.44% 16.42% :-) pybench 0.15% 0.02% 5.92% 7.61% :-( regex_v8 2.69% -0.01% -2.86% 7.96% :-| nbody 0.16% -0.15% 1.31% 6.70% :-| json_dump_v2 0.35% 0.06% -0.02% 9.45% :-) normal_startup 0.87% 3.08% 1.39% 5.59% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-default-2016-08-11/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Fri Aug 12 08:18:33 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 12 Aug 2016 12:18:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Close_HTTP_con?= =?utf-8?q?nections_and_responses_in_tests_to_avoid_ResourceWarnings?= Message-ID: <20160812121833.10771.86401.F824281F@psf.io> https://hg.python.org/cpython/rev/6a335f59c2ba changeset: 102614:6a335f59c2ba branch: 3.5 parent: 102609:dc31fa499359 user: Martin Panter date: Fri Aug 12 11:59:52 2016 +0000 summary: Close HTTP connections and responses in tests to avoid ResourceWarnings files: Lib/test/test_httplib.py | 12 +++++++++++- 1 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -1402,6 +1402,7 @@ resp = h.getresponse() h.close() self.assertIn('nginx', resp.getheader('server')) + resp.close() @support.system_must_validate_cert def test_networked_trusted_by_default_cert(self): @@ -1412,6 +1413,7 @@ h.request('GET', '/') resp = h.getresponse() content_type = resp.getheader('content-type') + resp.close() h.close() self.assertIn('text/html', content_type) @@ -1427,6 +1429,7 @@ h.request('GET', '/') resp = h.getresponse() server_string = resp.getheader('server') + resp.close() h.close() self.assertIn('nginx', server_string) @@ -1460,8 +1463,10 @@ context.verify_mode = ssl.CERT_REQUIRED context.load_verify_locations(CERT_localhost) h = client.HTTPSConnection('localhost', server.port, context=context) + self.addCleanup(h.close) h.request('GET', '/nonexistent') resp = h.getresponse() + self.addCleanup(resp.close) self.assertEqual(resp.status, 404) def test_local_bad_hostname(self): @@ -1486,13 +1491,18 @@ check_hostname=False) h.request('GET', '/nonexistent') resp = h.getresponse() + resp.close() + h.close() self.assertEqual(resp.status, 404) # The context's check_hostname setting is used if one isn't passed to # HTTPSConnection. context.check_hostname = False h = client.HTTPSConnection('localhost', server.port, context=context) h.request('GET', '/nonexistent') - self.assertEqual(h.getresponse().status, 404) + resp = h.getresponse() + self.assertEqual(resp.status, 404) + resp.close() + h.close() # Passing check_hostname to HTTPSConnection should override the # context's setting. h = client.HTTPSConnection('localhost', server.port, context=context, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 12 08:18:38 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 12 Aug 2016 12:18:38 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2NzQx?= =?utf-8?q?=3A_Clean_up_subprocess=2EPopen_object_in_test=5Fpoll?= Message-ID: <20160812121833.76008.88009.A7136685@psf.io> https://hg.python.org/cpython/rev/f78a682a3515 changeset: 102615:f78a682a3515 branch: 3.5 user: Martin Panter date: Fri Aug 12 11:59:52 2016 +0000 summary: Issue #26741: Clean up subprocess.Popen object in test_poll files: Lib/test/test_poll.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_poll.py b/Lib/test/test_poll.py --- a/Lib/test/test_poll.py +++ b/Lib/test/test_poll.py @@ -125,6 +125,8 @@ cmd = 'for i in 0 1 2 3 4 5 6 7 8 9; do echo testing...; sleep 1; done' proc = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, bufsize=0) + proc.__enter__() + self.addCleanup(proc.__exit__, None, None, None) p = proc.stdout pollster = select.poll() pollster.register( p, select.POLLIN ) @@ -147,7 +149,6 @@ continue else: self.fail('Unexpected return value from select.poll: %s' % fdlist) - p.close() def test_poll3(self): # test int overflow -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 12 08:18:38 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 12 Aug 2016 12:18:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326741=3A_Merge_ResourceWarning_fixes_from_3=2E5?= Message-ID: <20160812121834.62840.65972.DBA4ADE9@psf.io> https://hg.python.org/cpython/rev/7ff3ce0dfd45 changeset: 102617:7ff3ce0dfd45 parent: 102613:05120447f2c6 parent: 102615:f78a682a3515 user: Martin Panter date: Fri Aug 12 12:04:27 2016 +0000 summary: Issue #26741: Merge ResourceWarning fixes from 3.5 files: Lib/test/test_httplib.py | 12 +++++++++++- Lib/test/test_poll.py | 3 ++- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -1402,6 +1402,7 @@ resp = h.getresponse() h.close() self.assertIn('nginx', resp.getheader('server')) + resp.close() @support.system_must_validate_cert def test_networked_trusted_by_default_cert(self): @@ -1412,6 +1413,7 @@ h.request('GET', '/') resp = h.getresponse() content_type = resp.getheader('content-type') + resp.close() h.close() self.assertIn('text/html', content_type) @@ -1427,6 +1429,7 @@ h.request('GET', '/') resp = h.getresponse() server_string = resp.getheader('server') + resp.close() h.close() self.assertIn('nginx', server_string) @@ -1460,8 +1463,10 @@ context.verify_mode = ssl.CERT_REQUIRED context.load_verify_locations(CERT_localhost) h = client.HTTPSConnection('localhost', server.port, context=context) + self.addCleanup(h.close) h.request('GET', '/nonexistent') resp = h.getresponse() + self.addCleanup(resp.close) self.assertEqual(resp.status, 404) def test_local_bad_hostname(self): @@ -1486,13 +1491,18 @@ check_hostname=False) h.request('GET', '/nonexistent') resp = h.getresponse() + resp.close() + h.close() self.assertEqual(resp.status, 404) # The context's check_hostname setting is used if one isn't passed to # HTTPSConnection. context.check_hostname = False h = client.HTTPSConnection('localhost', server.port, context=context) h.request('GET', '/nonexistent') - self.assertEqual(h.getresponse().status, 404) + resp = h.getresponse() + self.assertEqual(resp.status, 404) + resp.close() + h.close() # Passing check_hostname to HTTPSConnection should override the # context's setting. h = client.HTTPSConnection('localhost', server.port, context=context, diff --git a/Lib/test/test_poll.py b/Lib/test/test_poll.py --- a/Lib/test/test_poll.py +++ b/Lib/test/test_poll.py @@ -125,6 +125,8 @@ cmd = 'for i in 0 1 2 3 4 5 6 7 8 9; do echo testing...; sleep 1; done' proc = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, bufsize=0) + proc.__enter__() + self.addCleanup(proc.__exit__, None, None, None) p = proc.stdout pollster = select.poll() pollster.register( p, select.POLLIN ) @@ -147,7 +149,6 @@ continue else: self.fail('Unexpected return value from select.poll: %s' % fdlist) - p.close() def test_poll3(self): # test int overflow -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 12 08:18:38 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 12 Aug 2016 12:18:38 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NzQ1?= =?utf-8?q?=3A_Fix_some_typos_in_Argument_Clinic_howto=2C_by_Lele_Gaifax?= Message-ID: <20160812121833.67868.922.3AF2E2DF@psf.io> https://hg.python.org/cpython/rev/129c601e7bfa changeset: 102616:129c601e7bfa branch: 3.5 user: Martin Panter date: Fri Aug 12 12:02:03 2016 +0000 summary: Issue #27745: Fix some typos in Argument Clinic howto, by Lele Gaifax files: Doc/howto/clinic.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -653,7 +653,7 @@ Functions that use *any* other approach for parsing arguments should *almost never* be converted to Argument Clinic using optional groups. Functions using optional groups currently - cannot have accurate sigantures in Python, because Python just + cannot have accurate signatures in Python, because Python just doesn't understand the concept. Please avoid using optional groups wherever possible. @@ -1337,7 +1337,7 @@ While changing Clinic's output in this manner can be a boon to readability, it may result in Clinic code using types before they are defined, or -your code attempting to use Clinic-generated code befire it is defined. +your code attempting to use Clinic-generated code before it is defined. These problems can be easily solved by rearranging the declarations in your file, or moving where Clinic's generated code goes. (This is why the default behavior of Clinic is to output everything into the current block; while many people @@ -1381,7 +1381,7 @@ ``buffer`` A text buffer where you can save text for later. Text sent - here is appended to the end of any exsiting text. It's an + here is appended to the end of any existing text. It's an error to have any text left in the buffer when Clinic finishes processing a file. @@ -1653,7 +1653,7 @@ However, this causes one ticklish problem: where should Argument Clinic put this extra code when using the "block" output preset? It can't go in the output block, -because that could be decativated by the ``#ifdef``. (That's the whole point!) +because that could be deactivated by the ``#ifdef``. (That's the whole point!) In this situation, Argument Clinic writes the extra code to the "buffer" destination. This may mean that you get a complaint from Argument Clinic:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 12 08:18:38 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 12 Aug 2016 12:18:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327745=3A_Merge_typo_fixes_from_3=2E5?= Message-ID: <20160812121834.76195.38495.66964555@psf.io> https://hg.python.org/cpython/rev/8c33152fd75c changeset: 102618:8c33152fd75c parent: 102617:7ff3ce0dfd45 parent: 102616:129c601e7bfa user: Martin Panter date: Fri Aug 12 12:05:48 2016 +0000 summary: Issue #27745: Merge typo fixes from 3.5 files: Doc/howto/clinic.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -653,7 +653,7 @@ Functions that use *any* other approach for parsing arguments should *almost never* be converted to Argument Clinic using optional groups. Functions using optional groups currently - cannot have accurate sigantures in Python, because Python just + cannot have accurate signatures in Python, because Python just doesn't understand the concept. Please avoid using optional groups wherever possible. @@ -1337,7 +1337,7 @@ While changing Clinic's output in this manner can be a boon to readability, it may result in Clinic code using types before they are defined, or -your code attempting to use Clinic-generated code befire it is defined. +your code attempting to use Clinic-generated code before it is defined. These problems can be easily solved by rearranging the declarations in your file, or moving where Clinic's generated code goes. (This is why the default behavior of Clinic is to output everything into the current block; while many people @@ -1381,7 +1381,7 @@ ``buffer`` A text buffer where you can save text for later. Text sent - here is appended to the end of any exsiting text. It's an + here is appended to the end of any existing text. It's an error to have any text left in the buffer when Clinic finishes processing a file. @@ -1653,7 +1653,7 @@ However, this causes one ticklish problem: where should Argument Clinic put this extra code when using the "block" output preset? It can't go in the output block, -because that could be decativated by the ``#ifdef``. (That's the whole point!) +because that could be deactivated by the ``#ifdef``. (That's the whole point!) In this situation, Argument Clinic writes the extra code to the "buffer" destination. This may mean that you get a complaint from Argument Clinic:: -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Fri Aug 12 08:22:38 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Fri, 12 Aug 2016 13:22:38 +0100 Subject: [Python-checkins] GOOD Benchmark Results for Python 2.7 2016-08-12 Message-ID: <6abe9a8a-4fb7-42b0-b441-1e2bedc74ac3@irsmsx151.ger.corp.intel.com> No new revisions. Here are the previous results: Results for project Python 2.7, build date 2016-08-12 02:45:14 +0000 commit: 0de509a79181 previous commit: e399930d14f0 revision date: 2016-08-10 23:41:39 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.10% 0.32% 5.61% 4.39% :-) pybench 0.22% -0.60% 5.11% 5.05% :-( regex_v8 0.61% 0.09% -2.25% 11.04% :-) nbody 0.13% 0.16% 8.78% 3.78% :-| json_dump_v2 0.29% -0.47% 1.41% 12.41% :-) normal_startup 0.97% 5.37% 0.10% 1.27% :-) ssbench 0.05% -0.13% 2.22% 2.33% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-2-7-2016-08-12/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Fri Aug 12 08:24:10 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Fri, 12 Aug 2016 13:24:10 +0100 Subject: [Python-checkins] NEUTRAL Benchmark Results for Python Default 2016-08-12 Message-ID: Results for project Python default, build date 2016-08-12 01:59:59 +0000 commit: 05120447f2c6 previous commit: ba8d87a27335 revision date: 2016-08-11 15:01:45 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.23% 0.20% 10.62% 16.34% :-) pybench 0.09% 0.01% 5.93% 7.15% :-( regex_v8 2.73% -0.02% -2.89% 8.22% :-| nbody 0.16% 0.06% 1.36% 7.78% :-| json_dump_v2 0.32% -0.00% -0.02% 11.27% :-| normal_startup 0.65% -0.61% 0.61% 6.18% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-default-2016-08-12/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Fri Aug 12 09:27:37 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 12 Aug 2016 13:27:37 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1ODI1?= =?utf-8?q?=3A_Fix_references_to_Modules/python=2Eexp?= Message-ID: <20160812132736.7404.24483.370FE350@psf.io> https://hg.python.org/cpython/rev/58c8cae6c61a changeset: 102619:58c8cae6c61a branch: 3.5 parent: 102616:129c601e7bfa user: Martin Panter date: Fri Aug 12 13:21:05 2016 +0000 summary: Issue #25825: Fix references to Modules/python.exp These were incorrectly changed in revision 88a532a31eb3. files: Makefile.pre.in | 6 +++--- Misc/NEWS | 3 +++ 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1407,10 +1407,10 @@ $(INSTALL_SCRIPT) $(srcdir)/install-sh $(DESTDIR)$(LIBPL)/install-sh $(INSTALL_SCRIPT) python-config.py $(DESTDIR)$(LIBPL)/python-config.py $(INSTALL_SCRIPT) python-config $(DESTDIR)$(BINDIR)/python$(LDVERSION)-config - @if [ -s Programs/python.exp -a \ + @if [ -s Modules/python.exp -a \ "`echo $(MACHDEP) | sed 's/^\(...\).*/\1/'`" = "aix" ]; then \ echo; echo "Installing support files for building shared extension modules on AIX:"; \ - $(INSTALL_DATA) Programs/python.exp \ + $(INSTALL_DATA) Modules/python.exp \ $(DESTDIR)$(LIBPL)/python.exp; \ echo; echo "$(LIBPL)/python.exp"; \ $(INSTALL_SCRIPT) $(srcdir)/Modules/makexp_aix \ @@ -1620,7 +1620,7 @@ done -rm -f core Makefile Makefile.pre config.status \ Modules/Setup Modules/Setup.local Modules/Setup.config \ - Modules/ld_so_aix Programs/python.exp Misc/python.pc + Modules/ld_so_aix Modules/python.exp Misc/python.pc -rm -f python*-gdb.py find $(srcdir)/[a-zA-Z]* '(' -name '*.fdc' -o -name '*~' \ -o -name '[@,#]*' -o -name '*.old' \ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -154,6 +154,9 @@ Build ----- +- Issue #25825: Correct the references to Modules/python.exp, which is + required on AIX. The references were accidentally changed in 3.5.0a1. + - Issue #27453: CPP invocation in configure must use CPPFLAGS. Patch by Chi Hsuan Yen. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 12 09:27:37 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 12 Aug 2016 13:27:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325825=3A_Merge_AIX_fix_from_3=2E5?= Message-ID: <20160812132736.1395.43292.556BFA76@psf.io> https://hg.python.org/cpython/rev/4d4b5b978b7e changeset: 102620:4d4b5b978b7e parent: 102618:8c33152fd75c parent: 102619:58c8cae6c61a user: Martin Panter date: Fri Aug 12 13:26:01 2016 +0000 summary: Issue #25825: Merge AIX fix from 3.5 files: Makefile.pre.in | 6 +++--- Misc/NEWS | 3 +++ 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1422,10 +1422,10 @@ $(INSTALL_SCRIPT) $(srcdir)/install-sh $(DESTDIR)$(LIBPL)/install-sh $(INSTALL_SCRIPT) python-config.py $(DESTDIR)$(LIBPL)/python-config.py $(INSTALL_SCRIPT) python-config $(DESTDIR)$(BINDIR)/python$(LDVERSION)-config - @if [ -s Programs/python.exp -a \ + @if [ -s Modules/python.exp -a \ "`echo $(MACHDEP) | sed 's/^\(...\).*/\1/'`" = "aix" ]; then \ echo; echo "Installing support files for building shared extension modules on AIX:"; \ - $(INSTALL_DATA) Programs/python.exp \ + $(INSTALL_DATA) Modules/python.exp \ $(DESTDIR)$(LIBPL)/python.exp; \ echo; echo "$(LIBPL)/python.exp"; \ $(INSTALL_SCRIPT) $(srcdir)/Modules/makexp_aix \ @@ -1638,7 +1638,7 @@ done -rm -f core Makefile Makefile.pre config.status \ Modules/Setup Modules/Setup.local Modules/Setup.config \ - Modules/ld_so_aix Programs/python.exp Misc/python.pc + Modules/ld_so_aix Modules/python.exp Misc/python.pc -rm -f python*-gdb.py find $(srcdir)/[a-zA-Z]* '(' -name '*.fdc' -o -name '*~' \ -o -name '[@,#]*' -o -name '*.old' \ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -116,6 +116,9 @@ Build ----- +- Issue #25825: Correct the references to Modules/python.exp, which is + required on AIX. The references were accidentally changed in 3.5.0a1. + - Issue #27453: CPP invocation in configure must use CPPFLAGS. Patch by Chi Hsuan Yen. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 12 12:44:25 2016 From: python-checkins at python.org (raymond.hettinger) Date: Fri, 12 Aug 2016 16:44:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgMjc3MTk6?= =?utf-8?q?__Remove_a_doc_example_that_is_not_applicable_in_Python_3?= Message-ID: <20160812164424.76003.96595.D8018982@psf.io> https://hg.python.org/cpython/rev/6fdd29a9d5d4 changeset: 102621:6fdd29a9d5d4 branch: 3.5 parent: 102619:58c8cae6c61a user: Raymond Hettinger date: Fri Aug 12 09:43:59 2016 -0700 summary: Issue 27719: Remove a doc example that is not applicable in Python 3 files: Doc/tutorial/errors.rst | 24 +----------------------- 1 files changed, 1 insertions(+), 23 deletions(-) diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -244,29 +244,7 @@ Programs may name their own exceptions by creating a new exception class (see :ref:`tut-classes` for more about Python classes). Exceptions should typically -be derived from the :exc:`Exception` class, either directly or indirectly. For -example:: - - >>> class MyError(Exception): - ... def __init__(self, value): - ... self.value = value - ... def __str__(self): - ... return repr(self.value) - ... - >>> try: - ... raise MyError(2*2) - ... except MyError as e: - ... print('My exception occurred, value:', e.value) - ... - My exception occurred, value: 4 - >>> raise MyError('oops!') - Traceback (most recent call last): - File "", line 1, in ? - __main__.MyError: 'oops!' - -In this example, the default :meth:`__init__` of :class:`Exception` has been -overridden. The new behavior simply creates the *value* attribute. This -replaces the default behavior of creating the *args* attribute. +be derived from the :exc:`Exception` class, either directly or indirectly. Exception classes can be defined which do anything any other class can do, but are usually kept simple, often only offering a number of attributes that allow -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 12 12:44:25 2016 From: python-checkins at python.org (raymond.hettinger) Date: Fri, 12 Aug 2016 16:44:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge?= Message-ID: <20160812164425.62886.87812.F5563AC5@psf.io> https://hg.python.org/cpython/rev/0c3adb3b0570 changeset: 102622:0c3adb3b0570 parent: 102620:4d4b5b978b7e parent: 102621:6fdd29a9d5d4 user: Raymond Hettinger date: Fri Aug 12 09:44:18 2016 -0700 summary: merge files: Doc/tutorial/errors.rst | 24 +----------------------- 1 files changed, 1 insertions(+), 23 deletions(-) diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -244,29 +244,7 @@ Programs may name their own exceptions by creating a new exception class (see :ref:`tut-classes` for more about Python classes). Exceptions should typically -be derived from the :exc:`Exception` class, either directly or indirectly. For -example:: - - >>> class MyError(Exception): - ... def __init__(self, value): - ... self.value = value - ... def __str__(self): - ... return repr(self.value) - ... - >>> try: - ... raise MyError(2*2) - ... except MyError as e: - ... print('My exception occurred, value:', e.value) - ... - My exception occurred, value: 4 - >>> raise MyError('oops!') - Traceback (most recent call last): - File "", line 1, in ? - __main__.MyError: 'oops!' - -In this example, the default :meth:`__init__` of :class:`Exception` has been -overridden. The new behavior simply creates the *value* attribute. This -replaces the default behavior of creating the *args* attribute. +be derived from the :exc:`Exception` class, either directly or indirectly. Exception classes can be defined which do anything any other class can do, but are usually kept simple, often only offering a number of attributes that allow -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 12 13:53:59 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 12 Aug 2016 17:53:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325805=3A_Skip_a_t?= =?utf-8?b?ZXN0IGZvciB0ZXN0X3BrZ3V0aWwgd2hlbiBfX25hbWVfXyA9PSBfX21haW5f?= =?utf-8?b?Xy4=?= Message-ID: <20160812175359.79049.13018.4D1BD8F0@psf.io> https://hg.python.org/cpython/rev/863258dcb745 changeset: 102623:863258dcb745 user: Brett Cannon date: Fri Aug 12 10:53:53 2016 -0700 summary: Issue #25805: Skip a test for test_pkgutil when __name__ == __main__. Thanks to SilentGhost for the patch. files: Lib/test/test_pkgutil.py | 1 + Misc/NEWS | 3 +++ 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_pkgutil.py b/Lib/test/test_pkgutil.py --- a/Lib/test/test_pkgutil.py +++ b/Lib/test/test_pkgutil.py @@ -413,6 +413,7 @@ self.assertIsNotNone(pkgutil.get_loader("test.support")) self.assertEqual(len(w.warnings), 0) + @unittest.skipIf(__name__ == '__main__', 'not compatible with __main__') def test_get_loader_handles_missing_loader_attribute(self): global __loader__ this_loader = __loader__ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -96,6 +96,9 @@ Tests ----- +- Issue #25805: Skip a test in test_pkgutil as needed that doesn't work when + ``__name__ == __main__``. Patch by SilentGhost. + - Issue #27472: Add test.support.unix_shell as the path to the default shell. - Issue #27369: In test_pyexpat, avoid testing an error message detail that -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 12 13:57:29 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 12 Aug 2016 17:57:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_for_issue_=2327712?= Message-ID: <20160812175729.8211.96156.833B99F1@psf.io> https://hg.python.org/cpython/rev/42a461366d6c changeset: 102625:42a461366d6c parent: 102623:863258dcb745 parent: 102624:927c29e1d646 user: Brett Cannon date: Fri Aug 12 10:57:17 2016 -0700 summary: Merge for issue #27712 files: Doc/reference/import.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/reference/import.rst b/Doc/reference/import.rst --- a/Doc/reference/import.rst +++ b/Doc/reference/import.rst @@ -785,7 +785,7 @@ path entry to be searched. This callable may either return a :term:`path entry finder` that can handle the path entry, or it may raise :exc:`ImportError`. An :exc:`ImportError` is used by the path based finder to -signal that the hook cannot find a :term:`path entry finder`. +signal that the hook cannot find a :term:`path entry finder` for that :term:`path entry`. The exception is ignored and :term:`import path` iteration continues. The hook should expect either a string or bytes object; the encoding of bytes objects @@ -839,7 +839,7 @@ Older path entry finders may implement one of these two deprecated methods instead of ``find_spec()``. The methods are still respected for the - sake of backward compatibility. Howevever, if ``find_spec()`` is + sake of backward compatibility. However, if ``find_spec()`` is implemented on the path entry finder, the legacy methods are ignored. :meth:`~importlib.abc.PathEntryFinder.find_loader` takes one argument, the -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 12 13:57:29 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 12 Aug 2016 17:57:29 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NzEy?= =?utf-8?q?=3A_Fix_some_typos_in_the_import_docs=2E?= Message-ID: <20160812175729.68529.57064.F31F4DCC@psf.io> https://hg.python.org/cpython/rev/927c29e1d646 changeset: 102624:927c29e1d646 branch: 3.5 parent: 102621:6fdd29a9d5d4 user: Brett Cannon date: Fri Aug 12 10:56:48 2016 -0700 summary: Issue #27712: Fix some typos in the import docs. Thanks to Xiang Zhang for the patch. files: Doc/reference/import.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/reference/import.rst b/Doc/reference/import.rst --- a/Doc/reference/import.rst +++ b/Doc/reference/import.rst @@ -774,7 +774,7 @@ path entry to be searched. This callable may either return a :term:`path entry finder` that can handle the path entry, or it may raise :exc:`ImportError`. An :exc:`ImportError` is used by the path based finder to -signal that the hook cannot find a :term:`path entry finder`. +signal that the hook cannot find a :term:`path entry finder` for that :term:`path entry`. The exception is ignored and :term:`import path` iteration continues. The hook should expect either a string or bytes object; the encoding of bytes objects @@ -828,7 +828,7 @@ Older path entry finders may implement one of these two deprecated methods instead of ``find_spec()``. The methods are still respected for the - sake of backward compatibility. Howevever, if ``find_spec()`` is + sake of backward compatibility. However, if ``find_spec()`` is implemented on the path entry finder, the legacy methods are ignored. :meth:`~importlib.abc.PathEntryFinder.find_loader` takes one argument, the -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 12 19:08:17 2016 From: python-checkins at python.org (alexander.belopolsky) Date: Fri, 12 Aug 2016 23:08:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2324773=3A_Skip_sys?= =?utf-8?q?tem_tests_for_transitions_in_year_2037_and_later=2E?= Message-ID: <20160812230817.62986.78713.DE4D5DB8@psf.io> https://hg.python.org/cpython/rev/71a7db7ceabc changeset: 102626:71a7db7ceabc user: Alexander Belopolsky date: Fri Aug 12 19:08:15 2016 -0400 summary: Issue #24773: Skip system tests for transitions in year 2037 and later. files: Lib/test/datetimetester.py | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -4766,9 +4766,11 @@ try: _time.tzset() for udt, shift in tz.transitions(): - if (self.zonename == 'Europe/Tallinn' and udt.date() == date(1999, 10, 31) or - self.zonename.endswith(('Casablanca', 'El_Aaiun')) and - udt.date() == date(2037, 10, 4)): + if udt.year >= 2037: + # System support for times around the end of 32-bit time_t + # and later is flaky on many systems. + break + if self.zonename == 'Europe/Tallinn' and udt.date() == date(1999, 10, 31): print("Skip %s %s transition" % (self.zonename, udt)) continue s0 = (udt - datetime(1970, 1, 1)) // SEC -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 12 22:38:15 2016 From: python-checkins at python.org (berker.peksag) Date: Sat, 13 Aug 2016 02:38:15 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0NjM3?= =?utf-8?q?=3A_Document_that_locals_can_be_any_mapping_object?= Message-ID: <20160813023814.13758.13915.360A765C@psf.io> https://hg.python.org/cpython/rev/0f2a09950ac8 changeset: 102627:0f2a09950ac8 branch: 3.5 parent: 102624:927c29e1d646 user: Berker Peksag date: Sat Aug 13 05:37:49 2016 +0300 summary: Issue #24637: Document that locals can be any mapping object This behavior is already tested in test_general_eval in Lib/test/test_builtin.py Patch by Matthew Keeter. files: Doc/c-api/veryhigh.rst | 17 +++++++++-------- 1 files changed, 9 insertions(+), 8 deletions(-) diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -219,9 +219,10 @@ .. c:function:: PyObject* PyRun_StringFlags(const char *str, int start, PyObject *globals, PyObject *locals, PyCompilerFlags *flags) Execute Python source code from *str* in the context specified by the - dictionaries *globals* and *locals* with the compiler flags specified by - *flags*. The parameter *start* specifies the start token that should be used to - parse the source code. + objects *globals* and *locals* with the compiler flags specified by + *flags*. *globals* must be a dictionary; *locals* can be any object + that implements the mapping protocol. The parameter *start* specifies + the start token that should be used to parse the source code. Returns the result of executing the code as a Python object, or *NULL* if an exception was raised. @@ -295,16 +296,16 @@ .. c:function:: PyObject* PyEval_EvalCode(PyObject *co, PyObject *globals, PyObject *locals) This is a simplified interface to :c:func:`PyEval_EvalCodeEx`, with just - the code object, and the dictionaries of global and local variables. - The other arguments are set to *NULL*. + the code object, and global and local variables. The other arguments are + set to *NULL*. .. c:function:: PyObject* PyEval_EvalCodeEx(PyObject *co, PyObject *globals, PyObject *locals, PyObject **args, int argcount, PyObject **kws, int kwcount, PyObject **defs, int defcount, PyObject *closure) Evaluate a precompiled code object, given a particular environment for its - evaluation. This environment consists of dictionaries of global and local - variables, arrays of arguments, keywords and defaults, and a closure tuple of - cells. + evaluation. This environment consists of a dictionary of global variables, + a mapping object of local variables, arrays of arguments, keywords and + defaults, and a closure tuple of cells. .. c:type:: PyFrameObject -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 12 22:38:15 2016 From: python-checkins at python.org (berker.peksag) Date: Sat, 13 Aug 2016 02:38:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2324637=3A_Merge_from_3=2E5?= Message-ID: <20160813023814.401.36635.EDE1DE38@psf.io> https://hg.python.org/cpython/rev/61c4dbec2e2c changeset: 102628:61c4dbec2e2c parent: 102626:71a7db7ceabc parent: 102627:0f2a09950ac8 user: Berker Peksag date: Sat Aug 13 05:38:18 2016 +0300 summary: Issue #24637: Merge from 3.5 files: Doc/c-api/veryhigh.rst | 17 +++++++++-------- 1 files changed, 9 insertions(+), 8 deletions(-) diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -219,9 +219,10 @@ .. c:function:: PyObject* PyRun_StringFlags(const char *str, int start, PyObject *globals, PyObject *locals, PyCompilerFlags *flags) Execute Python source code from *str* in the context specified by the - dictionaries *globals* and *locals* with the compiler flags specified by - *flags*. The parameter *start* specifies the start token that should be used to - parse the source code. + objects *globals* and *locals* with the compiler flags specified by + *flags*. *globals* must be a dictionary; *locals* can be any object + that implements the mapping protocol. The parameter *start* specifies + the start token that should be used to parse the source code. Returns the result of executing the code as a Python object, or *NULL* if an exception was raised. @@ -295,16 +296,16 @@ .. c:function:: PyObject* PyEval_EvalCode(PyObject *co, PyObject *globals, PyObject *locals) This is a simplified interface to :c:func:`PyEval_EvalCodeEx`, with just - the code object, and the dictionaries of global and local variables. - The other arguments are set to *NULL*. + the code object, and global and local variables. The other arguments are + set to *NULL*. .. c:function:: PyObject* PyEval_EvalCodeEx(PyObject *co, PyObject *globals, PyObject *locals, PyObject **args, int argcount, PyObject **kws, int kwcount, PyObject **defs, int defcount, PyObject *closure) Evaluate a precompiled code object, given a particular environment for its - evaluation. This environment consists of dictionaries of global and local - variables, arrays of arguments, keywords and defaults, and a closure tuple of - cells. + evaluation. This environment consists of a dictionary of global variables, + a mapping object of local variables, arrays of arguments, keywords and + defaults, and a closure tuple of cells. .. c:type:: PyFrameObject -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 14:10:27 2016 From: python-checkins at python.org (raymond.hettinger) Date: Sat, 13 Aug 2016 18:10:27 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3NzIw?= =?utf-8?q?=3A_Fix_error_in_eng=5Fto=5Fdecimal_docs_and_add_examples_from_?= =?utf-8?q?the?= Message-ID: <20160813181027.79414.69783.2458998A@psf.io> https://hg.python.org/cpython/rev/825713d3786a changeset: 102629:825713d3786a branch: 2.7 parent: 102611:0de509a79181 user: Raymond Hettinger date: Sat Aug 13 11:10:23 2016 -0700 summary: Issue #27720: Fix error in eng_to_decimal docs and add examples from the specification. files: Doc/library/decimal.rst | 18 ++++++++++----- Lib/decimal.py | 33 ++++++++++++++++++++++------ 2 files changed, 38 insertions(+), 13 deletions(-) diff --git a/Doc/library/decimal.rst b/Doc/library/decimal.rst --- a/Doc/library/decimal.rst +++ b/Doc/library/decimal.rst @@ -883,11 +883,13 @@ .. method:: to_eng_string([context]) - Convert to an engineering-type string. - - Engineering notation has an exponent which is a multiple of 3, so there - are up to 3 digits left of the decimal place. For example, converts - ``Decimal('123E+1')`` to ``Decimal('1.23E+3')``. + Convert to a string, using engineering notation if an exponent is needed. + + Engineering notation has an exponent which is a multiple of 3. This + can leave up to 3 digits to the left of the decimal place and may + require the addition of either one or two trailing zeros. + + For example, this converts ``Decimal('123E+1')`` to ``Decimal('1.23E+3')``. .. method:: to_integral([rounding[, context]]) @@ -1453,7 +1455,11 @@ .. method:: to_eng_string(x) - Converts a number to a string, using scientific notation. + Convert to a string, using engineering notation if an exponent is needed. + + Engineering notation has an exponent which is a multiple of 3. This + can leave up to 3 digits to the left of the decimal place and may + require the addition of either one or two trailing zeros. .. method:: to_integral_exact(x) diff --git a/Lib/decimal.py b/Lib/decimal.py --- a/Lib/decimal.py +++ b/Lib/decimal.py @@ -1048,12 +1048,11 @@ return sign + intpart + fracpart + exp def to_eng_string(self, context=None): - """Convert to engineering-type string. - - Engineering notation has an exponent which is a multiple of 3, so there - are up to 3 digits left of the decimal place. - - Same rules for when in exponential and when as a value as in __str__. + """Convert to a string, using engineering notation if an exponent is needed. + + Engineering notation has an exponent which is a multiple of 3. This + can leave up to 3 digits to the left of the decimal place and may + require the addition of either one or two trailing zeros. """ return self.__str__(eng=True, context=context) @@ -5339,9 +5338,29 @@ return r def to_eng_string(self, a): - """Converts a number to a string, using scientific notation. + """Convert to a string, using engineering notation if an exponent is needed. + + Engineering notation has an exponent which is a multiple of 3. This + can leave up to 3 digits to the left of the decimal place and may + require the addition of either one or two trailing zeros. The operation is not affected by the context. + + >>> ExtendedContext.to_eng_string(Decimal('123E+1')) + '1.23E+3' + >>> ExtendedContext.to_eng_string(Decimal('123E+3')) + '123E+3' + >>> ExtendedContext.to_eng_string(Decimal('123E-10')) + '12.3E-9' + >>> ExtendedContext.to_eng_string(Decimal('-123E-12')) + '-123E-12' + >>> ExtendedContext.to_eng_string(Decimal('7E-7')) + '700E-9' + >>> ExtendedContext.to_eng_string(Decimal('7E+1')) + '70' + >>> ExtendedContext.to_eng_string(Decimal('0E+1')) + '0.00E+3' + """ a = _convert_other(a, raiseit=True) return a.to_eng_string(context=self) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 14:16:06 2016 From: python-checkins at python.org (raymond.hettinger) Date: Sat, 13 Aug 2016 18:16:06 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NzIw?= =?utf-8?q?=3A_Fix_error_in_eng=5Fto=5Fdecimal_docs_and_add_examples_from_?= =?utf-8?q?the?= Message-ID: <20160813181606.68825.65454.9C62BFB1@psf.io> https://hg.python.org/cpython/rev/de2a9c98948e changeset: 102630:de2a9c98948e branch: 3.5 parent: 102627:0f2a09950ac8 user: Raymond Hettinger date: Sat Aug 13 11:15:34 2016 -0700 summary: Issue #27720: Fix error in eng_to_decimal docs and add examples from the specification. (Based on a first draft patch from Evelyn Mitchell.) files: Doc/library/decimal.rst | 18 ++++++++++----- Lib/_pydecimal.py | 33 ++++++++++++++++++++++------ 2 files changed, 38 insertions(+), 13 deletions(-) diff --git a/Doc/library/decimal.rst b/Doc/library/decimal.rst --- a/Doc/library/decimal.rst +++ b/Doc/library/decimal.rst @@ -836,11 +836,13 @@ .. method:: to_eng_string(context=None) - Convert to an engineering-type string. - - Engineering notation has an exponent which is a multiple of 3, so there - are up to 3 digits left of the decimal place. For example, converts - ``Decimal('123E+1')`` to ``Decimal('1.23E+3')``. + Convert to a string, using engineering notation if an exponent is needed. + + Engineering notation has an exponent which is a multiple of 3. This + can leave up to 3 digits to the left of the decimal place and may + require the addition of either one or two trailing zeros. + + For example, this converts ``Decimal('123E+1')`` to ``Decimal('1.23E+3')``. .. method:: to_integral(rounding=None, context=None) @@ -1410,7 +1412,11 @@ .. method:: to_eng_string(x) - Converts a number to a string, using scientific notation. + Convert to a string, using engineering notation if an exponent is needed. + + Engineering notation has an exponent which is a multiple of 3. This + can leave up to 3 digits to the left of the decimal place and may + require the addition of either one or two trailing zeros. .. method:: to_integral_exact(x) diff --git a/Lib/_pydecimal.py b/Lib/_pydecimal.py --- a/Lib/_pydecimal.py +++ b/Lib/_pydecimal.py @@ -1068,12 +1068,11 @@ return sign + intpart + fracpart + exp def to_eng_string(self, context=None): - """Convert to engineering-type string. - - Engineering notation has an exponent which is a multiple of 3, so there - are up to 3 digits left of the decimal place. - - Same rules for when in exponential and when as a value as in __str__. + """Convert to a string, using engineering notation if an exponent is needed. + + Engineering notation has an exponent which is a multiple of 3. This + can leave up to 3 digits to the left of the decimal place and may + require the addition of either one or two trailing zeros. """ return self.__str__(eng=True, context=context) @@ -5502,9 +5501,29 @@ return r def to_eng_string(self, a): - """Converts a number to a string, using scientific notation. + """Convert to a string, using engineering notation if an exponent is needed. + + Engineering notation has an exponent which is a multiple of 3. This + can leave up to 3 digits to the left of the decimal place and may + require the addition of either one or two trailing zeros. The operation is not affected by the context. + + >>> ExtendedContext.to_eng_string(Decimal('123E+1')) + '1.23E+3' + >>> ExtendedContext.to_eng_string(Decimal('123E+3')) + '123E+3' + >>> ExtendedContext.to_eng_string(Decimal('123E-10')) + '12.3E-9' + >>> ExtendedContext.to_eng_string(Decimal('-123E-12')) + '-123E-12' + >>> ExtendedContext.to_eng_string(Decimal('7E-7')) + '700E-9' + >>> ExtendedContext.to_eng_string(Decimal('7E+1')) + '70' + >>> ExtendedContext.to_eng_string(Decimal('0E+1')) + '0.00E+3' + """ a = _convert_other(a, raiseit=True) return a.to_eng_string(context=self) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 14:16:06 2016 From: python-checkins at python.org (raymond.hettinger) Date: Sat, 13 Aug 2016 18:16:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge?= Message-ID: <20160813181606.79168.75079.C954148A@psf.io> https://hg.python.org/cpython/rev/1b3a987d6df0 changeset: 102631:1b3a987d6df0 parent: 102628:61c4dbec2e2c parent: 102630:de2a9c98948e user: Raymond Hettinger date: Sat Aug 13 11:15:59 2016 -0700 summary: merge files: Doc/library/decimal.rst | 18 ++++++++++----- Lib/_pydecimal.py | 33 ++++++++++++++++++++++------ 2 files changed, 38 insertions(+), 13 deletions(-) diff --git a/Doc/library/decimal.rst b/Doc/library/decimal.rst --- a/Doc/library/decimal.rst +++ b/Doc/library/decimal.rst @@ -849,11 +849,13 @@ .. method:: to_eng_string(context=None) - Convert to an engineering-type string. - - Engineering notation has an exponent which is a multiple of 3, so there - are up to 3 digits left of the decimal place. For example, converts - ``Decimal('123E+1')`` to ``Decimal('1.23E+3')``. + Convert to a string, using engineering notation if an exponent is needed. + + Engineering notation has an exponent which is a multiple of 3. This + can leave up to 3 digits to the left of the decimal place and may + require the addition of either one or two trailing zeros. + + For example, this converts ``Decimal('123E+1')`` to ``Decimal('1.23E+3')``. .. method:: to_integral(rounding=None, context=None) @@ -1423,7 +1425,11 @@ .. method:: to_eng_string(x) - Converts a number to a string, using scientific notation. + Convert to a string, using engineering notation if an exponent is needed. + + Engineering notation has an exponent which is a multiple of 3. This + can leave up to 3 digits to the left of the decimal place and may + require the addition of either one or two trailing zeros. .. method:: to_integral_exact(x) diff --git a/Lib/_pydecimal.py b/Lib/_pydecimal.py --- a/Lib/_pydecimal.py +++ b/Lib/_pydecimal.py @@ -1118,12 +1118,11 @@ return sign + intpart + fracpart + exp def to_eng_string(self, context=None): - """Convert to engineering-type string. - - Engineering notation has an exponent which is a multiple of 3, so there - are up to 3 digits left of the decimal place. - - Same rules for when in exponential and when as a value as in __str__. + """Convert to a string, using engineering notation if an exponent is needed. + + Engineering notation has an exponent which is a multiple of 3. This + can leave up to 3 digits to the left of the decimal place and may + require the addition of either one or two trailing zeros. """ return self.__str__(eng=True, context=context) @@ -5552,9 +5551,29 @@ return r def to_eng_string(self, a): - """Converts a number to a string, using scientific notation. + """Convert to a string, using engineering notation if an exponent is needed. + + Engineering notation has an exponent which is a multiple of 3. This + can leave up to 3 digits to the left of the decimal place and may + require the addition of either one or two trailing zeros. The operation is not affected by the context. + + >>> ExtendedContext.to_eng_string(Decimal('123E+1')) + '1.23E+3' + >>> ExtendedContext.to_eng_string(Decimal('123E+3')) + '123E+3' + >>> ExtendedContext.to_eng_string(Decimal('123E-10')) + '12.3E-9' + >>> ExtendedContext.to_eng_string(Decimal('-123E-12')) + '-123E-12' + >>> ExtendedContext.to_eng_string(Decimal('7E-7')) + '700E-9' + >>> ExtendedContext.to_eng_string(Decimal('7E+1')) + '70' + >>> ExtendedContext.to_eng_string(Decimal('0E+1')) + '0.00E+3' + """ a = _convert_other(a, raiseit=True) return a.to_eng_string(context=self) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 14:48:40 2016 From: python-checkins at python.org (r.david.murray) Date: Sat, 13 Aug 2016 18:48:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzI3NzUzOiBmaXgg?= =?utf-8?q?typo_=28name-=3Enamed=29=2E?= Message-ID: <20160813184840.13453.18212.4491AAAF@psf.io> https://hg.python.org/cpython/rev/b1e4c8a3e786 changeset: 102632:b1e4c8a3e786 branch: 2.7 parent: 102629:825713d3786a user: R David Murray date: Sat Aug 13 14:46:23 2016 -0400 summary: #27753: fix typo (name->named). files: Doc/library/py_compile.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/py_compile.rst b/Doc/library/py_compile.rst --- a/Doc/library/py_compile.rst +++ b/Doc/library/py_compile.rst @@ -29,7 +29,7 @@ .. function:: compile(file[, cfile[, dfile[, doraise]]]) Compile a source file to byte-code and write out the byte-code cache file. The - source code is loaded from the file name *file*. The byte-code is written to + source code is loaded from the file named *file*. The byte-code is written to *cfile*, which defaults to *file* ``+`` ``'c'`` (``'o'`` if optimization is enabled in the current interpreter). If *dfile* is specified, it is used as the name of the source file in error messages instead of *file*. If *doraise* is -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 14:48:40 2016 From: python-checkins at python.org (r.david.murray) Date: Sat, 13 Aug 2016 18:48:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2U6ICMyNzc1MzogZml4IHR5cG8gKG5hbWUtPm5hbWVkKS4=?= Message-ID: <20160813184840.79216.18936.DD5BD92C@psf.io> https://hg.python.org/cpython/rev/dda25c37d02b changeset: 102634:dda25c37d02b parent: 102631:1b3a987d6df0 parent: 102633:c1cc1f616285 user: R David Murray date: Sat Aug 13 14:47:54 2016 -0400 summary: Merge: #27753: fix typo (name->named). files: Doc/library/py_compile.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/py_compile.rst b/Doc/library/py_compile.rst --- a/Doc/library/py_compile.rst +++ b/Doc/library/py_compile.rst @@ -30,7 +30,7 @@ .. function:: compile(file, cfile=None, dfile=None, doraise=False, optimize=-1) Compile a source file to byte-code and write out the byte-code cache file. - The source code is loaded from the file name *file*. The byte-code is + The source code is loaded from the file named *file*. The byte-code is written to *cfile*, which defaults to the :pep:`3147`/:pep:`488` path, ending in ``.pyc``. For example, if *file* is ``/foo/bar/baz.py`` *cfile* will default to -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 14:48:40 2016 From: python-checkins at python.org (r.david.murray) Date: Sat, 13 Aug 2016 18:48:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzI3NzUzOiBmaXgg?= =?utf-8?q?typo_=28name-=3Enamed=29=2E?= Message-ID: <20160813184840.68534.31497.9CCC57FF@psf.io> https://hg.python.org/cpython/rev/c1cc1f616285 changeset: 102633:c1cc1f616285 branch: 3.5 parent: 102630:de2a9c98948e user: R David Murray date: Sat Aug 13 14:47:18 2016 -0400 summary: #27753: fix typo (name->named). files: Doc/library/py_compile.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/py_compile.rst b/Doc/library/py_compile.rst --- a/Doc/library/py_compile.rst +++ b/Doc/library/py_compile.rst @@ -30,7 +30,7 @@ .. function:: compile(file, cfile=None, dfile=None, doraise=False, optimize=-1) Compile a source file to byte-code and write out the byte-code cache file. - The source code is loaded from the file name *file*. The byte-code is + The source code is loaded from the file named *file*. The byte-code is written to *cfile*, which defaults to the :pep:`3147`/:pep:`488` path, ending in ``.pyc``. For example, if *file* is ``/foo/bar/baz.py`` *cfile* will default to -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 19:48:38 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 13 Aug 2016 23:48:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_fix_possible_o?= =?utf-8?q?verflow_in_encode=5Fbasestring=5Fascii_=28=2323369=29?= Message-ID: <20160813234838.67748.5533.4D872A1C@psf.io> https://hg.python.org/cpython/rev/6fa0ebfdc136 changeset: 102635:6fa0ebfdc136 branch: 2.7 parent: 102632:b1e4c8a3e786 user: Benjamin Peterson date: Sat Aug 13 16:47:25 2016 -0700 summary: fix possible overflow in encode_basestring_ascii (#23369) files: Misc/NEWS | 3 +++ Modules/_json.c | 4 ++++ 2 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,9 @@ Library ------- +- Issue #23369: Fixed possible integer overflow in + _json.encode_basestring_ascii. + - Issue #27568: Prevent HTTPoxy attack (CVE-2016-1000110). Ignore the HTTP_PROXY variable when REQUEST_METHOD environment is set, which indicates that the script is in CGI mode. diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -211,6 +211,10 @@ input_unicode = PyUnicode_AS_UNICODE(pystr); /* One char input can be up to 6 chars output, estimate 4 of these */ + if (input_chars > (PY_SSIZE_T_MAX - 2)/ MAX_EXPANSION) { + PyErr_SetString(PyExc_OverflowError, "string is too long to escape"); + return NULL; + } output_size = 2 + (MIN_EXPANSION * 4) + input_chars; max_output_size = 2 + (input_chars * MAX_EXPANSION); rval = PyString_FromStringAndSize(NULL, output_size); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 20:22:37 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 14 Aug 2016 00:22:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_check_for_over?= =?utf-8?q?flow_in_join=5Fappend=5Fdata_=28closes_=2327758=29?= Message-ID: <20160814002237.22085.7944.11A2C552@psf.io> https://hg.python.org/cpython/rev/afa356402217 changeset: 102637:afa356402217 branch: 3.3 parent: 102544:4d33bccb59a8 user: Benjamin Peterson date: Sat Aug 13 17:17:06 2016 -0700 summary: check for overflow in join_append_data (closes #27758) Reported by Thomas E. Hybel files: Misc/NEWS | 3 +++ Modules/_csv.c | 23 +++++++++++++++++++---- 2 files changed, 22 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,9 @@ Library ------- +- Issue #27758: Fix possible integer overflow in the _csv module for large record + lengths. + - Issue #27568: Prevent HTTPoxy attack (CVE-2016-1000110). Ignore the HTTP_PROXY variable when REQUEST_METHOD environment is set, which indicates that the script is in CGI mode. diff --git a/Modules/_csv.c b/Modules/_csv.c --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -1002,11 +1002,19 @@ int i; Py_ssize_t rec_len; -#define ADDCH(c) \ +#define INCLEN \ + do {\ + if (!copy_phase && rec_len == PY_SSIZE_T_MAX) { \ + goto overflow; \ + } \ + rec_len++; \ + } while(0) + +#define ADDCH(c) \ do {\ if (copy_phase) \ self->rec[rec_len] = c;\ - rec_len++;\ + INCLEN;\ } while(0) rec_len = self->rec_len; @@ -1072,11 +1080,18 @@ if (*quoted) { if (copy_phase) ADDCH(dialect->quotechar); - else - rec_len += 2; + else { + INCLEN; /* starting quote */ + INCLEN; /* ending quote */ + } } return rec_len; + + overflow: + PyErr_NoMemory(); + return -1; #undef ADDCH +#undef INCLEN } static int -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 20:22:37 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 14 Aug 2016 00:22:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_check_for_over?= =?utf-8?q?flow_in_join=5Fappend=5Fdata_=28closes_=2327758=29?= Message-ID: <20160814002237.67748.83326.29231E4D@psf.io> https://hg.python.org/cpython/rev/fdae903db33a changeset: 102636:fdae903db33a branch: 2.7 user: Benjamin Peterson date: Sat Aug 13 17:17:06 2016 -0700 summary: check for overflow in join_append_data (closes #27758) Reported by Thomas E. Hybel files: Misc/NEWS | 3 +++ Modules/_csv.c | 23 +++++++++++++++++++---- 2 files changed, 22 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,9 @@ Library ------- +- Issue #27758: Fix possible integer overflow in the _csv module for large record + lengths. + - Issue #23369: Fixed possible integer overflow in _json.encode_basestring_ascii. diff --git a/Modules/_csv.c b/Modules/_csv.c --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -985,11 +985,19 @@ int i, rec_len; char *lineterm; -#define ADDCH(c) \ +#define INCLEN \ + do {\ + if (!copy_phase && rec_len == INT_MAX) { \ + goto overflow; \ + } \ + rec_len++; \ + } while(0) + +#define ADDCH(c) \ do {\ if (copy_phase) \ self->rec[rec_len] = c;\ - rec_len++;\ + INCLEN;\ } while(0) lineterm = PyString_AsString(dialect->lineterminator); @@ -1059,11 +1067,18 @@ if (*quoted) { if (copy_phase) ADDCH(dialect->quotechar); - else - rec_len += 2; + else { + INCLEN; /* starting quote */ + INCLEN; /* ending quote */ + } } return rec_len; + + overflow: + PyErr_NoMemory(); + return -1; #undef ADDCH +#undef INCLEN } static int -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 20:22:37 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 14 Aug 2016 00:22:37 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_merge_3=2E3_=28=2327758=29?= Message-ID: <20160814002237.13453.91654.69C427F2@psf.io> https://hg.python.org/cpython/rev/10b89df93c58 changeset: 102638:10b89df93c58 branch: 3.4 parent: 102545:190ebf99bf45 parent: 102637:afa356402217 user: Benjamin Peterson date: Sat Aug 13 17:21:22 2016 -0700 summary: merge 3.3 (#27758) files: Misc/NEWS | 3 +++ Modules/_csv.c | 23 +++++++++++++++++++---- 2 files changed, 22 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,6 +13,9 @@ Library ------- +- Issue #27758: Fix possible integer overflow in the _csv module for large record + lengths. + - Issue #27568: Prevent HTTPoxy attack (CVE-2016-1000110). Ignore the HTTP_PROXY variable when REQUEST_METHOD environment is set, which indicates that the script is in CGI mode. diff --git a/Modules/_csv.c b/Modules/_csv.c --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -1016,11 +1016,19 @@ int i; Py_ssize_t rec_len; -#define ADDCH(c) \ +#define INCLEN \ + do {\ + if (!copy_phase && rec_len == PY_SSIZE_T_MAX) { \ + goto overflow; \ + } \ + rec_len++; \ + } while(0) + +#define ADDCH(c) \ do {\ if (copy_phase) \ self->rec[rec_len] = c;\ - rec_len++;\ + INCLEN;\ } while(0) rec_len = self->rec_len; @@ -1086,11 +1094,18 @@ if (*quoted) { if (copy_phase) ADDCH(dialect->quotechar); - else - rec_len += 2; + else { + INCLEN; /* starting quote */ + INCLEN; /* ending quote */ + } } return rec_len; + + overflow: + PyErr_NoMemory(); + return -1; #undef ADDCH +#undef INCLEN } static int -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 20:22:37 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 14 Aug 2016 00:22:37 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4_=28closes_=2327758=29?= Message-ID: <20160814002237.13924.45564.FACF3791@psf.io> https://hg.python.org/cpython/rev/55e8d3e542bd changeset: 102639:55e8d3e542bd branch: 3.5 parent: 102633:c1cc1f616285 parent: 102638:10b89df93c58 user: Benjamin Peterson date: Sat Aug 13 17:21:54 2016 -0700 summary: merge 3.4 (closes #27758) files: Misc/NEWS | 3 +++ Modules/_csv.c | 23 +++++++++++++++++++---- 2 files changed, 22 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,9 @@ - Issue #26750: unittest.mock.create_autospec() now works properly for subclasses of property() and other data descriptors. +- Issue #27758: Fix possible integer overflow in the _csv module for large record + lengths. + - Issue #27568: Prevent HTTPoxy attack (CVE-2016-1000110). Ignore the HTTP_PROXY variable when REQUEST_METHOD environment is set, which indicates that the script is in CGI mode. diff --git a/Modules/_csv.c b/Modules/_csv.c --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -1014,11 +1014,19 @@ int i; Py_ssize_t rec_len; -#define ADDCH(c) \ +#define INCLEN \ + do {\ + if (!copy_phase && rec_len == PY_SSIZE_T_MAX) { \ + goto overflow; \ + } \ + rec_len++; \ + } while(0) + +#define ADDCH(c) \ do {\ if (copy_phase) \ self->rec[rec_len] = c;\ - rec_len++;\ + INCLEN;\ } while(0) rec_len = self->rec_len; @@ -1072,11 +1080,18 @@ if (*quoted) { if (copy_phase) ADDCH(dialect->quotechar); - else - rec_len += 2; + else { + INCLEN; /* starting quote */ + INCLEN; /* ending quote */ + } } return rec_len; + + overflow: + PyErr_NoMemory(); + return -1; #undef ADDCH +#undef INCLEN } static int -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 20:22:38 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 14 Aug 2016 00:22:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_3=2E5_=28closes_=2327758=29?= Message-ID: <20160814002238.67839.77130.91D0DB61@psf.io> https://hg.python.org/cpython/rev/609b554dd4a2 changeset: 102640:609b554dd4a2 parent: 102634:dda25c37d02b parent: 102639:55e8d3e542bd user: Benjamin Peterson date: Sat Aug 13 17:22:18 2016 -0700 summary: merge 3.5 (closes #27758) files: Misc/NEWS | 3 +++ Modules/_csv.c | 23 +++++++++++++++++++---- 2 files changed, 22 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -58,6 +58,9 @@ - Issue #27661: Added tzinfo keyword argument to datetime.combine. +- Issue #27758: Fix possible integer overflow in the _csv module for large record + lengths. + - Issue #27568: Prevent HTTPoxy attack (CVE-2016-1000110). Ignore the HTTP_PROXY variable when REQUEST_METHOD environment is set, which indicates that the script is in CGI mode. diff --git a/Modules/_csv.c b/Modules/_csv.c --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -1014,11 +1014,19 @@ int i; Py_ssize_t rec_len; -#define ADDCH(c) \ +#define INCLEN \ + do {\ + if (!copy_phase && rec_len == PY_SSIZE_T_MAX) { \ + goto overflow; \ + } \ + rec_len++; \ + } while(0) + +#define ADDCH(c) \ do {\ if (copy_phase) \ self->rec[rec_len] = c;\ - rec_len++;\ + INCLEN;\ } while(0) rec_len = self->rec_len; @@ -1072,11 +1080,18 @@ if (*quoted) { if (copy_phase) ADDCH(dialect->quotechar); - else - rec_len += 2; + else { + INCLEN; /* starting quote */ + INCLEN; /* ending quote */ + } } return rec_len; + + overflow: + PyErr_NoMemory(); + return -1; #undef ADDCH +#undef INCLEN } static int -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 21:21:42 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 14 Aug 2016 01:21:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_do_not_allow_r?= =?utf-8?q?eading_negative_values_with_getstr=28=29?= Message-ID: <20160814012142.67964.77848.5A4BFDE2@psf.io> https://hg.python.org/cpython/rev/1ab60d0686df changeset: 102641:1ab60d0686df branch: 2.7 parent: 102636:fdae903db33a user: Benjamin Peterson date: Sat Aug 13 18:15:28 2016 -0700 summary: do not allow reading negative values with getstr() files: Lib/test/test_curses.py | 3 +++ Misc/NEWS | 3 +++ Modules/_cursesmodule.c | 8 ++++++++ 3 files changed, 14 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py --- a/Lib/test/test_curses.py +++ b/Lib/test/test_curses.py @@ -185,6 +185,9 @@ if hasattr(curses, 'enclose'): stdscr.enclose() + self.assertRaises(ValueError, stdscr.getstr, -400) + self.assertRaises(ValueError, stdscr.getstr, 2, 3, -400) + def test_module_funcs(self): "Test module-level functions" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,9 @@ Library ------- +- In the curses module, raise an error if window.getstr() is passed a negative + value. + - Issue #27758: Fix possible integer overflow in the _csv module for large record lengths. diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -918,6 +918,10 @@ case 1: if (!PyArg_ParseTuple(args,"i;n", &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } Py_BEGIN_ALLOW_THREADS rtn2 = wgetnstr(self->win,rtn,MIN(n, 1023)); Py_END_ALLOW_THREADS @@ -936,6 +940,10 @@ case 3: if (!PyArg_ParseTuple(args,"iii;y,x,n", &y, &x, &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } #ifdef STRICT_SYSV_CURSES Py_BEGIN_ALLOW_THREADS rtn2 = wmove(self->win,y,x)==ERR ? ERR : -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 21:21:42 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 14 Aug 2016 01:21:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_do_not_allow_r?= =?utf-8?q?eading_negative_values_with_getstr=28=29?= Message-ID: <20160814012142.23539.48503.33B58693@psf.io> https://hg.python.org/cpython/rev/d5f6bc45b376 changeset: 102642:d5f6bc45b376 branch: 3.4 parent: 102638:10b89df93c58 user: Benjamin Peterson date: Sat Aug 13 18:15:28 2016 -0700 summary: do not allow reading negative values with getstr() files: Lib/test/test_curses.py | 3 +++ Misc/NEWS | 3 +++ Modules/_cursesmodule.c | 8 ++++++++ 3 files changed, 14 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py --- a/Lib/test/test_curses.py +++ b/Lib/test/test_curses.py @@ -163,6 +163,9 @@ if hasattr(curses, 'enclose'): stdscr.enclose() + self.assertRaises(ValueError, stdscr.getstr, -400) + self.assertRaises(ValueError, stdscr.getstr, 2, 3, -400) + def test_module_funcs(self): "Test module-level functions" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,6 +13,9 @@ Library ------- +- In the curses module, raise an error if window.getstr() is passed a negative + value. + - Issue #27758: Fix possible integer overflow in the _csv module for large record lengths. diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -1284,6 +1284,10 @@ case 1: if (!PyArg_ParseTuple(args,"i;n", &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } Py_BEGIN_ALLOW_THREADS rtn2 = wgetnstr(self->win, rtn, Py_MIN(n, 1023)); Py_END_ALLOW_THREADS @@ -1302,6 +1306,10 @@ case 3: if (!PyArg_ParseTuple(args,"iii;y,x,n", &y, &x, &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } #ifdef STRICT_SYSV_CURSES Py_BEGIN_ALLOW_THREADS rtn2 = wmove(self->win,y,x)==ERR ? ERR : -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 21:21:42 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 14 Aug 2016 01:21:42 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4?= Message-ID: <20160814012142.62945.91740.57D50FBE@psf.io> https://hg.python.org/cpython/rev/27557392ac51 changeset: 102643:27557392ac51 branch: 3.5 parent: 102639:55e8d3e542bd parent: 102642:d5f6bc45b376 user: Benjamin Peterson date: Sat Aug 13 18:21:16 2016 -0700 summary: merge 3.4 files: Lib/test/test_curses.py | 3 +++ Misc/NEWS | 3 +++ Modules/_cursesmodule.c | 8 ++++++++ 3 files changed, 14 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py --- a/Lib/test/test_curses.py +++ b/Lib/test/test_curses.py @@ -188,6 +188,9 @@ if hasattr(curses, 'enclose'): stdscr.enclose() + self.assertRaises(ValueError, stdscr.getstr, -400) + self.assertRaises(ValueError, stdscr.getstr, 2, 3, -400) + def test_module_funcs(self): "Test module-level functions" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,9 @@ - Issue #26750: unittest.mock.create_autospec() now works properly for subclasses of property() and other data descriptors. +- In the curses module, raise an error if window.getstr() is passed a negative + value. + - Issue #27758: Fix possible integer overflow in the _csv module for large record lengths. diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -1221,6 +1221,10 @@ case 1: if (!PyArg_ParseTuple(args,"i;n", &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } Py_BEGIN_ALLOW_THREADS rtn2 = wgetnstr(self->win, rtn, Py_MIN(n, 1023)); Py_END_ALLOW_THREADS @@ -1239,6 +1243,10 @@ case 3: if (!PyArg_ParseTuple(args,"iii;y,x,n", &y, &x, &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } #ifdef STRICT_SYSV_CURSES Py_BEGIN_ALLOW_THREADS rtn2 = wmove(self->win,y,x)==ERR ? ERR : -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 21:21:44 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 14 Aug 2016 01:21:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160814012143.67839.4996.8F1AFEB1@psf.io> https://hg.python.org/cpython/rev/e723d9052eb3 changeset: 102644:e723d9052eb3 parent: 102640:609b554dd4a2 parent: 102643:27557392ac51 user: Benjamin Peterson date: Sat Aug 13 18:21:32 2016 -0700 summary: merge 3.5 files: Lib/test/test_curses.py | 3 +++ Misc/NEWS | 3 +++ Modules/_cursesmodule.c | 8 ++++++++ 3 files changed, 14 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py --- a/Lib/test/test_curses.py +++ b/Lib/test/test_curses.py @@ -188,6 +188,9 @@ if hasattr(curses, 'enclose'): stdscr.enclose() + self.assertRaises(ValueError, stdscr.getstr, -400) + self.assertRaises(ValueError, stdscr.getstr, 2, 3, -400) + def test_module_funcs(self): "Test module-level functions" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -58,6 +58,9 @@ - Issue #27661: Added tzinfo keyword argument to datetime.combine. +- In the curses module, raise an error if window.getstr() is passed a negative + value. + - Issue #27758: Fix possible integer overflow in the _csv module for large record lengths. diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -1221,6 +1221,10 @@ case 1: if (!PyArg_ParseTuple(args,"i;n", &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } Py_BEGIN_ALLOW_THREADS rtn2 = wgetnstr(self->win, rtn, Py_MIN(n, 1023)); Py_END_ALLOW_THREADS @@ -1239,6 +1243,10 @@ case 3: if (!PyArg_ParseTuple(args,"iii;y,x,n", &y, &x, &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } #ifdef STRICT_SYSV_CURSES Py_BEGIN_ALLOW_THREADS rtn2 = wmove(self->win,y,x)==ERR ? ERR : -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 21:37:33 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 14 Aug 2016 01:37:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_fix_possible_i?= =?utf-8?q?nteger_overflow_in_binascii=2Eb2a=5Fqp_=28closes_=2327760=29?= Message-ID: <20160814013732.58290.89016.E792094E@psf.io> https://hg.python.org/cpython/rev/54c74212db91 changeset: 102646:54c74212db91 branch: 3.3 parent: 102637:afa356402217 user: Benjamin Peterson date: Sat Aug 13 18:33:33 2016 -0700 summary: fix possible integer overflow in binascii.b2a_qp (closes #27760) Reported by Thomas E. Hybel files: Misc/NEWS | 2 ++ Modules/binascii.c | 25 ++++++++++++++++--------- 2 files changed, 18 insertions(+), 9 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,8 @@ Library ------- +- Issue #27760: Fix possible integer overflow in binascii.b2a_qp. + - Issue #27758: Fix possible integer overflow in the _csv module for large record lengths. diff --git a/Modules/binascii.c b/Modules/binascii.c --- a/Modules/binascii.c +++ b/Modules/binascii.c @@ -1365,6 +1365,7 @@ /* First, scan to see how many characters need to be encoded */ in = 0; while (in < datalen) { + Py_ssize_t delta = 0; if ((data[in] > 126) || (data[in] == '=') || (header && data[in] == '_') || @@ -1379,12 +1380,12 @@ if ((linelen + 3) >= MAXLINESIZE) { linelen = 0; if (crlf) - odatalen += 3; + delta += 3; else - odatalen += 2; + delta += 2; } linelen += 3; - odatalen += 3; + delta += 3; in++; } else { @@ -1396,11 +1397,11 @@ linelen = 0; /* Protect against whitespace on end of line */ if (in && ((data[in-1] == ' ') || (data[in-1] == '\t'))) - odatalen += 2; + delta += 2; if (crlf) - odatalen += 2; + delta += 2; else - odatalen += 1; + delta += 1; if (data[in] == '\r') in += 2; else @@ -1412,15 +1413,21 @@ (linelen + 1) >= MAXLINESIZE) { linelen = 0; if (crlf) - odatalen += 3; + delta += 3; else - odatalen += 2; + delta += 2; } linelen++; - odatalen++; + delta++; in++; } } + if (PY_SSIZE_T_MAX - delta < odatalen) { + PyBuffer_Release(&pdata); + PyErr_NoMemory(); + return NULL; + } + odatalen += delta; } /* We allocate the output same size as input, this is overkill. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 21:37:32 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 14 Aug 2016 01:37:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_fix_possible_i?= =?utf-8?q?nteger_overflow_in_binascii=2Eb2a=5Fqp_=28closes_=2327760=29?= Message-ID: <20160814013732.19221.91272.56FDB509@psf.io> https://hg.python.org/cpython/rev/af42635b5ed1 changeset: 102645:af42635b5ed1 branch: 2.7 parent: 102641:1ab60d0686df user: Benjamin Peterson date: Sat Aug 13 18:33:33 2016 -0700 summary: fix possible integer overflow in binascii.b2a_qp (closes #27760) Reported by Thomas E. Hybel files: Misc/NEWS | 2 ++ Modules/binascii.c | 25 ++++++++++++++++--------- 2 files changed, 18 insertions(+), 9 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,8 @@ Library ------- +- Issue #27760: Fix possible integer overflow in binascii.b2a_qp. + - In the curses module, raise an error if window.getstr() is passed a negative value. diff --git a/Modules/binascii.c b/Modules/binascii.c --- a/Modules/binascii.c +++ b/Modules/binascii.c @@ -1316,6 +1316,7 @@ /* First, scan to see how many characters need to be encoded */ in = 0; while (in < datalen) { + Py_ssize_t delta = 0; if ((data[in] > 126) || (data[in] == '=') || (header && data[in] == '_') || @@ -1331,12 +1332,12 @@ if ((linelen + 3) >= MAXLINESIZE) { linelen = 0; if (crlf) - odatalen += 3; + delta += 3; else - odatalen += 2; + delta += 2; } linelen += 3; - odatalen += 3; + delta += 3; in++; } else { @@ -1348,11 +1349,11 @@ linelen = 0; /* Protect against whitespace on end of line */ if (in && ((data[in-1] == ' ') || (data[in-1] == '\t'))) - odatalen += 2; + delta += 2; if (crlf) - odatalen += 2; + delta += 2; else - odatalen += 1; + delta += 1; if (data[in] == '\r') in += 2; else @@ -1364,15 +1365,21 @@ (linelen + 1) >= MAXLINESIZE) { linelen = 0; if (crlf) - odatalen += 3; + delta += 3; else - odatalen += 2; + delta += 2; } linelen++; - odatalen++; + delta++; in++; } } + if (PY_SSIZE_T_MAX - delta < odatalen) { + PyBuffer_Release(&pdata); + PyErr_NoMemory(); + return NULL; + } + odatalen += delta; } /* We allocate the output same size as input, this is overkill. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 21:37:33 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 14 Aug 2016 01:37:33 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_merge_3=2E3_=28closes_=2327760=29?= Message-ID: <20160814013733.19373.23917.A624CCE1@psf.io> https://hg.python.org/cpython/rev/9822bf4bcece changeset: 102647:9822bf4bcece branch: 3.4 parent: 102642:d5f6bc45b376 parent: 102646:54c74212db91 user: Benjamin Peterson date: Sat Aug 13 18:36:55 2016 -0700 summary: merge 3.3 (closes #27760) files: Misc/NEWS | 2 ++ Modules/binascii.c | 24 +++++++++++++++--------- 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -16,6 +16,8 @@ - In the curses module, raise an error if window.getstr() is passed a negative value. +- Issue #27760: Fix possible integer overflow in binascii.b2a_qp. + - Issue #27758: Fix possible integer overflow in the _csv module for large record lengths. diff --git a/Modules/binascii.c b/Modules/binascii.c --- a/Modules/binascii.c +++ b/Modules/binascii.c @@ -1408,6 +1408,7 @@ /* First, scan to see how many characters need to be encoded */ in = 0; while (in < datalen) { + Py_ssize_t delta = 0; if ((databuf[in] > 126) || (databuf[in] == '=') || (header && databuf[in] == '_') || @@ -1422,12 +1423,12 @@ if ((linelen + 3) >= MAXLINESIZE) { linelen = 0; if (crlf) - odatalen += 3; + delta += 3; else - odatalen += 2; + delta += 2; } linelen += 3; - odatalen += 3; + delta += 3; in++; } else { @@ -1439,11 +1440,11 @@ linelen = 0; /* Protect against whitespace on end of line */ if (in && ((databuf[in-1] == ' ') || (databuf[in-1] == '\t'))) - odatalen += 2; + delta += 2; if (crlf) - odatalen += 2; + delta += 2; else - odatalen += 1; + delta += 1; if (databuf[in] == '\r') in += 2; else @@ -1455,15 +1456,20 @@ (linelen + 1) >= MAXLINESIZE) { linelen = 0; if (crlf) - odatalen += 3; + delta += 3; else - odatalen += 2; + delta += 2; } linelen++; - odatalen++; + delta++; in++; } } + if (PY_SSIZE_T_MAX - delta < odatalen) { + PyErr_NoMemory(); + return NULL; + } + odatalen += delta; } /* We allocate the output same size as input, this is overkill. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 21:37:33 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 14 Aug 2016 01:37:33 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4_=28closes_=2327760=29?= Message-ID: <20160814013733.79138.2327.E088BDC0@psf.io> https://hg.python.org/cpython/rev/a277ab6bf66b changeset: 102648:a277ab6bf66b branch: 3.5 parent: 102643:27557392ac51 parent: 102647:9822bf4bcece user: Benjamin Peterson date: Sat Aug 13 18:37:12 2016 -0700 summary: merge 3.4 (closes #27760) files: Misc/NEWS | 2 ++ Modules/binascii.c | 24 +++++++++++++++--------- 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -40,6 +40,8 @@ - In the curses module, raise an error if window.getstr() is passed a negative value. +- Issue #27760: Fix possible integer overflow in binascii.b2a_qp. + - Issue #27758: Fix possible integer overflow in the _csv module for large record lengths. diff --git a/Modules/binascii.c b/Modules/binascii.c --- a/Modules/binascii.c +++ b/Modules/binascii.c @@ -1383,6 +1383,7 @@ /* First, scan to see how many characters need to be encoded */ in = 0; while (in < datalen) { + Py_ssize_t delta = 0; if ((databuf[in] > 126) || (databuf[in] == '=') || (header && databuf[in] == '_') || @@ -1397,12 +1398,12 @@ if ((linelen + 3) >= MAXLINESIZE) { linelen = 0; if (crlf) - odatalen += 3; + delta += 3; else - odatalen += 2; + delta += 2; } linelen += 3; - odatalen += 3; + delta += 3; in++; } else { @@ -1414,11 +1415,11 @@ linelen = 0; /* Protect against whitespace on end of line */ if (in && ((databuf[in-1] == ' ') || (databuf[in-1] == '\t'))) - odatalen += 2; + delta += 2; if (crlf) - odatalen += 2; + delta += 2; else - odatalen += 1; + delta += 1; if (databuf[in] == '\r') in += 2; else @@ -1430,15 +1431,20 @@ (linelen + 1) >= MAXLINESIZE) { linelen = 0; if (crlf) - odatalen += 3; + delta += 3; else - odatalen += 2; + delta += 2; } linelen++; - odatalen++; + delta++; in++; } } + if (PY_SSIZE_T_MAX - delta < odatalen) { + PyErr_NoMemory(); + return NULL; + } + odatalen += delta; } /* We allocate the output same size as input, this is overkill. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 13 21:37:33 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 14 Aug 2016 01:37:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_3=2E5_=28closes_=2327760=29?= Message-ID: <20160814013733.781.41920.338A99E2@psf.io> https://hg.python.org/cpython/rev/4a00d4ebf60f changeset: 102649:4a00d4ebf60f parent: 102644:e723d9052eb3 parent: 102648:a277ab6bf66b user: Benjamin Peterson date: Sat Aug 13 18:37:20 2016 -0700 summary: merge 3.5 (closes #27760) files: Misc/NEWS | 2 ++ Modules/binascii.c | 24 +++++++++++++++--------- 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -61,6 +61,8 @@ - In the curses module, raise an error if window.getstr() is passed a negative value. +- Issue #27760: Fix possible integer overflow in binascii.b2a_qp. + - Issue #27758: Fix possible integer overflow in the _csv module for large record lengths. diff --git a/Modules/binascii.c b/Modules/binascii.c --- a/Modules/binascii.c +++ b/Modules/binascii.c @@ -1370,6 +1370,7 @@ /* First, scan to see how many characters need to be encoded */ in = 0; while (in < datalen) { + Py_ssize_t delta = 0; if ((databuf[in] > 126) || (databuf[in] == '=') || (header && databuf[in] == '_') || @@ -1384,12 +1385,12 @@ if ((linelen + 3) >= MAXLINESIZE) { linelen = 0; if (crlf) - odatalen += 3; + delta += 3; else - odatalen += 2; + delta += 2; } linelen += 3; - odatalen += 3; + delta += 3; in++; } else { @@ -1401,11 +1402,11 @@ linelen = 0; /* Protect against whitespace on end of line */ if (in && ((databuf[in-1] == ' ') || (databuf[in-1] == '\t'))) - odatalen += 2; + delta += 2; if (crlf) - odatalen += 2; + delta += 2; else - odatalen += 1; + delta += 1; if (databuf[in] == '\r') in += 2; else @@ -1417,15 +1418,20 @@ (linelen + 1) >= MAXLINESIZE) { linelen = 0; if (crlf) - odatalen += 3; + delta += 3; else - odatalen += 2; + delta += 2; } linelen++; - odatalen++; + delta++; in++; } } + if (PY_SSIZE_T_MAX - delta < odatalen) { + PyErr_NoMemory(); + return NULL; + } + odatalen += delta; } /* We allocate the output same size as input, this is overkill. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 14 03:53:16 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 14 Aug 2016 07:53:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327574=3A_Decrease?= =?utf-8?q?d_an_overhead_of_parsing_keyword_arguments_in_functions?= Message-ID: <20160814075259.23391.13853.5AA5BD9E@psf.io> https://hg.python.org/cpython/rev/e527715bd0b3 changeset: 102650:e527715bd0b3 user: Serhiy Storchaka date: Sun Aug 14 10:52:18 2016 +0300 summary: Issue #27574: Decreased an overhead of parsing keyword arguments in functions implemented with using Argument Clinic. files: Doc/whatsnew/3.6.rst | 7 + Include/modsupport.h | 23 + Misc/NEWS | 3 + Modules/_io/clinic/_iomodule.c.h | 7 +- Modules/_io/clinic/bufferedio.c.h | 17 +- Modules/_io/clinic/bytesio.c.h | 7 +- Modules/_io/clinic/fileio.c.h | 7 +- Modules/_io/clinic/stringio.c.h | 7 +- Modules/_io/clinic/textio.c.h | 17 +- Modules/cjkcodecs/clinic/multibytecodec.c.h | 22 +- Modules/clinic/_bz2module.c.h | 7 +- Modules/clinic/_codecsmodule.c.h | 12 +- Modules/clinic/_datetimemodule.c.h | 7 +- Modules/clinic/_elementtree.c.h | 42 +- Modules/clinic/_lzmamodule.c.h | 12 +- Modules/clinic/_pickle.c.h | 32 +- Modules/clinic/_sre.c.h | 77 +- Modules/clinic/_ssl.c.h | 42 +- Modules/clinic/_winapi.c.h | 17 +- Modules/clinic/binascii.c.h | 17 +- Modules/clinic/cmathmodule.c.h | 7 +- Modules/clinic/grpmodule.c.h | 12 +- Modules/clinic/md5module.c.h | 7 +- Modules/clinic/posixmodule.c.h | 312 ++++-- Modules/clinic/pyexpat.c.h | 7 +- Modules/clinic/sha1module.c.h | 7 +- Modules/clinic/sha256module.c.h | 12 +- Modules/clinic/sha512module.c.h | 12 +- Modules/clinic/zlibmodule.c.h | 17 +- Objects/clinic/bytearrayobject.c.h | 22 +- Objects/clinic/bytesobject.c.h | 22 +- PC/clinic/winreg.c.h | 27 +- Python/clinic/bltinmodule.c.h | 7 +- Python/getargs.c | 449 +++++++++- Python/pylifecycle.c | 1 + Tools/clinic/clinic.py | 14 +- 36 files changed, 963 insertions(+), 354 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -579,6 +579,13 @@ deserializing many small objects (Contributed by Victor Stinner in :issue:`27056`). +- Passing :term:`keyword arguments ` to a function has an + overhead in comparison with passing :term:`positional arguments + `. Now in extension functions implemented with using + Argument Clinic this overhead is significantly decreased. + (Contributed by Serhiy Storchaka in :issue:`27574`). + + Build and C API Changes ======================= diff --git a/Include/modsupport.h b/Include/modsupport.h --- a/Include/modsupport.h +++ b/Include/modsupport.h @@ -44,6 +44,29 @@ #endif PyAPI_FUNC(PyObject *) Py_VaBuildValue(const char *, va_list); +#ifndef Py_LIMITED_API +typedef struct _PyArg_Parser { + const char *format; + const char * const *keywords; + const char *fname; + const char *custom_msg; + int pos; /* number of positional-only arguments */ + int min; /* minimal number of arguments */ + int max; /* maximal number of positional arguments */ + PyObject *kwtuple; /* tuple of keyword parameter names */ + struct _PyArg_Parser *next; +} _PyArg_Parser; +#ifdef PY_SSIZE_T_CLEAN +#define _PyArg_ParseTupleAndKeywordsFast _PyArg_ParseTupleAndKeywordsFast_SizeT +#define _PyArg_VaParseTupleAndKeywordsFast _PyArg_VaParseTupleAndKeywordsFast_SizeT +#endif +PyAPI_FUNC(int) _PyArg_ParseTupleAndKeywordsFast(PyObject *, PyObject *, + struct _PyArg_Parser *, ...); +PyAPI_FUNC(int) _PyArg_VaParseTupleAndKeywordsFast(PyObject *, PyObject *, + struct _PyArg_Parser *, va_list); +void _PyArg_Fini(void); +#endif + PyAPI_FUNC(int) PyModule_AddObject(PyObject *, const char *, PyObject *); PyAPI_FUNC(int) PyModule_AddIntConstant(PyObject *, const char *, long); PyAPI_FUNC(int) PyModule_AddStringConstant(PyObject *, const char *, const char *); diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #27574: Decreased an overhead of parsing keyword arguments in functions + implemented with using Argument Clinic. + - Issue #22557: Now importing already imported modules is up to 2.5 times faster. - Issue #17596: Include to help with Min GW building. diff --git a/Modules/_io/clinic/_iomodule.c.h b/Modules/_io/clinic/_iomodule.c.h --- a/Modules/_io/clinic/_iomodule.c.h +++ b/Modules/_io/clinic/_iomodule.c.h @@ -138,7 +138,8 @@ _io_open(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"file", "mode", "buffering", "encoding", "errors", "newline", "closefd", "opener", NULL}; + static const char * const _keywords[] = {"file", "mode", "buffering", "encoding", "errors", "newline", "closefd", "opener", NULL}; + static _PyArg_Parser _parser = {"O|sizzziO:open", _keywords, 0}; PyObject *file; const char *mode = "r"; int buffering = -1; @@ -148,7 +149,7 @@ int closefd = 1; PyObject *opener = Py_None; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|sizzziO:open", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &file, &mode, &buffering, &encoding, &errors, &newline, &closefd, &opener)) { goto exit; } @@ -157,4 +158,4 @@ exit: return return_value; } -/*[clinic end generated code: output=ae2facf262cf464e input=a9049054013a1b77]*/ +/*[clinic end generated code: output=14769629391a3130 input=a9049054013a1b77]*/ diff --git a/Modules/_io/clinic/bufferedio.c.h b/Modules/_io/clinic/bufferedio.c.h --- a/Modules/_io/clinic/bufferedio.c.h +++ b/Modules/_io/clinic/bufferedio.c.h @@ -324,11 +324,12 @@ _io_BufferedReader___init__(PyObject *self, PyObject *args, PyObject *kwargs) { int return_value = -1; - static char *_keywords[] = {"raw", "buffer_size", NULL}; + static const char * const _keywords[] = {"raw", "buffer_size", NULL}; + static _PyArg_Parser _parser = {"O|n:BufferedReader", _keywords, 0}; PyObject *raw; Py_ssize_t buffer_size = DEFAULT_BUFFER_SIZE; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|n:BufferedReader", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &raw, &buffer_size)) { goto exit; } @@ -356,11 +357,12 @@ _io_BufferedWriter___init__(PyObject *self, PyObject *args, PyObject *kwargs) { int return_value = -1; - static char *_keywords[] = {"raw", "buffer_size", NULL}; + static const char * const _keywords[] = {"raw", "buffer_size", NULL}; + static _PyArg_Parser _parser = {"O|n:BufferedWriter", _keywords, 0}; PyObject *raw; Py_ssize_t buffer_size = DEFAULT_BUFFER_SIZE; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|n:BufferedWriter", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &raw, &buffer_size)) { goto exit; } @@ -459,11 +461,12 @@ _io_BufferedRandom___init__(PyObject *self, PyObject *args, PyObject *kwargs) { int return_value = -1; - static char *_keywords[] = {"raw", "buffer_size", NULL}; + static const char * const _keywords[] = {"raw", "buffer_size", NULL}; + static _PyArg_Parser _parser = {"O|n:BufferedRandom", _keywords, 0}; PyObject *raw; Py_ssize_t buffer_size = DEFAULT_BUFFER_SIZE; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|n:BufferedRandom", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &raw, &buffer_size)) { goto exit; } @@ -472,4 +475,4 @@ exit: return return_value; } -/*[clinic end generated code: output=4f6196c756b880c8 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=a956f394ecde4cf9 input=a9049054013a1b77]*/ diff --git a/Modules/_io/clinic/bytesio.c.h b/Modules/_io/clinic/bytesio.c.h --- a/Modules/_io/clinic/bytesio.c.h +++ b/Modules/_io/clinic/bytesio.c.h @@ -415,10 +415,11 @@ _io_BytesIO___init__(PyObject *self, PyObject *args, PyObject *kwargs) { int return_value = -1; - static char *_keywords[] = {"initial_bytes", NULL}; + static const char * const _keywords[] = {"initial_bytes", NULL}; + static _PyArg_Parser _parser = {"|O:BytesIO", _keywords, 0}; PyObject *initvalue = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O:BytesIO", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &initvalue)) { goto exit; } @@ -427,4 +428,4 @@ exit: return return_value; } -/*[clinic end generated code: output=3fdb62f3e3b0544d input=a9049054013a1b77]*/ +/*[clinic end generated code: output=6382e8eb578eea64 input=a9049054013a1b77]*/ diff --git a/Modules/_io/clinic/fileio.c.h b/Modules/_io/clinic/fileio.c.h --- a/Modules/_io/clinic/fileio.c.h +++ b/Modules/_io/clinic/fileio.c.h @@ -49,13 +49,14 @@ _io_FileIO___init__(PyObject *self, PyObject *args, PyObject *kwargs) { int return_value = -1; - static char *_keywords[] = {"file", "mode", "closefd", "opener", NULL}; + static const char * const _keywords[] = {"file", "mode", "closefd", "opener", NULL}; + static _PyArg_Parser _parser = {"O|siO:FileIO", _keywords, 0}; PyObject *nameobj; const char *mode = "r"; int closefd = 1; PyObject *opener = Py_None; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|siO:FileIO", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &nameobj, &mode, &closefd, &opener)) { goto exit; } @@ -372,4 +373,4 @@ #ifndef _IO_FILEIO_TRUNCATE_METHODDEF #define _IO_FILEIO_TRUNCATE_METHODDEF #endif /* !defined(_IO_FILEIO_TRUNCATE_METHODDEF) */ -/*[clinic end generated code: output=bf4b4bd6b976346d input=a9049054013a1b77]*/ +/*[clinic end generated code: output=51924bc0ee11d58e input=a9049054013a1b77]*/ diff --git a/Modules/_io/clinic/stringio.c.h b/Modules/_io/clinic/stringio.c.h --- a/Modules/_io/clinic/stringio.c.h +++ b/Modules/_io/clinic/stringio.c.h @@ -221,11 +221,12 @@ _io_StringIO___init__(PyObject *self, PyObject *args, PyObject *kwargs) { int return_value = -1; - static char *_keywords[] = {"initial_value", "newline", NULL}; + static const char * const _keywords[] = {"initial_value", "newline", NULL}; + static _PyArg_Parser _parser = {"|OO:StringIO", _keywords, 0}; PyObject *value = NULL; PyObject *newline_obj = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|OO:StringIO", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &value, &newline_obj)) { goto exit; } @@ -288,4 +289,4 @@ { return _io_StringIO_seekable_impl(self); } -/*[clinic end generated code: output=0513219581cbe952 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=5dd5c2a213e75405 input=a9049054013a1b77]*/ diff --git a/Modules/_io/clinic/textio.c.h b/Modules/_io/clinic/textio.c.h --- a/Modules/_io/clinic/textio.c.h +++ b/Modules/_io/clinic/textio.c.h @@ -24,12 +24,13 @@ _io_IncrementalNewlineDecoder___init__(PyObject *self, PyObject *args, PyObject *kwargs) { int return_value = -1; - static char *_keywords[] = {"decoder", "translate", "errors", NULL}; + static const char * const _keywords[] = {"decoder", "translate", "errors", NULL}; + static _PyArg_Parser _parser = {"Oi|O:IncrementalNewlineDecoder", _keywords, 0}; PyObject *decoder; int translate; PyObject *errors = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "Oi|O:IncrementalNewlineDecoder", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &decoder, &translate, &errors)) { goto exit; } @@ -55,11 +56,12 @@ _io_IncrementalNewlineDecoder_decode(nldecoder_object *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"input", "final", NULL}; + static const char * const _keywords[] = {"input", "final", NULL}; + static _PyArg_Parser _parser = {"O|i:decode", _keywords, 0}; PyObject *input; int final = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|i:decode", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &input, &final)) { goto exit; } @@ -155,7 +157,8 @@ _io_TextIOWrapper___init__(PyObject *self, PyObject *args, PyObject *kwargs) { int return_value = -1; - static char *_keywords[] = {"buffer", "encoding", "errors", "newline", "line_buffering", "write_through", NULL}; + static const char * const _keywords[] = {"buffer", "encoding", "errors", "newline", "line_buffering", "write_through", NULL}; + static _PyArg_Parser _parser = {"O|zzzii:TextIOWrapper", _keywords, 0}; PyObject *buffer; const char *encoding = NULL; const char *errors = NULL; @@ -163,7 +166,7 @@ int line_buffering = 0; int write_through = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|zzzii:TextIOWrapper", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &buffer, &encoding, &errors, &newline, &line_buffering, &write_through)) { goto exit; } @@ -461,4 +464,4 @@ { return _io_TextIOWrapper_close_impl(self); } -/*[clinic end generated code: output=31a39bbbe07ae4e7 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=7ec624a9bf6393f5 input=a9049054013a1b77]*/ diff --git a/Modules/cjkcodecs/clinic/multibytecodec.c.h b/Modules/cjkcodecs/clinic/multibytecodec.c.h --- a/Modules/cjkcodecs/clinic/multibytecodec.c.h +++ b/Modules/cjkcodecs/clinic/multibytecodec.c.h @@ -25,11 +25,12 @@ _multibytecodec_MultibyteCodec_encode(MultibyteCodecObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"input", "errors", NULL}; + static const char * const _keywords[] = {"input", "errors", NULL}; + static _PyArg_Parser _parser = {"O|z:encode", _keywords, 0}; PyObject *input; const char *errors = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|z:encode", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &input, &errors)) { goto exit; } @@ -62,11 +63,12 @@ _multibytecodec_MultibyteCodec_decode(MultibyteCodecObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"input", "errors", NULL}; + static const char * const _keywords[] = {"input", "errors", NULL}; + static _PyArg_Parser _parser = {"y*|z:decode", _keywords, 0}; Py_buffer input = {NULL, NULL}; const char *errors = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "y*|z:decode", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &input, &errors)) { goto exit; } @@ -98,11 +100,12 @@ _multibytecodec_MultibyteIncrementalEncoder_encode(MultibyteIncrementalEncoderObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"input", "final", NULL}; + static const char * const _keywords[] = {"input", "final", NULL}; + static _PyArg_Parser _parser = {"O|i:encode", _keywords, 0}; PyObject *input; int final = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|i:encode", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &input, &final)) { goto exit; } @@ -146,11 +149,12 @@ _multibytecodec_MultibyteIncrementalDecoder_decode(MultibyteIncrementalDecoderObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"input", "final", NULL}; + static const char * const _keywords[] = {"input", "final", NULL}; + static _PyArg_Parser _parser = {"y*|i:decode", _keywords, 0}; Py_buffer input = {NULL, NULL}; int final = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "y*|i:decode", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &input, &final)) { goto exit; } @@ -326,4 +330,4 @@ #define _MULTIBYTECODEC___CREATE_CODEC_METHODDEF \ {"__create_codec", (PyCFunction)_multibytecodec___create_codec, METH_O, _multibytecodec___create_codec__doc__}, -/*[clinic end generated code: output=f837bc56b2fa2a4e input=a9049054013a1b77]*/ +/*[clinic end generated code: output=8e86fa162c85230b input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_bz2module.c.h b/Modules/clinic/_bz2module.c.h --- a/Modules/clinic/_bz2module.c.h +++ b/Modules/clinic/_bz2module.c.h @@ -125,11 +125,12 @@ _bz2_BZ2Decompressor_decompress(BZ2Decompressor *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"data", "max_length", NULL}; + static const char * const _keywords[] = {"data", "max_length", NULL}; + static _PyArg_Parser _parser = {"y*|n:decompress", _keywords, 0}; Py_buffer data = {NULL, NULL}; Py_ssize_t max_length = -1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "y*|n:decompress", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &data, &max_length)) { goto exit; } @@ -173,4 +174,4 @@ exit: return return_value; } -/*[clinic end generated code: output=71be22f38224fe84 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=40e5ef049f9e719b input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_codecsmodule.c.h b/Modules/clinic/_codecsmodule.c.h --- a/Modules/clinic/_codecsmodule.c.h +++ b/Modules/clinic/_codecsmodule.c.h @@ -65,12 +65,13 @@ _codecs_encode(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"obj", "encoding", "errors", NULL}; + static const char * const _keywords[] = {"obj", "encoding", "errors", NULL}; + static _PyArg_Parser _parser = {"O|ss:encode", _keywords, 0}; PyObject *obj; const char *encoding = NULL; const char *errors = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|ss:encode", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &obj, &encoding, &errors)) { goto exit; } @@ -103,12 +104,13 @@ _codecs_decode(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"obj", "encoding", "errors", NULL}; + static const char * const _keywords[] = {"obj", "encoding", "errors", NULL}; + static _PyArg_Parser _parser = {"O|ss:decode", _keywords, 0}; PyObject *obj; const char *encoding = NULL; const char *errors = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|ss:decode", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &obj, &encoding, &errors)) { goto exit; } @@ -1455,4 +1457,4 @@ #ifndef _CODECS_CODE_PAGE_ENCODE_METHODDEF #define _CODECS_CODE_PAGE_ENCODE_METHODDEF #endif /* !defined(_CODECS_CODE_PAGE_ENCODE_METHODDEF) */ -/*[clinic end generated code: output=6e89ff4423c12a9b input=a9049054013a1b77]*/ +/*[clinic end generated code: output=0221e4eece62c905 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_datetimemodule.c.h b/Modules/clinic/_datetimemodule.c.h --- a/Modules/clinic/_datetimemodule.c.h +++ b/Modules/clinic/_datetimemodule.c.h @@ -23,10 +23,11 @@ datetime_datetime_now(PyTypeObject *type, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"tz", NULL}; + static const char * const _keywords[] = {"tz", NULL}; + static _PyArg_Parser _parser = {"|O:now", _keywords, 0}; PyObject *tz = Py_None; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O:now", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &tz)) { goto exit; } @@ -35,4 +36,4 @@ exit: return return_value; } -/*[clinic end generated code: output=a82e6bd057a5dab9 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=61f85af5637df8b5 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_elementtree.c.h b/Modules/clinic/_elementtree.c.h --- a/Modules/clinic/_elementtree.c.h +++ b/Modules/clinic/_elementtree.c.h @@ -146,11 +146,12 @@ _elementtree_Element_find(ElementObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "namespaces", NULL}; + static const char * const _keywords[] = {"path", "namespaces", NULL}; + static _PyArg_Parser _parser = {"O|O:find", _keywords, 0}; PyObject *path; PyObject *namespaces = Py_None; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|O:find", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &path, &namespaces)) { goto exit; } @@ -177,12 +178,13 @@ _elementtree_Element_findtext(ElementObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "default", "namespaces", NULL}; + static const char * const _keywords[] = {"path", "default", "namespaces", NULL}; + static _PyArg_Parser _parser = {"O|OO:findtext", _keywords, 0}; PyObject *path; PyObject *default_value = Py_None; PyObject *namespaces = Py_None; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|OO:findtext", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &path, &default_value, &namespaces)) { goto exit; } @@ -208,11 +210,12 @@ _elementtree_Element_findall(ElementObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "namespaces", NULL}; + static const char * const _keywords[] = {"path", "namespaces", NULL}; + static _PyArg_Parser _parser = {"O|O:findall", _keywords, 0}; PyObject *path; PyObject *namespaces = Py_None; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|O:findall", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &path, &namespaces)) { goto exit; } @@ -238,11 +241,12 @@ _elementtree_Element_iterfind(ElementObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "namespaces", NULL}; + static const char * const _keywords[] = {"path", "namespaces", NULL}; + static _PyArg_Parser _parser = {"O|O:iterfind", _keywords, 0}; PyObject *path; PyObject *namespaces = Py_None; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|O:iterfind", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &path, &namespaces)) { goto exit; } @@ -268,11 +272,12 @@ _elementtree_Element_get(ElementObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"key", "default", NULL}; + static const char * const _keywords[] = {"key", "default", NULL}; + static _PyArg_Parser _parser = {"O|O:get", _keywords, 0}; PyObject *key; PyObject *default_value = Py_None; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|O:get", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &key, &default_value)) { goto exit; } @@ -314,10 +319,11 @@ _elementtree_Element_iter(ElementObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"tag", NULL}; + static const char * const _keywords[] = {"tag", NULL}; + static _PyArg_Parser _parser = {"|O:iter", _keywords, 0}; PyObject *tag = Py_None; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O:iter", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &tag)) { goto exit; } @@ -501,10 +507,11 @@ _elementtree_TreeBuilder___init__(PyObject *self, PyObject *args, PyObject *kwargs) { int return_value = -1; - static char *_keywords[] = {"element_factory", NULL}; + static const char * const _keywords[] = {"element_factory", NULL}; + static _PyArg_Parser _parser = {"|O:TreeBuilder", _keywords, 0}; PyObject *element_factory = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O:TreeBuilder", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &element_factory)) { goto exit; } @@ -585,12 +592,13 @@ _elementtree_XMLParser___init__(PyObject *self, PyObject *args, PyObject *kwargs) { int return_value = -1; - static char *_keywords[] = {"html", "target", "encoding", NULL}; + static const char * const _keywords[] = {"html", "target", "encoding", NULL}; + static _PyArg_Parser _parser = {"|OOz:XMLParser", _keywords, 0}; PyObject *html = NULL; PyObject *target = NULL; const char *encoding = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|OOz:XMLParser", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &html, &target, &encoding)) { goto exit; } @@ -694,4 +702,4 @@ exit: return return_value; } -/*[clinic end generated code: output=491eb5718c1ae64b input=a9049054013a1b77]*/ +/*[clinic end generated code: output=4c5e94c28a009ce6 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_lzmamodule.c.h b/Modules/clinic/_lzmamodule.c.h --- a/Modules/clinic/_lzmamodule.c.h +++ b/Modules/clinic/_lzmamodule.c.h @@ -91,11 +91,12 @@ _lzma_LZMADecompressor_decompress(Decompressor *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"data", "max_length", NULL}; + static const char * const _keywords[] = {"data", "max_length", NULL}; + static _PyArg_Parser _parser = {"y*|n:decompress", _keywords, 0}; Py_buffer data = {NULL, NULL}; Py_ssize_t max_length = -1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "y*|n:decompress", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &data, &max_length)) { goto exit; } @@ -141,12 +142,13 @@ _lzma_LZMADecompressor___init__(PyObject *self, PyObject *args, PyObject *kwargs) { int return_value = -1; - static char *_keywords[] = {"format", "memlimit", "filters", NULL}; + static const char * const _keywords[] = {"format", "memlimit", "filters", NULL}; + static _PyArg_Parser _parser = {"|iOO:LZMADecompressor", _keywords, 0}; int format = FORMAT_AUTO; PyObject *memlimit = Py_None; PyObject *filters = Py_None; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|iOO:LZMADecompressor", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &format, &memlimit, &filters)) { goto exit; } @@ -254,4 +256,4 @@ return return_value; } -/*[clinic end generated code: output=25bf57a0845d147a input=a9049054013a1b77]*/ +/*[clinic end generated code: output=9434583fe111c771 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_pickle.c.h b/Modules/clinic/_pickle.c.h --- a/Modules/clinic/_pickle.c.h +++ b/Modules/clinic/_pickle.c.h @@ -93,12 +93,13 @@ _pickle_Pickler___init__(PyObject *self, PyObject *args, PyObject *kwargs) { int return_value = -1; - static char *_keywords[] = {"file", "protocol", "fix_imports", NULL}; + static const char * const _keywords[] = {"file", "protocol", "fix_imports", NULL}; + static _PyArg_Parser _parser = {"O|Op:Pickler", _keywords, 0}; PyObject *file; PyObject *protocol = NULL; int fix_imports = 1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|Op:Pickler", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &file, &protocol, &fix_imports)) { goto exit; } @@ -285,13 +286,14 @@ _pickle_Unpickler___init__(PyObject *self, PyObject *args, PyObject *kwargs) { int return_value = -1; - static char *_keywords[] = {"file", "fix_imports", "encoding", "errors", NULL}; + static const char * const _keywords[] = {"file", "fix_imports", "encoding", "errors", NULL}; + static _PyArg_Parser _parser = {"O|$pss:Unpickler", _keywords, 0}; PyObject *file; int fix_imports = 1; const char *encoding = "ASCII"; const char *errors = "strict"; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|$pss:Unpickler", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &file, &fix_imports, &encoding, &errors)) { goto exit; } @@ -392,13 +394,14 @@ _pickle_dump(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"obj", "file", "protocol", "fix_imports", NULL}; + static const char * const _keywords[] = {"obj", "file", "protocol", "fix_imports", NULL}; + static _PyArg_Parser _parser = {"OO|O$p:dump", _keywords, 0}; PyObject *obj; PyObject *file; PyObject *protocol = NULL; int fix_imports = 1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "OO|O$p:dump", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &obj, &file, &protocol, &fix_imports)) { goto exit; } @@ -437,12 +440,13 @@ _pickle_dumps(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"obj", "protocol", "fix_imports", NULL}; + static const char * const _keywords[] = {"obj", "protocol", "fix_imports", NULL}; + static _PyArg_Parser _parser = {"O|O$p:dumps", _keywords, 0}; PyObject *obj; PyObject *protocol = NULL; int fix_imports = 1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|O$p:dumps", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &obj, &protocol, &fix_imports)) { goto exit; } @@ -492,13 +496,14 @@ _pickle_load(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"file", "fix_imports", "encoding", "errors", NULL}; + static const char * const _keywords[] = {"file", "fix_imports", "encoding", "errors", NULL}; + static _PyArg_Parser _parser = {"O|$pss:load", _keywords, 0}; PyObject *file; int fix_imports = 1; const char *encoding = "ASCII"; const char *errors = "strict"; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|$pss:load", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &file, &fix_imports, &encoding, &errors)) { goto exit; } @@ -539,13 +544,14 @@ _pickle_loads(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"data", "fix_imports", "encoding", "errors", NULL}; + static const char * const _keywords[] = {"data", "fix_imports", "encoding", "errors", NULL}; + static _PyArg_Parser _parser = {"O|$pss:loads", _keywords, 0}; PyObject *data; int fix_imports = 1; const char *encoding = "ASCII"; const char *errors = "strict"; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|$pss:loads", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &data, &fix_imports, &encoding, &errors)) { goto exit; } @@ -554,4 +560,4 @@ exit: return return_value; } -/*[clinic end generated code: output=5c5f9149df292ce4 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=50f9127109673c98 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_sre.c.h b/Modules/clinic/_sre.c.h --- a/Modules/clinic/_sre.c.h +++ b/Modules/clinic/_sre.c.h @@ -80,13 +80,14 @@ _sre_SRE_Pattern_match(PatternObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"string", "pos", "endpos", "pattern", NULL}; + static const char * const _keywords[] = {"string", "pos", "endpos", "pattern", NULL}; + static _PyArg_Parser _parser = {"|Onn$O:match", _keywords, 0}; PyObject *string = NULL; Py_ssize_t pos = 0; Py_ssize_t endpos = PY_SSIZE_T_MAX; PyObject *pattern = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|Onn$O:match", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &string, &pos, &endpos, &pattern)) { goto exit; } @@ -115,13 +116,14 @@ _sre_SRE_Pattern_fullmatch(PatternObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"string", "pos", "endpos", "pattern", NULL}; + static const char * const _keywords[] = {"string", "pos", "endpos", "pattern", NULL}; + static _PyArg_Parser _parser = {"|Onn$O:fullmatch", _keywords, 0}; PyObject *string = NULL; Py_ssize_t pos = 0; Py_ssize_t endpos = PY_SSIZE_T_MAX; PyObject *pattern = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|Onn$O:fullmatch", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &string, &pos, &endpos, &pattern)) { goto exit; } @@ -152,13 +154,14 @@ _sre_SRE_Pattern_search(PatternObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"string", "pos", "endpos", "pattern", NULL}; + static const char * const _keywords[] = {"string", "pos", "endpos", "pattern", NULL}; + static _PyArg_Parser _parser = {"|Onn$O:search", _keywords, 0}; PyObject *string = NULL; Py_ssize_t pos = 0; Py_ssize_t endpos = PY_SSIZE_T_MAX; PyObject *pattern = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|Onn$O:search", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &string, &pos, &endpos, &pattern)) { goto exit; } @@ -187,13 +190,14 @@ _sre_SRE_Pattern_findall(PatternObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"string", "pos", "endpos", "source", NULL}; + static const char * const _keywords[] = {"string", "pos", "endpos", "source", NULL}; + static _PyArg_Parser _parser = {"|Onn$O:findall", _keywords, 0}; PyObject *string = NULL; Py_ssize_t pos = 0; Py_ssize_t endpos = PY_SSIZE_T_MAX; PyObject *source = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|Onn$O:findall", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &string, &pos, &endpos, &source)) { goto exit; } @@ -222,12 +226,13 @@ _sre_SRE_Pattern_finditer(PatternObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"string", "pos", "endpos", NULL}; + static const char * const _keywords[] = {"string", "pos", "endpos", NULL}; + static _PyArg_Parser _parser = {"O|nn:finditer", _keywords, 0}; PyObject *string; Py_ssize_t pos = 0; Py_ssize_t endpos = PY_SSIZE_T_MAX; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|nn:finditer", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &string, &pos, &endpos)) { goto exit; } @@ -253,12 +258,13 @@ _sre_SRE_Pattern_scanner(PatternObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"string", "pos", "endpos", NULL}; + static const char * const _keywords[] = {"string", "pos", "endpos", NULL}; + static _PyArg_Parser _parser = {"O|nn:scanner", _keywords, 0}; PyObject *string; Py_ssize_t pos = 0; Py_ssize_t endpos = PY_SSIZE_T_MAX; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|nn:scanner", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &string, &pos, &endpos)) { goto exit; } @@ -285,12 +291,13 @@ _sre_SRE_Pattern_split(PatternObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"string", "maxsplit", "source", NULL}; + static const char * const _keywords[] = {"string", "maxsplit", "source", NULL}; + static _PyArg_Parser _parser = {"|On$O:split", _keywords, 0}; PyObject *string = NULL; Py_ssize_t maxsplit = 0; PyObject *source = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|On$O:split", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &string, &maxsplit, &source)) { goto exit; } @@ -317,12 +324,13 @@ _sre_SRE_Pattern_sub(PatternObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"repl", "string", "count", NULL}; + static const char * const _keywords[] = {"repl", "string", "count", NULL}; + static _PyArg_Parser _parser = {"OO|n:sub", _keywords, 0}; PyObject *repl; PyObject *string; Py_ssize_t count = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "OO|n:sub", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &repl, &string, &count)) { goto exit; } @@ -349,12 +357,13 @@ _sre_SRE_Pattern_subn(PatternObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"repl", "string", "count", NULL}; + static const char * const _keywords[] = {"repl", "string", "count", NULL}; + static _PyArg_Parser _parser = {"OO|n:subn", _keywords, 0}; PyObject *repl; PyObject *string; Py_ssize_t count = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "OO|n:subn", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &repl, &string, &count)) { goto exit; } @@ -396,10 +405,11 @@ _sre_SRE_Pattern___deepcopy__(PatternObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"memo", NULL}; + static const char * const _keywords[] = {"memo", NULL}; + static _PyArg_Parser _parser = {"O:__deepcopy__", _keywords, 0}; PyObject *memo; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O:__deepcopy__", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &memo)) { goto exit; } @@ -427,7 +437,8 @@ _sre_compile(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"pattern", "flags", "code", "groups", "groupindex", "indexgroup", NULL}; + static const char * const _keywords[] = {"pattern", "flags", "code", "groups", "groupindex", "indexgroup", NULL}; + static _PyArg_Parser _parser = {"OiO!nOO:compile", _keywords, 0}; PyObject *pattern; int flags; PyObject *code; @@ -435,7 +446,7 @@ PyObject *groupindex; PyObject *indexgroup; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "OiO!nOO:compile", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &pattern, &flags, &PyList_Type, &code, &groups, &groupindex, &indexgroup)) { goto exit; } @@ -461,10 +472,11 @@ _sre_SRE_Match_expand(MatchObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"template", NULL}; + static const char * const _keywords[] = {"template", NULL}; + static _PyArg_Parser _parser = {"O:expand", _keywords, 0}; PyObject *template; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O:expand", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &template)) { goto exit; } @@ -493,10 +505,11 @@ _sre_SRE_Match_groups(MatchObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"default", NULL}; + static const char * const _keywords[] = {"default", NULL}; + static _PyArg_Parser _parser = {"|O:groups", _keywords, 0}; PyObject *default_value = Py_None; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O:groups", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &default_value)) { goto exit; } @@ -525,10 +538,11 @@ _sre_SRE_Match_groupdict(MatchObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"default", NULL}; + static const char * const _keywords[] = {"default", NULL}; + static _PyArg_Parser _parser = {"|O:groupdict", _keywords, 0}; PyObject *default_value = Py_None; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O:groupdict", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &default_value)) { goto exit; } @@ -667,10 +681,11 @@ _sre_SRE_Match___deepcopy__(MatchObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"memo", NULL}; + static const char * const _keywords[] = {"memo", NULL}; + static _PyArg_Parser _parser = {"O:__deepcopy__", _keywords, 0}; PyObject *memo; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O:__deepcopy__", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &memo)) { goto exit; } @@ -713,4 +728,4 @@ { return _sre_SRE_Scanner_search_impl(self); } -/*[clinic end generated code: output=af9455cb54b2a907 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=2cbc2b1482738e54 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_ssl.c.h b/Modules/clinic/_ssl.c.h --- a/Modules/clinic/_ssl.c.h +++ b/Modules/clinic/_ssl.c.h @@ -458,12 +458,13 @@ _ssl__SSLContext_load_cert_chain(PySSLContext *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"certfile", "keyfile", "password", NULL}; + static const char * const _keywords[] = {"certfile", "keyfile", "password", NULL}; + static _PyArg_Parser _parser = {"O|OO:load_cert_chain", _keywords, 0}; PyObject *certfile; PyObject *keyfile = NULL; PyObject *password = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|OO:load_cert_chain", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &certfile, &keyfile, &password)) { goto exit; } @@ -491,12 +492,13 @@ _ssl__SSLContext_load_verify_locations(PySSLContext *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"cafile", "capath", "cadata", NULL}; + static const char * const _keywords[] = {"cafile", "capath", "cadata", NULL}; + static _PyArg_Parser _parser = {"|OOO:load_verify_locations", _keywords, 0}; PyObject *cafile = NULL; PyObject *capath = NULL; PyObject *cadata = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|OOO:load_verify_locations", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &cafile, &capath, &cadata)) { goto exit; } @@ -530,12 +532,13 @@ _ssl__SSLContext__wrap_socket(PySSLContext *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"sock", "server_side", "server_hostname", NULL}; + static const char * const _keywords[] = {"sock", "server_side", "server_hostname", NULL}; + static _PyArg_Parser _parser = {"O!i|O:_wrap_socket", _keywords, 0}; PyObject *sock; int server_side; PyObject *hostname_obj = Py_None; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O!i|O:_wrap_socket", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, PySocketModule.Sock_Type, &sock, &server_side, &hostname_obj)) { goto exit; } @@ -563,13 +566,14 @@ _ssl__SSLContext__wrap_bio(PySSLContext *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"incoming", "outgoing", "server_side", "server_hostname", NULL}; + static const char * const _keywords[] = {"incoming", "outgoing", "server_side", "server_hostname", NULL}; + static _PyArg_Parser _parser = {"O!O!i|O:_wrap_bio", _keywords, 0}; PySSLMemoryBIO *incoming; PySSLMemoryBIO *outgoing; int server_side; PyObject *hostname_obj = Py_None; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O!O!i|O:_wrap_bio", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &PySSLMemoryBIO_Type, &incoming, &PySSLMemoryBIO_Type, &outgoing, &server_side, &hostname_obj)) { goto exit; } @@ -684,10 +688,11 @@ _ssl__SSLContext_get_ca_certs(PySSLContext *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"binary_form", NULL}; + static const char * const _keywords[] = {"binary_form", NULL}; + static _PyArg_Parser _parser = {"|p:get_ca_certs", _keywords, 0}; int binary_form = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|p:get_ca_certs", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &binary_form)) { goto exit; } @@ -994,11 +999,12 @@ _ssl_txt2obj(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"txt", "name", NULL}; + static const char * const _keywords[] = {"txt", "name", NULL}; + static _PyArg_Parser _parser = {"s|p:txt2obj", _keywords, 0}; const char *txt; int name = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "s|p:txt2obj", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &txt, &name)) { goto exit; } @@ -1059,10 +1065,11 @@ _ssl_enum_certificates(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"store_name", NULL}; + static const char * const _keywords[] = {"store_name", NULL}; + static _PyArg_Parser _parser = {"s:enum_certificates", _keywords, 0}; const char *store_name; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "s:enum_certificates", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &store_name)) { goto exit; } @@ -1097,10 +1104,11 @@ _ssl_enum_crls(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"store_name", NULL}; + static const char * const _keywords[] = {"store_name", NULL}; + static _PyArg_Parser _parser = {"s:enum_crls", _keywords, 0}; const char *store_name; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "s:enum_crls", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &store_name)) { goto exit; } @@ -1135,4 +1143,4 @@ #ifndef _SSL_ENUM_CRLS_METHODDEF #define _SSL_ENUM_CRLS_METHODDEF #endif /* !defined(_SSL_ENUM_CRLS_METHODDEF) */ -/*[clinic end generated code: output=02444732c19722b3 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=6057f95343369849 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_winapi.c.h b/Modules/clinic/_winapi.c.h --- a/Modules/clinic/_winapi.c.h +++ b/Modules/clinic/_winapi.c.h @@ -105,11 +105,12 @@ _winapi_ConnectNamedPipe(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"handle", "overlapped", NULL}; + static const char * const _keywords[] = {"handle", "overlapped", NULL}; + static _PyArg_Parser _parser = {"" F_HANDLE "|i:ConnectNamedPipe", _keywords, 0}; HANDLE handle; int use_overlapped = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "" F_HANDLE "|i:ConnectNamedPipe", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &handle, &use_overlapped)) { goto exit; } @@ -679,12 +680,13 @@ _winapi_ReadFile(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"handle", "size", "overlapped", NULL}; + static const char * const _keywords[] = {"handle", "size", "overlapped", NULL}; + static _PyArg_Parser _parser = {"" F_HANDLE "i|i:ReadFile", _keywords, 0}; HANDLE handle; int size; int use_overlapped = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "" F_HANDLE "i|i:ReadFile", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &handle, &size, &use_overlapped)) { goto exit; } @@ -872,12 +874,13 @@ _winapi_WriteFile(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"handle", "buffer", "overlapped", NULL}; + static const char * const _keywords[] = {"handle", "buffer", "overlapped", NULL}; + static _PyArg_Parser _parser = {"" F_HANDLE "O|i:WriteFile", _keywords, 0}; HANDLE handle; PyObject *buffer; int use_overlapped = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "" F_HANDLE "O|i:WriteFile", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &handle, &buffer, &use_overlapped)) { goto exit; } @@ -886,4 +889,4 @@ exit: return return_value; } -/*[clinic end generated code: output=8032f3371c14749e input=a9049054013a1b77]*/ +/*[clinic end generated code: output=4bfccfb32ab726e8 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/binascii.c.h b/Modules/clinic/binascii.c.h --- a/Modules/clinic/binascii.c.h +++ b/Modules/clinic/binascii.c.h @@ -112,11 +112,12 @@ binascii_b2a_base64(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"data", "newline", NULL}; + static const char * const _keywords[] = {"data", "newline", NULL}; + static _PyArg_Parser _parser = {"y*|$i:b2a_base64", _keywords, 0}; Py_buffer data = {NULL, NULL}; int newline = 1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "y*|$i:b2a_base64", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &data, &newline)) { goto exit; } @@ -488,11 +489,12 @@ binascii_a2b_qp(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"data", "header", NULL}; + static const char * const _keywords[] = {"data", "header", NULL}; + static _PyArg_Parser _parser = {"O&|i:a2b_qp", _keywords, 0}; Py_buffer data = {NULL, NULL}; int header = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&|i:a2b_qp", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, ascii_buffer_converter, &data, &header)) { goto exit; } @@ -527,13 +529,14 @@ binascii_b2a_qp(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"data", "quotetabs", "istext", "header", NULL}; + static const char * const _keywords[] = {"data", "quotetabs", "istext", "header", NULL}; + static _PyArg_Parser _parser = {"y*|iii:b2a_qp", _keywords, 0}; Py_buffer data = {NULL, NULL}; int quotetabs = 0; int istext = 1; int header = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "y*|iii:b2a_qp", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &data, "etabs, &istext, &header)) { goto exit; } @@ -547,4 +550,4 @@ return return_value; } -/*[clinic end generated code: output=d91d1058dc0590e1 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=12611b05d8bf4a9c input=a9049054013a1b77]*/ diff --git a/Modules/clinic/cmathmodule.c.h b/Modules/clinic/cmathmodule.c.h --- a/Modules/clinic/cmathmodule.c.h +++ b/Modules/clinic/cmathmodule.c.h @@ -861,14 +861,15 @@ cmath_isclose(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"a", "b", "rel_tol", "abs_tol", NULL}; + static const char * const _keywords[] = {"a", "b", "rel_tol", "abs_tol", NULL}; + static _PyArg_Parser _parser = {"DD|$dd:isclose", _keywords, 0}; Py_complex a; Py_complex b; double rel_tol = 1e-09; double abs_tol = 0.0; int _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "DD|$dd:isclose", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &a, &b, &rel_tol, &abs_tol)) { goto exit; } @@ -881,4 +882,4 @@ exit: return return_value; } -/*[clinic end generated code: output=0f49dd11b50175bc input=a9049054013a1b77]*/ +/*[clinic end generated code: output=aa2e77ca9fc26928 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/grpmodule.c.h b/Modules/clinic/grpmodule.c.h --- a/Modules/clinic/grpmodule.c.h +++ b/Modules/clinic/grpmodule.c.h @@ -20,10 +20,11 @@ grp_getgrgid(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"id", NULL}; + static const char * const _keywords[] = {"id", NULL}; + static _PyArg_Parser _parser = {"O:getgrgid", _keywords, 0}; PyObject *id; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O:getgrgid", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &id)) { goto exit; } @@ -51,10 +52,11 @@ grp_getgrnam(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"name", NULL}; + static const char * const _keywords[] = {"name", NULL}; + static _PyArg_Parser _parser = {"U:getgrnam", _keywords, 0}; PyObject *name; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "U:getgrnam", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &name)) { goto exit; } @@ -84,4 +86,4 @@ { return grp_getgrall_impl(module); } -/*[clinic end generated code: output=8b7502970a29e7f1 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=c06081097b7fffe7 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/md5module.c.h b/Modules/clinic/md5module.c.h --- a/Modules/clinic/md5module.c.h +++ b/Modules/clinic/md5module.c.h @@ -81,10 +81,11 @@ _md5_md5(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"string", NULL}; + static const char * const _keywords[] = {"string", NULL}; + static _PyArg_Parser _parser = {"|O:md5", _keywords, 0}; PyObject *string = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O:md5", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &string)) { goto exit; } @@ -93,4 +94,4 @@ exit: return return_value; } -/*[clinic end generated code: output=8f640a98761daffe input=a9049054013a1b77]*/ +/*[clinic end generated code: output=f86fc2f3f21831e2 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -36,12 +36,13 @@ os_stat(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "dir_fd", "follow_symlinks", NULL}; + static const char * const _keywords[] = {"path", "dir_fd", "follow_symlinks", NULL}; + static _PyArg_Parser _parser = {"O&|$O&p:stat", _keywords, 0}; path_t path = PATH_T_INITIALIZE("stat", "path", 0, 1); int dir_fd = DEFAULT_DIR_FD; int follow_symlinks = 1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&|$O&p:stat", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, FSTATAT_DIR_FD_CONVERTER, &dir_fd, &follow_symlinks)) { goto exit; } @@ -73,11 +74,12 @@ os_lstat(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "dir_fd", NULL}; + static const char * const _keywords[] = {"path", "dir_fd", NULL}; + static _PyArg_Parser _parser = {"O&|$O&:lstat", _keywords, 0}; path_t path = PATH_T_INITIALIZE("lstat", "path", 0, 0); int dir_fd = DEFAULT_DIR_FD; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&|$O&:lstat", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, FSTATAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -133,7 +135,8 @@ os_access(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "mode", "dir_fd", "effective_ids", "follow_symlinks", NULL}; + static const char * const _keywords[] = {"path", "mode", "dir_fd", "effective_ids", "follow_symlinks", NULL}; + static _PyArg_Parser _parser = {"O&i|$O&pp:access", _keywords, 0}; path_t path = PATH_T_INITIALIZE("access", "path", 0, 1); int mode; int dir_fd = DEFAULT_DIR_FD; @@ -141,7 +144,7 @@ int follow_symlinks = 1; int _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&i|$O&pp:access", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, &mode, FACCESSAT_DIR_FD_CONVERTER, &dir_fd, &effective_ids, &follow_symlinks)) { goto exit; } @@ -239,10 +242,11 @@ os_chdir(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", NULL}; + static const char * const _keywords[] = {"path", NULL}; + static _PyArg_Parser _parser = {"O&:chdir", _keywords, 0}; path_t path = PATH_T_INITIALIZE("chdir", "path", 0, PATH_HAVE_FCHDIR); - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&:chdir", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path)) { goto exit; } @@ -276,10 +280,11 @@ os_fchdir(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"fd", NULL}; + static const char * const _keywords[] = {"fd", NULL}; + static _PyArg_Parser _parser = {"O&:fchdir", _keywords, 0}; int fd; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&:fchdir", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, fildes_converter, &fd)) { goto exit; } @@ -328,13 +333,14 @@ os_chmod(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "mode", "dir_fd", "follow_symlinks", NULL}; + static const char * const _keywords[] = {"path", "mode", "dir_fd", "follow_symlinks", NULL}; + static _PyArg_Parser _parser = {"O&i|$O&p:chmod", _keywords, 0}; path_t path = PATH_T_INITIALIZE("chmod", "path", 0, PATH_HAVE_FCHMOD); int mode; int dir_fd = DEFAULT_DIR_FD; int follow_symlinks = 1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&i|$O&p:chmod", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, &mode, FCHMODAT_DIR_FD_CONVERTER, &dir_fd, &follow_symlinks)) { goto exit; } @@ -367,11 +373,12 @@ os_fchmod(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"fd", "mode", NULL}; + static const char * const _keywords[] = {"fd", "mode", NULL}; + static _PyArg_Parser _parser = {"ii:fchmod", _keywords, 0}; int fd; int mode; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "ii:fchmod", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &fd, &mode)) { goto exit; } @@ -404,11 +411,12 @@ os_lchmod(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "mode", NULL}; + static const char * const _keywords[] = {"path", "mode", NULL}; + static _PyArg_Parser _parser = {"O&i:lchmod", _keywords, 0}; path_t path = PATH_T_INITIALIZE("lchmod", "path", 0, 0); int mode; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&i:lchmod", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, &mode)) { goto exit; } @@ -448,12 +456,13 @@ os_chflags(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "flags", "follow_symlinks", NULL}; + static const char * const _keywords[] = {"path", "flags", "follow_symlinks", NULL}; + static _PyArg_Parser _parser = {"O&k|p:chflags", _keywords, 0}; path_t path = PATH_T_INITIALIZE("chflags", "path", 0, 0); unsigned long flags; int follow_symlinks = 1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&k|p:chflags", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, &flags, &follow_symlinks)) { goto exit; } @@ -489,11 +498,12 @@ os_lchflags(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "flags", NULL}; + static const char * const _keywords[] = {"path", "flags", NULL}; + static _PyArg_Parser _parser = {"O&k:lchflags", _keywords, 0}; path_t path = PATH_T_INITIALIZE("lchflags", "path", 0, 0); unsigned long flags; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&k:lchflags", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, &flags)) { goto exit; } @@ -526,10 +536,11 @@ os_chroot(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", NULL}; + static const char * const _keywords[] = {"path", NULL}; + static _PyArg_Parser _parser = {"O&:chroot", _keywords, 0}; path_t path = PATH_T_INITIALIZE("chroot", "path", 0, 0); - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&:chroot", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path)) { goto exit; } @@ -562,10 +573,11 @@ os_fsync(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"fd", NULL}; + static const char * const _keywords[] = {"fd", NULL}; + static _PyArg_Parser _parser = {"O&:fsync", _keywords, 0}; int fd; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&:fsync", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, fildes_converter, &fd)) { goto exit; } @@ -617,10 +629,11 @@ os_fdatasync(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"fd", NULL}; + static const char * const _keywords[] = {"fd", NULL}; + static _PyArg_Parser _parser = {"O&:fdatasync", _keywords, 0}; int fd; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&:fdatasync", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, fildes_converter, &fd)) { goto exit; } @@ -675,14 +688,15 @@ os_chown(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "uid", "gid", "dir_fd", "follow_symlinks", NULL}; + static const char * const _keywords[] = {"path", "uid", "gid", "dir_fd", "follow_symlinks", NULL}; + static _PyArg_Parser _parser = {"O&O&O&|$O&p:chown", _keywords, 0}; path_t path = PATH_T_INITIALIZE("chown", "path", 0, PATH_HAVE_FCHOWN); uid_t uid; gid_t gid; int dir_fd = DEFAULT_DIR_FD; int follow_symlinks = 1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&O&O&|$O&p:chown", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, _Py_Uid_Converter, &uid, _Py_Gid_Converter, &gid, FCHOWNAT_DIR_FD_CONVERTER, &dir_fd, &follow_symlinks)) { goto exit; } @@ -717,12 +731,13 @@ os_fchown(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"fd", "uid", "gid", NULL}; + static const char * const _keywords[] = {"fd", "uid", "gid", NULL}; + static _PyArg_Parser _parser = {"iO&O&:fchown", _keywords, 0}; int fd; uid_t uid; gid_t gid; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "iO&O&:fchown", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &fd, _Py_Uid_Converter, &uid, _Py_Gid_Converter, &gid)) { goto exit; } @@ -755,12 +770,13 @@ os_lchown(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "uid", "gid", NULL}; + static const char * const _keywords[] = {"path", "uid", "gid", NULL}; + static _PyArg_Parser _parser = {"O&O&O&:lchown", _keywords, 0}; path_t path = PATH_T_INITIALIZE("lchown", "path", 0, 0); uid_t uid; gid_t gid; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&O&O&:lchown", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, _Py_Uid_Converter, &uid, _Py_Gid_Converter, &gid)) { goto exit; } @@ -841,14 +857,15 @@ os_link(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"src", "dst", "src_dir_fd", "dst_dir_fd", "follow_symlinks", NULL}; + static const char * const _keywords[] = {"src", "dst", "src_dir_fd", "dst_dir_fd", "follow_symlinks", NULL}; + static _PyArg_Parser _parser = {"O&O&|$O&O&p:link", _keywords, 0}; path_t src = PATH_T_INITIALIZE("link", "src", 0, 0); path_t dst = PATH_T_INITIALIZE("link", "dst", 0, 0); int src_dir_fd = DEFAULT_DIR_FD; int dst_dir_fd = DEFAULT_DIR_FD; int follow_symlinks = 1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&O&|$O&O&p:link", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &src, path_converter, &dst, dir_fd_converter, &src_dir_fd, dir_fd_converter, &dst_dir_fd, &follow_symlinks)) { goto exit; } @@ -892,10 +909,11 @@ os_listdir(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", NULL}; + static const char * const _keywords[] = {"path", NULL}; + static _PyArg_Parser _parser = {"|O&:listdir", _keywords, 0}; path_t path = PATH_T_INITIALIZE("listdir", "path", 1, PATH_HAVE_FDOPENDIR); - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O&:listdir", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path)) { goto exit; } @@ -1023,10 +1041,11 @@ os__getvolumepathname(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", NULL}; + static const char * const _keywords[] = {"path", NULL}; + static _PyArg_Parser _parser = {"U:_getvolumepathname", _keywords, 0}; PyObject *path; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "U:_getvolumepathname", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &path)) { goto exit; } @@ -1061,12 +1080,13 @@ os_mkdir(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "mode", "dir_fd", NULL}; + static const char * const _keywords[] = {"path", "mode", "dir_fd", NULL}; + static _PyArg_Parser _parser = {"O&|i$O&:mkdir", _keywords, 0}; path_t path = PATH_T_INITIALIZE("mkdir", "path", 0, 0); int mode = 511; int dir_fd = DEFAULT_DIR_FD; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&|i$O&:mkdir", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, &mode, MKDIRAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -1128,11 +1148,12 @@ os_getpriority(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"which", "who", NULL}; + static const char * const _keywords[] = {"which", "who", NULL}; + static _PyArg_Parser _parser = {"ii:getpriority", _keywords, 0}; int which; int who; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "ii:getpriority", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &which, &who)) { goto exit; } @@ -1162,12 +1183,13 @@ os_setpriority(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"which", "who", "priority", NULL}; + static const char * const _keywords[] = {"which", "who", "priority", NULL}; + static _PyArg_Parser _parser = {"iii:setpriority", _keywords, 0}; int which; int who; int priority; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "iii:setpriority", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &which, &who, &priority)) { goto exit; } @@ -1202,13 +1224,14 @@ os_rename(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"src", "dst", "src_dir_fd", "dst_dir_fd", NULL}; + static const char * const _keywords[] = {"src", "dst", "src_dir_fd", "dst_dir_fd", NULL}; + static _PyArg_Parser _parser = {"O&O&|$O&O&:rename", _keywords, 0}; path_t src = PATH_T_INITIALIZE("rename", "src", 0, 0); path_t dst = PATH_T_INITIALIZE("rename", "dst", 0, 0); int src_dir_fd = DEFAULT_DIR_FD; int dst_dir_fd = DEFAULT_DIR_FD; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&O&|$O&O&:rename", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &src, path_converter, &dst, dir_fd_converter, &src_dir_fd, dir_fd_converter, &dst_dir_fd)) { goto exit; } @@ -1246,13 +1269,14 @@ os_replace(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"src", "dst", "src_dir_fd", "dst_dir_fd", NULL}; + static const char * const _keywords[] = {"src", "dst", "src_dir_fd", "dst_dir_fd", NULL}; + static _PyArg_Parser _parser = {"O&O&|$O&O&:replace", _keywords, 0}; path_t src = PATH_T_INITIALIZE("replace", "src", 0, 0); path_t dst = PATH_T_INITIALIZE("replace", "dst", 0, 0); int src_dir_fd = DEFAULT_DIR_FD; int dst_dir_fd = DEFAULT_DIR_FD; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&O&|$O&O&:replace", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &src, path_converter, &dst, dir_fd_converter, &src_dir_fd, dir_fd_converter, &dst_dir_fd)) { goto exit; } @@ -1288,11 +1312,12 @@ os_rmdir(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "dir_fd", NULL}; + static const char * const _keywords[] = {"path", "dir_fd", NULL}; + static _PyArg_Parser _parser = {"O&|$O&:rmdir", _keywords, 0}; path_t path = PATH_T_INITIALIZE("rmdir", "path", 0, 0); int dir_fd = DEFAULT_DIR_FD; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&|$O&:rmdir", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, UNLINKAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -1323,11 +1348,12 @@ os_system(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"command", NULL}; + static const char * const _keywords[] = {"command", NULL}; + static _PyArg_Parser _parser = {"u:system", _keywords, 0}; Py_UNICODE *command; long _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "u:system", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &command)) { goto exit; } @@ -1361,11 +1387,12 @@ os_system(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"command", NULL}; + static const char * const _keywords[] = {"command", NULL}; + static _PyArg_Parser _parser = {"O&:system", _keywords, 0}; PyObject *command = NULL; long _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&:system", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, PyUnicode_FSConverter, &command)) { goto exit; } @@ -1432,11 +1459,12 @@ os_unlink(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "dir_fd", NULL}; + static const char * const _keywords[] = {"path", "dir_fd", NULL}; + static _PyArg_Parser _parser = {"O&|$O&:unlink", _keywords, 0}; path_t path = PATH_T_INITIALIZE("unlink", "path", 0, 0); int dir_fd = DEFAULT_DIR_FD; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&|$O&:unlink", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, UNLINKAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -1470,11 +1498,12 @@ os_remove(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "dir_fd", NULL}; + static const char * const _keywords[] = {"path", "dir_fd", NULL}; + static _PyArg_Parser _parser = {"O&|$O&:remove", _keywords, 0}; path_t path = PATH_T_INITIALIZE("remove", "path", 0, 0); int dir_fd = DEFAULT_DIR_FD; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&|$O&:remove", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, UNLINKAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -1552,14 +1581,15 @@ os_utime(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "times", "ns", "dir_fd", "follow_symlinks", NULL}; + static const char * const _keywords[] = {"path", "times", "ns", "dir_fd", "follow_symlinks", NULL}; + static _PyArg_Parser _parser = {"O&|O$OO&p:utime", _keywords, 0}; path_t path = PATH_T_INITIALIZE("utime", "path", 0, PATH_UTIME_HAVE_FD); PyObject *times = NULL; PyObject *ns = NULL; int dir_fd = DEFAULT_DIR_FD; int follow_symlinks = 1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&|O$OO&p:utime", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, ×, &ns, FUTIMENSAT_DIR_FD_CONVERTER, &dir_fd, &follow_symlinks)) { goto exit; } @@ -1588,10 +1618,11 @@ os__exit(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"status", NULL}; + static const char * const _keywords[] = {"status", NULL}; + static _PyArg_Parser _parser = {"i:_exit", _keywords, 0}; int status; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i:_exit", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &status)) { goto exit; } @@ -1667,12 +1698,13 @@ os_execve(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "argv", "env", NULL}; + static const char * const _keywords[] = {"path", "argv", "env", NULL}; + static _PyArg_Parser _parser = {"O&OO:execve", _keywords, 0}; path_t path = PATH_T_INITIALIZE("execve", "path", 0, PATH_HAVE_FEXECVE); PyObject *argv; PyObject *env; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&OO:execve", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, &argv, &env)) { goto exit; } @@ -1845,10 +1877,11 @@ os_sched_get_priority_max(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"policy", NULL}; + static const char * const _keywords[] = {"policy", NULL}; + static _PyArg_Parser _parser = {"i:sched_get_priority_max", _keywords, 0}; int policy; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i:sched_get_priority_max", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &policy)) { goto exit; } @@ -1878,10 +1911,11 @@ os_sched_get_priority_min(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"policy", NULL}; + static const char * const _keywords[] = {"policy", NULL}; + static _PyArg_Parser _parser = {"i:sched_get_priority_min", _keywords, 0}; int policy; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i:sched_get_priority_min", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &policy)) { goto exit; } @@ -1944,10 +1978,11 @@ os_sched_param(PyTypeObject *type, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"sched_priority", NULL}; + static const char * const _keywords[] = {"sched_priority", NULL}; + static _PyArg_Parser _parser = {"O:sched_param", _keywords, 0}; PyObject *sched_priority; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O:sched_param", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &sched_priority)) { goto exit; } @@ -2372,10 +2407,11 @@ os_getpgid(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"pid", NULL}; + static const char * const _keywords[] = {"pid", NULL}; + static _PyArg_Parser _parser = {"" _Py_PARSE_PID ":getpgid", _keywords, 0}; pid_t pid; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "" _Py_PARSE_PID ":getpgid", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &pid)) { goto exit; } @@ -2821,10 +2857,11 @@ os_wait3(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"options", NULL}; + static const char * const _keywords[] = {"options", NULL}; + static _PyArg_Parser _parser = {"i:wait3", _keywords, 0}; int options; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i:wait3", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &options)) { goto exit; } @@ -2857,11 +2894,12 @@ os_wait4(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"pid", "options", NULL}; + static const char * const _keywords[] = {"pid", "options", NULL}; + static _PyArg_Parser _parser = {"" _Py_PARSE_PID "i:wait4", _keywords, 0}; pid_t pid; int options; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "" _Py_PARSE_PID "i:wait4", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &pid, &options)) { goto exit; } @@ -3048,13 +3086,14 @@ os_symlink(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"src", "dst", "target_is_directory", "dir_fd", NULL}; + static const char * const _keywords[] = {"src", "dst", "target_is_directory", "dir_fd", NULL}; + static _PyArg_Parser _parser = {"O&O&|p$O&:symlink", _keywords, 0}; path_t src = PATH_T_INITIALIZE("symlink", "src", 0, 0); path_t dst = PATH_T_INITIALIZE("symlink", "dst", 0, 0); int target_is_directory = 0; int dir_fd = DEFAULT_DIR_FD; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&O&|p$O&:symlink", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &src, path_converter, &dst, &target_is_directory, SYMLINKAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -3268,14 +3307,15 @@ os_open(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "flags", "mode", "dir_fd", NULL}; + static const char * const _keywords[] = {"path", "flags", "mode", "dir_fd", NULL}; + static _PyArg_Parser _parser = {"O&i|i$O&:open", _keywords, 0}; path_t path = PATH_T_INITIALIZE("open", "path", 0, 0); int flags; int mode = 511; int dir_fd = DEFAULT_DIR_FD; int _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&i|i$O&:open", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, &flags, &mode, OPENAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -3308,10 +3348,11 @@ os_close(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"fd", NULL}; + static const char * const _keywords[] = {"fd", NULL}; + static _PyArg_Parser _parser = {"i:close", _keywords, 0}; int fd; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i:close", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &fd)) { goto exit; } @@ -3398,12 +3439,13 @@ os_dup2(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"fd", "fd2", "inheritable", NULL}; + static const char * const _keywords[] = {"fd", "fd2", "inheritable", NULL}; + static _PyArg_Parser _parser = {"ii|p:dup2", _keywords, 0}; int fd; int fd2; int inheritable = 1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "ii|p:dup2", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &fd, &fd2, &inheritable)) { goto exit; } @@ -3662,10 +3704,11 @@ os_fstat(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"fd", NULL}; + static const char * const _keywords[] = {"fd", NULL}; + static _PyArg_Parser _parser = {"i:fstat", _keywords, 0}; int fd; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i:fstat", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &fd)) { goto exit; } @@ -3884,12 +3927,13 @@ os_mkfifo(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "mode", "dir_fd", NULL}; + static const char * const _keywords[] = {"path", "mode", "dir_fd", NULL}; + static _PyArg_Parser _parser = {"O&|i$O&:mkfifo", _keywords, 0}; path_t path = PATH_T_INITIALIZE("mkfifo", "path", 0, 0); int mode = 438; int dir_fd = DEFAULT_DIR_FD; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&|i$O&:mkfifo", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, &mode, MKFIFOAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -3935,13 +3979,14 @@ os_mknod(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "mode", "device", "dir_fd", NULL}; + static const char * const _keywords[] = {"path", "mode", "device", "dir_fd", NULL}; + static _PyArg_Parser _parser = {"O&|iO&$O&:mknod", _keywords, 0}; path_t path = PATH_T_INITIALIZE("mknod", "path", 0, 0); int mode = 384; dev_t device = 0; int dir_fd = DEFAULT_DIR_FD; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&|iO&$O&:mknod", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, &mode, _Py_Dev_Converter, &device, MKNODAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -4120,11 +4165,12 @@ os_truncate(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "length", NULL}; + static const char * const _keywords[] = {"path", "length", NULL}; + static _PyArg_Parser _parser = {"O&O&:truncate", _keywords, 0}; path_t path = PATH_T_INITIALIZE("truncate", "path", 0, PATH_HAVE_FTRUNCATE); Py_off_t length; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&O&:truncate", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, Py_off_t_converter, &length)) { goto exit; } @@ -4410,11 +4456,12 @@ os_WIFCONTINUED(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"status", NULL}; + static const char * const _keywords[] = {"status", NULL}; + static _PyArg_Parser _parser = {"i:WIFCONTINUED", _keywords, 0}; int status; int _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i:WIFCONTINUED", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &status)) { goto exit; } @@ -4448,11 +4495,12 @@ os_WIFSTOPPED(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"status", NULL}; + static const char * const _keywords[] = {"status", NULL}; + static _PyArg_Parser _parser = {"i:WIFSTOPPED", _keywords, 0}; int status; int _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i:WIFSTOPPED", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &status)) { goto exit; } @@ -4486,11 +4534,12 @@ os_WIFSIGNALED(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"status", NULL}; + static const char * const _keywords[] = {"status", NULL}; + static _PyArg_Parser _parser = {"i:WIFSIGNALED", _keywords, 0}; int status; int _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i:WIFSIGNALED", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &status)) { goto exit; } @@ -4524,11 +4573,12 @@ os_WIFEXITED(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"status", NULL}; + static const char * const _keywords[] = {"status", NULL}; + static _PyArg_Parser _parser = {"i:WIFEXITED", _keywords, 0}; int status; int _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i:WIFEXITED", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &status)) { goto exit; } @@ -4562,11 +4612,12 @@ os_WEXITSTATUS(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"status", NULL}; + static const char * const _keywords[] = {"status", NULL}; + static _PyArg_Parser _parser = {"i:WEXITSTATUS", _keywords, 0}; int status; int _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i:WEXITSTATUS", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &status)) { goto exit; } @@ -4600,11 +4651,12 @@ os_WTERMSIG(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"status", NULL}; + static const char * const _keywords[] = {"status", NULL}; + static _PyArg_Parser _parser = {"i:WTERMSIG", _keywords, 0}; int status; int _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i:WTERMSIG", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &status)) { goto exit; } @@ -4638,11 +4690,12 @@ os_WSTOPSIG(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"status", NULL}; + static const char * const _keywords[] = {"status", NULL}; + static _PyArg_Parser _parser = {"i:WSTOPSIG", _keywords, 0}; int status; int _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i:WSTOPSIG", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &status)) { goto exit; } @@ -4713,10 +4766,11 @@ os_statvfs(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", NULL}; + static const char * const _keywords[] = {"path", NULL}; + static _PyArg_Parser _parser = {"O&:statvfs", _keywords, 0}; path_t path = PATH_T_INITIALIZE("statvfs", "path", 0, PATH_HAVE_FSTATVFS); - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&:statvfs", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path)) { goto exit; } @@ -4749,10 +4803,11 @@ os__getdiskusage(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", NULL}; + static const char * const _keywords[] = {"path", NULL}; + static _PyArg_Parser _parser = {"u:_getdiskusage", _keywords, 0}; Py_UNICODE *path; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "u:_getdiskusage", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &path)) { goto exit; } @@ -4826,12 +4881,13 @@ os_pathconf(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "name", NULL}; + static const char * const _keywords[] = {"path", "name", NULL}; + static _PyArg_Parser _parser = {"O&O&:pathconf", _keywords, 0}; path_t path = PATH_T_INITIALIZE("pathconf", "path", 0, PATH_HAVE_FPATHCONF); int name; long _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&O&:pathconf", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, conv_path_confname, &name)) { goto exit; } @@ -4983,10 +5039,11 @@ os_device_encoding(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"fd", NULL}; + static const char * const _keywords[] = {"fd", NULL}; + static _PyArg_Parser _parser = {"i:device_encoding", _keywords, 0}; int fd; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i:device_encoding", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &fd)) { goto exit; } @@ -5132,12 +5189,13 @@ os_getxattr(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "attribute", "follow_symlinks", NULL}; + static const char * const _keywords[] = {"path", "attribute", "follow_symlinks", NULL}; + static _PyArg_Parser _parser = {"O&O&|$p:getxattr", _keywords, 0}; path_t path = PATH_T_INITIALIZE("getxattr", "path", 0, 1); path_t attribute = PATH_T_INITIALIZE("getxattr", "attribute", 0, 0); int follow_symlinks = 1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&O&|$p:getxattr", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, path_converter, &attribute, &follow_symlinks)) { goto exit; } @@ -5179,14 +5237,15 @@ os_setxattr(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "attribute", "value", "flags", "follow_symlinks", NULL}; + static const char * const _keywords[] = {"path", "attribute", "value", "flags", "follow_symlinks", NULL}; + static _PyArg_Parser _parser = {"O&O&y*|i$p:setxattr", _keywords, 0}; path_t path = PATH_T_INITIALIZE("setxattr", "path", 0, 1); path_t attribute = PATH_T_INITIALIZE("setxattr", "attribute", 0, 0); Py_buffer value = {NULL, NULL}; int flags = 0; int follow_symlinks = 1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&O&y*|i$p:setxattr", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, path_converter, &attribute, &value, &flags, &follow_symlinks)) { goto exit; } @@ -5231,12 +5290,13 @@ os_removexattr(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "attribute", "follow_symlinks", NULL}; + static const char * const _keywords[] = {"path", "attribute", "follow_symlinks", NULL}; + static _PyArg_Parser _parser = {"O&O&|$p:removexattr", _keywords, 0}; path_t path = PATH_T_INITIALIZE("removexattr", "path", 0, 1); path_t attribute = PATH_T_INITIALIZE("removexattr", "attribute", 0, 0); int follow_symlinks = 1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&O&|$p:removexattr", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, path_converter, &attribute, &follow_symlinks)) { goto exit; } @@ -5277,11 +5337,12 @@ os_listxattr(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", "follow_symlinks", NULL}; + static const char * const _keywords[] = {"path", "follow_symlinks", NULL}; + static _PyArg_Parser _parser = {"|O&$p:listxattr", _keywords, 0}; path_t path = PATH_T_INITIALIZE("listxattr", "path", 1, 1); int follow_symlinks = 1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O&$p:listxattr", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, path_converter, &path, &follow_symlinks)) { goto exit; } @@ -5496,10 +5557,11 @@ os_fspath(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"path", NULL}; + static const char * const _keywords[] = {"path", NULL}; + static _PyArg_Parser _parser = {"O:fspath", _keywords, 0}; PyObject *path; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O:fspath", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &path)) { goto exit; } @@ -5980,4 +6042,4 @@ #ifndef OS_SET_HANDLE_INHERITABLE_METHODDEF #define OS_SET_HANDLE_INHERITABLE_METHODDEF #endif /* !defined(OS_SET_HANDLE_INHERITABLE_METHODDEF) */ -/*[clinic end generated code: output=e91e62d8e8f1b6ac input=a9049054013a1b77]*/ +/*[clinic end generated code: output=97180b6734421a7d input=a9049054013a1b77]*/ diff --git a/Modules/clinic/pyexpat.c.h b/Modules/clinic/pyexpat.c.h --- a/Modules/clinic/pyexpat.c.h +++ b/Modules/clinic/pyexpat.c.h @@ -243,12 +243,13 @@ pyexpat_ParserCreate(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"encoding", "namespace_separator", "intern", NULL}; + static const char * const _keywords[] = {"encoding", "namespace_separator", "intern", NULL}; + static _PyArg_Parser _parser = {"|zzO:ParserCreate", _keywords, 0}; const char *encoding = NULL; const char *namespace_separator = NULL; PyObject *intern = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|zzO:ParserCreate", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &encoding, &namespace_separator, &intern)) { goto exit; } @@ -288,4 +289,4 @@ #ifndef PYEXPAT_XMLPARSER_USEFOREIGNDTD_METHODDEF #define PYEXPAT_XMLPARSER_USEFOREIGNDTD_METHODDEF #endif /* !defined(PYEXPAT_XMLPARSER_USEFOREIGNDTD_METHODDEF) */ -/*[clinic end generated code: output=9de21f46734b1311 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=93cfe662f2bc48e5 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/sha1module.c.h b/Modules/clinic/sha1module.c.h --- a/Modules/clinic/sha1module.c.h +++ b/Modules/clinic/sha1module.c.h @@ -81,10 +81,11 @@ _sha1_sha1(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"string", NULL}; + static const char * const _keywords[] = {"string", NULL}; + static _PyArg_Parser _parser = {"|O:sha1", _keywords, 0}; PyObject *string = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O:sha1", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &string)) { goto exit; } @@ -93,4 +94,4 @@ exit: return return_value; } -/*[clinic end generated code: output=475c4cc749ab31b1 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=549a5d08c248337d input=a9049054013a1b77]*/ diff --git a/Modules/clinic/sha256module.c.h b/Modules/clinic/sha256module.c.h --- a/Modules/clinic/sha256module.c.h +++ b/Modules/clinic/sha256module.c.h @@ -81,10 +81,11 @@ _sha256_sha256(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"string", NULL}; + static const char * const _keywords[] = {"string", NULL}; + static _PyArg_Parser _parser = {"|O:sha256", _keywords, 0}; PyObject *string = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O:sha256", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &string)) { goto exit; } @@ -110,10 +111,11 @@ _sha256_sha224(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"string", NULL}; + static const char * const _keywords[] = {"string", NULL}; + static _PyArg_Parser _parser = {"|O:sha224", _keywords, 0}; PyObject *string = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O:sha224", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &string)) { goto exit; } @@ -122,4 +124,4 @@ exit: return return_value; } -/*[clinic end generated code: output=a41a21c08fcddd70 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=a1296ba6d0780051 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/sha512module.c.h b/Modules/clinic/sha512module.c.h --- a/Modules/clinic/sha512module.c.h +++ b/Modules/clinic/sha512module.c.h @@ -99,10 +99,11 @@ _sha512_sha512(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"string", NULL}; + static const char * const _keywords[] = {"string", NULL}; + static _PyArg_Parser _parser = {"|O:sha512", _keywords, 0}; PyObject *string = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O:sha512", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &string)) { goto exit; } @@ -132,10 +133,11 @@ _sha512_sha384(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"string", NULL}; + static const char * const _keywords[] = {"string", NULL}; + static _PyArg_Parser _parser = {"|O:sha384", _keywords, 0}; PyObject *string = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O:sha384", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &string)) { goto exit; } @@ -170,4 +172,4 @@ #ifndef _SHA512_SHA384_METHODDEF #define _SHA512_SHA384_METHODDEF #endif /* !defined(_SHA512_SHA384_METHODDEF) */ -/*[clinic end generated code: output=e314c0f773abd5d7 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=cf0da76cb603d1bf input=a9049054013a1b77]*/ diff --git a/Modules/clinic/zlibmodule.c.h b/Modules/clinic/zlibmodule.c.h --- a/Modules/clinic/zlibmodule.c.h +++ b/Modules/clinic/zlibmodule.c.h @@ -23,11 +23,12 @@ zlib_compress(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"", "level", NULL}; + static const char * const _keywords[] = {"", "level", NULL}; + static _PyArg_Parser _parser = {"y*|i:compress", _keywords, 0}; Py_buffer data = {NULL, NULL}; int level = Z_DEFAULT_COMPRESSION; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "y*|i:compress", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &data, &level)) { goto exit; } @@ -126,7 +127,8 @@ zlib_compressobj(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"level", "method", "wbits", "memLevel", "strategy", "zdict", NULL}; + static const char * const _keywords[] = {"level", "method", "wbits", "memLevel", "strategy", "zdict", NULL}; + static _PyArg_Parser _parser = {"|iiiiiy*:compressobj", _keywords, 0}; int level = Z_DEFAULT_COMPRESSION; int method = DEFLATED; int wbits = MAX_WBITS; @@ -134,7 +136,7 @@ int strategy = Z_DEFAULT_STRATEGY; Py_buffer zdict = {NULL, NULL}; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|iiiiiy*:compressobj", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &level, &method, &wbits, &memLevel, &strategy, &zdict)) { goto exit; } @@ -171,11 +173,12 @@ zlib_decompressobj(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"wbits", "zdict", NULL}; + static const char * const _keywords[] = {"wbits", "zdict", NULL}; + static _PyArg_Parser _parser = {"|iO:decompressobj", _keywords, 0}; int wbits = MAX_WBITS; PyObject *zdict = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|iO:decompressobj", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &wbits, &zdict)) { goto exit; } @@ -460,4 +463,4 @@ #ifndef ZLIB_COMPRESS_COPY_METHODDEF #define ZLIB_COMPRESS_COPY_METHODDEF #endif /* !defined(ZLIB_COMPRESS_COPY_METHODDEF) */ -/*[clinic end generated code: output=9046866b1ac5de7e input=a9049054013a1b77]*/ +/*[clinic end generated code: output=1fed251c15a9bffa input=a9049054013a1b77]*/ diff --git a/Objects/clinic/bytearrayobject.c.h b/Objects/clinic/bytearrayobject.c.h --- a/Objects/clinic/bytearrayobject.c.h +++ b/Objects/clinic/bytearrayobject.c.h @@ -200,11 +200,12 @@ bytearray_split(PyByteArrayObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"sep", "maxsplit", NULL}; + static const char * const _keywords[] = {"sep", "maxsplit", NULL}; + static _PyArg_Parser _parser = {"|On:split", _keywords, 0}; PyObject *sep = Py_None; Py_ssize_t maxsplit = -1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|On:split", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &sep, &maxsplit)) { goto exit; } @@ -273,11 +274,12 @@ bytearray_rsplit(PyByteArrayObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"sep", "maxsplit", NULL}; + static const char * const _keywords[] = {"sep", "maxsplit", NULL}; + static _PyArg_Parser _parser = {"|On:rsplit", _keywords, 0}; PyObject *sep = Py_None; Py_ssize_t maxsplit = -1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|On:rsplit", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &sep, &maxsplit)) { goto exit; } @@ -564,11 +566,12 @@ bytearray_decode(PyByteArrayObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"encoding", "errors", NULL}; + static const char * const _keywords[] = {"encoding", "errors", NULL}; + static _PyArg_Parser _parser = {"|ss:decode", _keywords, 0}; const char *encoding = NULL; const char *errors = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|ss:decode", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &encoding, &errors)) { goto exit; } @@ -610,10 +613,11 @@ bytearray_splitlines(PyByteArrayObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"keepends", NULL}; + static const char * const _keywords[] = {"keepends", NULL}; + static _PyArg_Parser _parser = {"|i:splitlines", _keywords, 0}; int keepends = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|i:splitlines", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &keepends)) { goto exit; } @@ -716,4 +720,4 @@ { return bytearray_sizeof_impl(self); } -/*[clinic end generated code: output=a32f183ebef159cc input=a9049054013a1b77]*/ +/*[clinic end generated code: output=0af30f8c0b1ecd76 input=a9049054013a1b77]*/ diff --git a/Objects/clinic/bytesobject.c.h b/Objects/clinic/bytesobject.c.h --- a/Objects/clinic/bytesobject.c.h +++ b/Objects/clinic/bytesobject.c.h @@ -26,11 +26,12 @@ bytes_split(PyBytesObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"sep", "maxsplit", NULL}; + static const char * const _keywords[] = {"sep", "maxsplit", NULL}; + static _PyArg_Parser _parser = {"|On:split", _keywords, 0}; PyObject *sep = Py_None; Py_ssize_t maxsplit = -1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|On:split", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &sep, &maxsplit)) { goto exit; } @@ -144,11 +145,12 @@ bytes_rsplit(PyBytesObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"sep", "maxsplit", NULL}; + static const char * const _keywords[] = {"sep", "maxsplit", NULL}; + static _PyArg_Parser _parser = {"|On:rsplit", _keywords, 0}; PyObject *sep = Py_None; Py_ssize_t maxsplit = -1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|On:rsplit", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &sep, &maxsplit)) { goto exit; } @@ -429,11 +431,12 @@ bytes_decode(PyBytesObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"encoding", "errors", NULL}; + static const char * const _keywords[] = {"encoding", "errors", NULL}; + static _PyArg_Parser _parser = {"|ss:decode", _keywords, 0}; const char *encoding = NULL; const char *errors = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|ss:decode", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &encoding, &errors)) { goto exit; } @@ -462,10 +465,11 @@ bytes_splitlines(PyBytesObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"keepends", NULL}; + static const char * const _keywords[] = {"keepends", NULL}; + static _PyArg_Parser _parser = {"|i:splitlines", _keywords, 0}; int keepends = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|i:splitlines", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &keepends)) { goto exit; } @@ -504,4 +508,4 @@ exit: return return_value; } -/*[clinic end generated code: output=6fe884a74e7d49cf input=a9049054013a1b77]*/ +/*[clinic end generated code: output=637c2c14610d3c8d input=a9049054013a1b77]*/ diff --git a/PC/clinic/winreg.c.h b/PC/clinic/winreg.c.h --- a/PC/clinic/winreg.c.h +++ b/PC/clinic/winreg.c.h @@ -87,12 +87,13 @@ winreg_HKEYType___exit__(PyHKEYObject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"exc_type", "exc_value", "traceback", NULL}; + static const char * const _keywords[] = {"exc_type", "exc_value", "traceback", NULL}; + static _PyArg_Parser _parser = {"OOO:__exit__", _keywords, 0}; PyObject *exc_type; PyObject *exc_value; PyObject *traceback; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "OOO:__exit__", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &exc_type, &exc_value, &traceback)) { goto exit; } @@ -244,14 +245,15 @@ winreg_CreateKeyEx(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"key", "sub_key", "reserved", "access", NULL}; + static const char * const _keywords[] = {"key", "sub_key", "reserved", "access", NULL}; + static _PyArg_Parser _parser = {"O&Z|ii:CreateKeyEx", _keywords, 0}; HKEY key; Py_UNICODE *sub_key; int reserved = 0; REGSAM access = KEY_WRITE; HKEY _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&Z|ii:CreateKeyEx", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, clinic_HKEY_converter, &key, &sub_key, &reserved, &access)) { goto exit; } @@ -342,13 +344,14 @@ winreg_DeleteKeyEx(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"key", "sub_key", "access", "reserved", NULL}; + static const char * const _keywords[] = {"key", "sub_key", "access", "reserved", NULL}; + static _PyArg_Parser _parser = {"O&u|ii:DeleteKeyEx", _keywords, 0}; HKEY key; Py_UNICODE *sub_key; REGSAM access = KEY_WOW64_64KEY; int reserved = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&u|ii:DeleteKeyEx", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, clinic_HKEY_converter, &key, &sub_key, &access, &reserved)) { goto exit; } @@ -627,14 +630,15 @@ winreg_OpenKey(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"key", "sub_key", "reserved", "access", NULL}; + static const char * const _keywords[] = {"key", "sub_key", "reserved", "access", NULL}; + static _PyArg_Parser _parser = {"O&Z|ii:OpenKey", _keywords, 0}; HKEY key; Py_UNICODE *sub_key; int reserved = 0; REGSAM access = KEY_READ; HKEY _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&Z|ii:OpenKey", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, clinic_HKEY_converter, &key, &sub_key, &reserved, &access)) { goto exit; } @@ -678,14 +682,15 @@ winreg_OpenKeyEx(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"key", "sub_key", "reserved", "access", NULL}; + static const char * const _keywords[] = {"key", "sub_key", "reserved", "access", NULL}; + static _PyArg_Parser _parser = {"O&Z|ii:OpenKeyEx", _keywords, 0}; HKEY key; Py_UNICODE *sub_key; int reserved = 0; REGSAM access = KEY_READ; HKEY _return_value; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&Z|ii:OpenKeyEx", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, clinic_HKEY_converter, &key, &sub_key, &reserved, &access)) { goto exit; } @@ -1086,4 +1091,4 @@ exit: return return_value; } -/*[clinic end generated code: output=c35ce71f825424d1 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=5b53d19cbe3f37cd input=a9049054013a1b77]*/ diff --git a/Python/clinic/bltinmodule.c.h b/Python/clinic/bltinmodule.c.h --- a/Python/clinic/bltinmodule.c.h +++ b/Python/clinic/bltinmodule.c.h @@ -159,7 +159,8 @@ builtin_compile(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"source", "filename", "mode", "flags", "dont_inherit", "optimize", NULL}; + static const char * const _keywords[] = {"source", "filename", "mode", "flags", "dont_inherit", "optimize", NULL}; + static _PyArg_Parser _parser = {"OO&s|iii:compile", _keywords, 0}; PyObject *source; PyObject *filename; const char *mode; @@ -167,7 +168,7 @@ int dont_inherit = 0; int optimize = -1; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "OO&s|iii:compile", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &source, PyUnicode_FSDecoder, &filename, &mode, &flags, &dont_inherit, &optimize)) { goto exit; } @@ -673,4 +674,4 @@ exit: return return_value; } -/*[clinic end generated code: output=6ab37e6c6d2e7b19 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=790cb3d26531dfda input=a9049054013a1b77]*/ diff --git a/Python/getargs.c b/Python/getargs.c --- a/Python/getargs.c +++ b/Python/getargs.c @@ -18,6 +18,11 @@ int PyArg_VaParseTupleAndKeywords(PyObject *, PyObject *, const char *, char **, va_list); +int _PyArg_ParseTupleAndKeywordsFast(PyObject *, PyObject *, + struct _PyArg_Parser *, ...); +int _PyArg_VaParseTupleAndKeywordsFast(PyObject *, PyObject *, + struct _PyArg_Parser *, va_list); + #ifdef HAVE_DECLSPEC_DLL /* Export functions */ PyAPI_FUNC(int) _PyArg_Parse_SizeT(PyObject *, const char *, ...); @@ -28,6 +33,11 @@ PyAPI_FUNC(int) _PyArg_VaParse_SizeT(PyObject *, const char *, va_list); PyAPI_FUNC(int) _PyArg_VaParseTupleAndKeywords_SizeT(PyObject *, PyObject *, const char *, char **, va_list); + +PyAPI_FUNC(int) _PyArg_ParseTupleAndKeywordsFast_SizeT(PyObject *, PyObject *, + struct _PyArg_Parser *, ...); +PyAPI_FUNC(int) _PyArg_VaParseTupleAndKeywordsFast_SizeT(PyObject *, PyObject *, + struct _PyArg_Parser *, va_list); #endif #define FLAG_COMPAT 1 @@ -67,6 +77,8 @@ static int vgetargskeywords(PyObject *, PyObject *, const char *, char **, va_list *, int); +static int vgetargskeywordsfast(PyObject *, PyObject *, + struct _PyArg_Parser *, va_list *, int); static const char *skipitem(const char **, va_list *, int); int @@ -1418,6 +1430,91 @@ } int +_PyArg_ParseTupleAndKeywordsFast(PyObject *args, PyObject *keywords, + struct _PyArg_Parser *parser, ...) +{ + int retval; + va_list va; + + if ((args == NULL || !PyTuple_Check(args)) || + (keywords != NULL && !PyDict_Check(keywords)) || + parser == NULL) + { + PyErr_BadInternalCall(); + return 0; + } + + va_start(va, parser); + retval = vgetargskeywordsfast(args, keywords, parser, &va, 0); + va_end(va); + return retval; +} + +int +_PyArg_ParseTupleAndKeywordsFast_SizeT(PyObject *args, PyObject *keywords, + struct _PyArg_Parser *parser, ...) +{ + int retval; + va_list va; + + if ((args == NULL || !PyTuple_Check(args)) || + (keywords != NULL && !PyDict_Check(keywords)) || + parser == NULL) + { + PyErr_BadInternalCall(); + return 0; + } + + va_start(va, parser); + retval = vgetargskeywordsfast(args, keywords, parser, &va, FLAG_SIZE_T); + va_end(va); + return retval; +} + + +int +_PyArg_VaParseTupleAndKeywordsFast(PyObject *args, PyObject *keywords, + struct _PyArg_Parser *parser, va_list va) +{ + int retval; + va_list lva; + + if ((args == NULL || !PyTuple_Check(args)) || + (keywords != NULL && !PyDict_Check(keywords)) || + parser == NULL) + { + PyErr_BadInternalCall(); + return 0; + } + + Py_VA_COPY(lva, va); + + retval = vgetargskeywordsfast(args, keywords, parser, &lva, 0); + return retval; +} + +int +_PyArg_VaParseTupleAndKeywordsFast_SizeT(PyObject *args, PyObject *keywords, + struct _PyArg_Parser *parser, va_list va) +{ + int retval; + va_list lva; + + if ((args == NULL || !PyTuple_Check(args)) || + (keywords != NULL && !PyDict_Check(keywords)) || + parser == NULL) + { + PyErr_BadInternalCall(); + return 0; + } + + Py_VA_COPY(lva, va); + + retval = vgetargskeywordsfast(args, keywords, parser, &lva, FLAG_SIZE_T); + return retval; +} + +int PyArg_ValidateKeywordArguments(PyObject *kwargs) { if (!PyDict_Check(kwargs)) { @@ -1541,6 +1638,9 @@ return cleanreturn(0, &freelist); } if (skip) { + /* Now we know the minimal and the maximal numbers of + * positional arguments and can raise an exception with + * informative message (see below). */ break; } if (max < nargs) { @@ -1595,6 +1695,10 @@ assert (min == INT_MAX); assert (max == INT_MAX); skip = 1; + /* At that moment we still don't know the minimal and + * the maximal numbers of positional arguments. Raising + * an exception is deferred until we encounter | and $ + * or the end of the format. */ } else { PyErr_Format(PyExc_TypeError, "Required argument " @@ -1669,6 +1773,300 @@ } +/* List of static parsers. */ +static struct _PyArg_Parser *static_arg_parsers = NULL; + +static int +parser_init(struct _PyArg_Parser *parser) +{ + const char * const *keywords; + const char *format, *msg; + int i, len, min, max, nkw; + PyObject *kwtuple; + + assert(parser->format != NULL); + assert(parser->keywords != NULL); + if (parser->kwtuple != NULL) { + return 1; + } + + /* grab the function name or custom error msg first (mutually exclusive) */ + parser->fname = strchr(parser->format, ':'); + if (parser->fname) { + parser->fname++; + parser->custom_msg = NULL; + } + else { + parser->custom_msg = strchr(parser->format,';'); + if (parser->custom_msg) + parser->custom_msg++; + } + + keywords = parser->keywords; + /* scan keywords and count the number of positional-only parameters */ + for (i = 0; keywords[i] && !*keywords[i]; i++) { + } + parser->pos = i; + /* scan keywords and get greatest possible nbr of args */ + for (; keywords[i]; i++) { + if (!*keywords[i]) { + PyErr_SetString(PyExc_SystemError, + "Empty keyword parameter name"); + return 0; + } + } + len = i; + + min = max = INT_MAX; + format = parser->format; + for (i = 0; i < len; i++) { + if (*format == '|') { + if (min != INT_MAX) { + PyErr_SetString(PyExc_SystemError, + "Invalid format string (| specified twice)"); + return 0; + } + if (max != INT_MAX) { + PyErr_SetString(PyExc_SystemError, + "Invalid format string ($ before |)"); + return 0; + } + min = i; + format++; + } + if (*format == '$') { + if (max != INT_MAX) { + PyErr_SetString(PyExc_SystemError, + "Invalid format string ($ specified twice)"); + return 0; + } + if (i < parser->pos) { + PyErr_SetString(PyExc_SystemError, + "Empty parameter name after $"); + return 0; + } + max = i; + format++; + } + if (IS_END_OF_FORMAT(*format)) { + PyErr_Format(PyExc_SystemError, + "More keyword list entries (%d) than " + "format specifiers (%d)", len, i); + return 0; + } + + msg = skipitem(&format, NULL, 0); + if (msg) { + PyErr_Format(PyExc_SystemError, "%s: '%s'", msg, + format); + return 0; + } + } + parser->min = Py_MIN(min, len); + parser->max = Py_MIN(max, len); + + if (!IS_END_OF_FORMAT(*format) && (*format != '|') && (*format != '$')) { + PyErr_Format(PyExc_SystemError, + "more argument specifiers than keyword list entries " + "(remaining format:'%s')", format); + return 0; + } + + nkw = len - parser->pos; + kwtuple = PyTuple_New(nkw); + if (kwtuple == NULL) { + return 0; + } + keywords = parser->keywords + parser->pos; + for (i = 0; i < nkw; i++) { + PyObject *str = PyUnicode_FromString(keywords[i]); + if (str == NULL) { + Py_DECREF(kwtuple); + return 0; + } + PyUnicode_InternInPlace(&str); + PyTuple_SET_ITEM(kwtuple, i, str); + } + parser->kwtuple = kwtuple; + + assert(parser->next == NULL); + parser->next = static_arg_parsers; + static_arg_parsers = parser; + return 1; +} + +static void +parser_clear(struct _PyArg_Parser *parser) +{ + Py_CLEAR(parser->kwtuple); +} + +static int +vgetargskeywordsfast(PyObject *args, PyObject *keywords, + struct _PyArg_Parser *parser, + va_list *p_va, int flags) +{ + PyObject *kwtuple; + char msgbuf[512]; + int levels[32]; + const char *format; + const char *msg; + PyObject *keyword; + int i, pos, len; + Py_ssize_t nargs, nkeywords; + PyObject *current_arg; + freelistentry_t static_entries[STATIC_FREELIST_ENTRIES]; + freelist_t freelist; + + freelist.entries = static_entries; + freelist.first_available = 0; + freelist.entries_malloced = 0; + + assert(args != NULL && PyTuple_Check(args)); + assert(keywords == NULL || PyDict_Check(keywords)); + assert(parser != NULL); + assert(p_va != NULL); + + if (!parser_init(parser)) { + return 0; + } + + kwtuple = parser->kwtuple; + pos = parser->pos; + len = pos + PyTuple_GET_SIZE(kwtuple); + + if (len > STATIC_FREELIST_ENTRIES) { + freelist.entries = PyMem_NEW(freelistentry_t, len); + if (freelist.entries == NULL) { + PyErr_NoMemory(); + return 0; + } + freelist.entries_malloced = 1; + } + + nargs = PyTuple_GET_SIZE(args); + nkeywords = (keywords == NULL) ? 0 : PyDict_Size(keywords); + if (nargs + nkeywords > len) { + PyErr_Format(PyExc_TypeError, + "%s%s takes at most %d argument%s (%zd given)", + (parser->fname == NULL) ? "function" : parser->fname, + (parser->fname == NULL) ? "" : "()", + len, + (len == 1) ? "" : "s", + nargs + nkeywords); + return cleanreturn(0, &freelist); + } + if (parser->max < nargs) { + PyErr_Format(PyExc_TypeError, + "Function takes %s %d positional arguments (%d given)", + (parser->min != INT_MAX) ? "at most" : "exactly", + parser->max, nargs); + return cleanreturn(0, &freelist); + } + + format = parser->format; + /* convert tuple args and keyword args in same loop, using kwtuple to drive process */ + for (i = 0; i < len; i++) { + keyword = (i >= pos) ? PyTuple_GET_ITEM(kwtuple, i - pos) : NULL; + if (*format == '|') { + format++; + } + if (*format == '$') { + format++; + } + assert(!IS_END_OF_FORMAT(*format)); + + current_arg = NULL; + if (nkeywords && i >= pos) { + current_arg = PyDict_GetItem(keywords, keyword); + if (!current_arg && PyErr_Occurred()) { + return cleanreturn(0, &freelist); + } + } + if (current_arg) { + --nkeywords; + if (i < nargs) { + /* arg present in tuple and in dict */ + PyErr_Format(PyExc_TypeError, + "Argument given by name ('%U') " + "and position (%d)", + keyword, i+1); + return cleanreturn(0, &freelist); + } + } + else if (i < nargs) + current_arg = PyTuple_GET_ITEM(args, i); + + if (current_arg) { + msg = convertitem(current_arg, &format, p_va, flags, + levels, msgbuf, sizeof(msgbuf), &freelist); + if (msg) { + seterror(i+1, msg, levels, parser->fname, parser->custom_msg); + return cleanreturn(0, &freelist); + } + continue; + } + + if (i < parser->min) { + /* Less arguments than required */ + if (i < pos) { + PyErr_Format(PyExc_TypeError, + "Function takes %s %d positional arguments" + " (%d given)", + (Py_MIN(pos, parser->min) < parser->max) ? "at least" : "exactly", + Py_MIN(pos, parser->min), nargs); + } + else { + PyErr_Format(PyExc_TypeError, "Required argument " + "'%U' (pos %d) not found", + keyword, i+1); + } + return cleanreturn(0, &freelist); + } + /* current code reports success when all required args + * fulfilled and no keyword args left, with no further + * validation. XXX Maybe skip this in debug build ? + */ + if (!nkeywords) { + return cleanreturn(1, &freelist); + } + + /* We are into optional args, skip thru to any remaining + * keyword args */ + msg = skipitem(&format, p_va, flags); + assert(msg == NULL); + } + + assert(IS_END_OF_FORMAT(*format) || (*format == '|') || (*format == '$')); + + /* make sure there are no extraneous keyword arguments */ + if (nkeywords > 0) { + PyObject *key, *value; + Py_ssize_t pos = 0; + while (PyDict_Next(keywords, &pos, &key, &value)) { + int match; + if (!PyUnicode_Check(key)) { + PyErr_SetString(PyExc_TypeError, + "keywords must be strings"); + return cleanreturn(0, &freelist); + } + match = PySequence_Contains(kwtuple, key); + if (match <= 0) { + if (!match) { + PyErr_Format(PyExc_TypeError, + "'%U' is an invalid keyword " + "argument for this function", + key); + } + return cleanreturn(0, &freelist); + } + } + } + + return cleanreturn(1, &freelist); +} + + static const char * skipitem(const char **p_format, va_list *p_va, int flags) { @@ -1705,7 +2103,9 @@ case 'Y': /* string object */ case 'U': /* unicode string object */ { - (void) va_arg(*p_va, void *); + if (p_va != NULL) { + (void) va_arg(*p_va, void *); + } break; } @@ -1713,7 +2113,9 @@ case 'e': /* string with encoding */ { - (void) va_arg(*p_va, const char *); + if (p_va != NULL) { + (void) va_arg(*p_va, const char *); + } if (!(*format == 's' || *format == 't')) /* after 'e', only 's' and 't' is allowed */ goto err; @@ -1728,12 +2130,16 @@ case 'Z': /* unicode string or None */ case 'w': /* buffer, read-write */ { - (void) va_arg(*p_va, char **); + if (p_va != NULL) { + (void) va_arg(*p_va, char **); + } if (*format == '#') { - if (flags & FLAG_SIZE_T) - (void) va_arg(*p_va, Py_ssize_t *); - else - (void) va_arg(*p_va, int *); + if (p_va != NULL) { + if (flags & FLAG_SIZE_T) + (void) va_arg(*p_va, Py_ssize_t *); + else + (void) va_arg(*p_va, int *); + } format++; } else if ((c == 's' || c == 'z' || c == 'y') && *format == '*') { format++; @@ -1745,17 +2151,23 @@ { if (*format == '!') { format++; - (void) va_arg(*p_va, PyTypeObject*); - (void) va_arg(*p_va, PyObject **); + if (p_va != NULL) { + (void) va_arg(*p_va, PyTypeObject*); + (void) va_arg(*p_va, PyObject **); + } } else if (*format == '&') { typedef int (*converter)(PyObject *, void *); - (void) va_arg(*p_va, converter); - (void) va_arg(*p_va, void *); + if (p_va != NULL) { + (void) va_arg(*p_va, converter); + (void) va_arg(*p_va, void *); + } format++; } else { - (void) va_arg(*p_va, PyObject **); + if (p_va != NULL) { + (void) va_arg(*p_va, PyObject **); + } } break; } @@ -1891,6 +2303,19 @@ return 0; } +void +_PyArg_Fini(void) +{ + struct _PyArg_Parser *tmp, *s = static_arg_parsers; + while (s) { + tmp = s->next; + s->next = NULL; + parser_clear(s); + s = tmp; + } + static_arg_parsers = NULL; +} + #ifdef __cplusplus }; #endif diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -674,6 +674,7 @@ PySlice_Fini(); _PyGC_Fini(); _PyRandom_Fini(); + _PyArg_Fini(); /* Cleanup Unicode implementation */ _PyUnicode_Fini(); diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -743,7 +743,10 @@ return output() def insert_keywords(s): - return linear_format(s, declarations="static char *_keywords[] = {{{keywords}, NULL}};\n{declarations}") + return linear_format(s, declarations= + 'static const char * const _keywords[] = {{{keywords}, NULL}};\n' + 'static _PyArg_Parser _parser = {{"{format_units}:{name}", _keywords, 0}};\n' + '{declarations}') if not parameters: # no parameters, METH_NOARGS @@ -849,17 +852,12 @@ parser_prototype = parser_prototype_keyword body = normalize_snippet(""" - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "{format_units}:{name}", _keywords, + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, {parse_arguments})) {{ goto exit; }} """, indent=4) - parser_definition = parser_body(parser_prototype, normalize_snippet(""" - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "{format_units}:{name}", _keywords, - {parse_arguments})) {{ - goto exit; - }} - """, indent=4)) + parser_definition = parser_body(parser_prototype, body) parser_definition = insert_keywords(parser_definition) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 14 11:27:20 2016 From: python-checkins at python.org (steven.daprano) Date: Sun, 14 Aug 2016 15:27:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue6422_add_autorange_me?= =?utf-8?q?thod_to_timeit=2ETimer?= Message-ID: <20160814152719.68472.8937.0BCE42E5@psf.io> https://hg.python.org/cpython/rev/424eb46f7f3a changeset: 102651:424eb46f7f3a user: Steven D'Aprano date: Mon Aug 15 01:27:03 2016 +1000 summary: Issue6422 add autorange method to timeit.Timer files: Doc/library/timeit.rst | 19 ++++++++++++- Lib/test/test_timeit.py | 22 +++++++++++++++ Lib/timeit.py | 41 +++++++++++++++++++++------- 3 files changed, 69 insertions(+), 13 deletions(-) diff --git a/Doc/library/timeit.rst b/Doc/library/timeit.rst --- a/Doc/library/timeit.rst +++ b/Doc/library/timeit.rst @@ -100,8 +100,8 @@ can be controlled by passing a namespace to *globals*. To measure the execution time of the first statement, use the :meth:`.timeit` - method. The :meth:`.repeat` method is a convenience to call :meth:`.timeit` - multiple times and return a list of results. + method. The :meth:`.repeat` and :meth:`.autorange` methods are convenience + methods to call :meth:`.timeit` multiple times. The execution time of *setup* is excluded from the overall timed execution run. @@ -134,6 +134,21 @@ timeit.Timer('for i in range(10): oct(i)', 'gc.enable()').timeit() + .. method:: Timer.autorange(callback=None) + + Automatically determine how many times to call :meth:`.timeit`. + + This is a convenience function that calls :meth:`.timeit` repeatedly + so that the total time >= 0.2 second, returning the eventual + (number of loops, time taken for that number of loops). It calls + :meth:`.timeit` with *number* set to successive powers of ten (10, + 100, 1000, ...) up to a maximum of one billion, until the time taken + is at least 0.2 second, or the maximum is reached. + + If *callback* is given and is not *None*, it will be called after + each trial with two arguments: ``callback(number, time_taken)``. + + .. method:: Timer.repeat(repeat=3, number=1000000) Call :meth:`.timeit` a few times. diff --git a/Lib/test/test_timeit.py b/Lib/test/test_timeit.py --- a/Lib/test/test_timeit.py +++ b/Lib/test/test_timeit.py @@ -354,6 +354,28 @@ s = self.run_main(switches=['-n1', '1/0']) self.assert_exc_string(error_stringio.getvalue(), 'ZeroDivisionError') + def autorange(self, callback=None): + timer = FakeTimer(seconds_per_increment=0.001) + t = timeit.Timer(stmt=self.fake_stmt, setup=self.fake_setup, timer=timer) + return t.autorange(callback) + + def test_autorange(self): + num_loops, time_taken = self.autorange() + self.assertEqual(num_loops, 1000) + self.assertEqual(time_taken, 1.0) + + def test_autorange_with_callback(self): + def callback(a, b): + print("{} {:.3f}".format(a, b)) + with captured_stdout() as s: + num_loops, time_taken = self.autorange(callback) + self.assertEqual(num_loops, 1000) + self.assertEqual(time_taken, 1.0) + expected = ('10 0.010\n' + '100 0.100\n' + '1000 1.000\n') + self.assertEqual(s.getvalue(), expected) + if __name__ == '__main__': unittest.main() diff --git a/Lib/timeit.py b/Lib/timeit.py --- a/Lib/timeit.py +++ b/Lib/timeit.py @@ -207,6 +207,26 @@ r.append(t) return r + def autorange(self, callback=None): + """Return the number of loops so that total time >= 0.2. + + Calls the timeit method with *number* set to successive powers of + ten (10, 100, 1000, ...) up to a maximum of one billion, until + the time taken is at least 0.2 second, or the maximum is reached. + Returns ``(number, time_taken)``. + + If *callback* is given and is not None, it will be called after + each trial with two arguments: ``callback(number, time_taken)``. + """ + for i in range(1, 10): + number = 10**i + time_taken = self.timeit(number) + if callback: + callback(number, time_taken) + if time_taken >= 0.2: + break + return (number, time_taken) + def timeit(stmt="pass", setup="pass", timer=default_timer, number=default_number, globals=None): """Convenience function to create Timer object and call timeit method.""" @@ -295,17 +315,16 @@ t = Timer(stmt, setup, timer) if number == 0: # determine number so that 0.2 <= total time < 2.0 - for i in range(1, 10): - number = 10**i - try: - x = t.timeit(number) - except: - t.print_exc() - return 1 - if verbose: - print("%d loops -> %.*g secs" % (number, precision, x)) - if x >= 0.2: - break + callback = None + if verbose: + def callback(number, time_taken): + msg = "{num} loops -> {secs:.{prec}g} secs" + print(msg.format(num=number, secs=time_taken, prec=precision)) + try: + number, _ = t.autorange(callback) + except: + t.print_exc() + return 1 try: r = t.repeat(repeat, number) except: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 14 14:17:45 2016 From: python-checkins at python.org (steven.daprano) Date: Sun, 14 Aug 2016 18:17:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue27573_code=2Einteract?= =?utf-8?q?_prints_a_message_when_exiting=2E?= Message-ID: <20160814181745.609.83678.F779633A@psf.io> https://hg.python.org/cpython/rev/9410dc027505 changeset: 102653:9410dc027505 user: Steven D'Aprano date: Mon Aug 15 04:14:33 2016 +1000 summary: Issue27573 code.interact prints a message when exiting. files: Doc/library/code.rst | 3 +++ Lib/code.py | 1 + Lib/test/test_code_module.py | 12 ++++++++++-- 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/Doc/library/code.rst b/Doc/library/code.rst --- a/Doc/library/code.rst +++ b/Doc/library/code.rst @@ -147,6 +147,9 @@ .. versionchanged:: 3.4 To suppress printing any banner, pass an empty string. + .. versionchanged:: 3.6 + Now prints a brief message when exiting. + .. method:: InteractiveConsole.push(line) diff --git a/Lib/code.py b/Lib/code.py --- a/Lib/code.py +++ b/Lib/code.py @@ -230,6 +230,7 @@ self.write("\nKeyboardInterrupt\n") self.resetbuffer() more = 0 + self.write('now exiting %s...\n' % self.__class__.__name__) def push(self, line): """Push a line to the interpreter. diff --git a/Lib/test/test_code_module.py b/Lib/test/test_code_module.py --- a/Lib/test/test_code_module.py +++ b/Lib/test/test_code_module.py @@ -69,7 +69,7 @@ # with banner self.infunc.side_effect = EOFError('Finished') self.console.interact(banner='Foo') - self.assertEqual(len(self.stderr.method_calls), 2) + self.assertEqual(len(self.stderr.method_calls), 3) banner_call = self.stderr.method_calls[0] self.assertEqual(banner_call, ['write', ('Foo\n',), {}]) @@ -77,7 +77,15 @@ self.stderr.reset_mock() self.infunc.side_effect = EOFError('Finished') self.console.interact(banner='') - self.assertEqual(len(self.stderr.method_calls), 1) + self.assertEqual(len(self.stderr.method_calls), 2) + + def test_exit_msg(self): + self.infunc.side_effect = EOFError('Finished') + self.console.interact(banner='') + self.assertEqual(len(self.stderr.method_calls), 2) + err_msg = self.stderr.method_calls[1] + expected = 'now exiting InteractiveConsole...\n' + self.assertEqual(err_msg, ['write', (expected,), {}]) def test_cause_tb(self): self.infunc.side_effect = ["raise ValueError('') from AttributeError", -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 14 14:17:46 2016 From: python-checkins at python.org (steven.daprano) Date: Sun, 14 Aug 2016 18:17:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_versionadded_tag_to_do?= =?utf-8?q?cs_for_timeit=2Eautorange?= Message-ID: <20160814181745.67900.9992.ADAB47A4@psf.io> https://hg.python.org/cpython/rev/40b13da7f0f8 changeset: 102652:40b13da7f0f8 user: Steven D'Aprano date: Mon Aug 15 02:47:49 2016 +1000 summary: Add versionadded tag to docs for timeit.autorange files: Doc/library/timeit.rst | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Doc/library/timeit.rst b/Doc/library/timeit.rst --- a/Doc/library/timeit.rst +++ b/Doc/library/timeit.rst @@ -148,6 +148,8 @@ If *callback* is given and is not *None*, it will be called after each trial with two arguments: ``callback(number, time_taken)``. + .. versionadded:: 3.6 + .. method:: Timer.repeat(repeat=3, number=1000000) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 14 16:10:42 2016 From: python-checkins at python.org (donald.stufft) Date: Sun, 14 Aug 2016 20:10:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Upgrade_setupt?= =?utf-8?q?ools/pip_to_to_25=2E2=2E0/8=2E1=2E2?= Message-ID: <20160814201042.58504.54907.E5486628@psf.io> https://hg.python.org/cpython/rev/d568f66fa26c changeset: 102654:d568f66fa26c branch: 2.7 parent: 102645:af42635b5ed1 user: Donald Stufft date: Sun Aug 14 16:08:42 2016 -0400 summary: Upgrade setuptools/pip to to 25.2.0/8.1.2 files: Lib/ensurepip/__init__.py | 4 ++-- Lib/ensurepip/_bundled/pip-8.1.1-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl | Bin 5 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -12,9 +12,9 @@ __all__ = ["version", "bootstrap"] -_SETUPTOOLS_VERSION = "20.10.1" +_SETUPTOOLS_VERSION = "25.2.0" -_PIP_VERSION = "8.1.1" +_PIP_VERSION = "8.1.2" # pip currently requires ssl support, so we try to provide a nicer # error message when that is missing (http://bugs.python.org/issue19744) diff --git a/Lib/ensurepip/_bundled/pip-8.1.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.1.1-py2.py3-none-any.whl deleted file mode 100644 index 8632eb7af04c6337f0442a878ecb99cd2b1a67e0..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..cc49227a0c7e13757f4863a9b7ace1eb56c3ce61 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl deleted file mode 100644 index 9d1319a24aba103fe956ef6298e3649efacc0b93..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..02c8ce8737bea57315cfff3b6b3b1a49eadacc6e GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 14 16:10:42 2016 From: python-checkins at python.org (donald.stufft) Date: Sun, 14 Aug 2016 20:10:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Update_setupto?= =?utf-8?q?ols/pip_to_25=2E2=2E0/8=2E1=2E2?= Message-ID: <20160814201042.14112.20222.06FF3CDF@psf.io> https://hg.python.org/cpython/rev/dc1bacd79e66 changeset: 102655:dc1bacd79e66 branch: 3.4 parent: 102647:9822bf4bcece user: Donald Stufft date: Sun Aug 14 16:09:56 2016 -0400 summary: Update setuptools/pip to 25.2.0/8.1.2 files: Lib/ensurepip/__init__.py | 4 ++-- Lib/ensurepip/_bundled/pip-8.1.1-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl | Bin 5 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -8,9 +8,9 @@ __all__ = ["version", "bootstrap"] -_SETUPTOOLS_VERSION = "20.10.1" +_SETUPTOOLS_VERSION = "25.2.0" -_PIP_VERSION = "8.1.1" +_PIP_VERSION = "8.1.2" # pip currently requires ssl support, so we try to provide a nicer # error message when that is missing (http://bugs.python.org/issue19744) diff --git a/Lib/ensurepip/_bundled/pip-8.1.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.1.1-py2.py3-none-any.whl deleted file mode 100644 index 8632eb7af04c6337f0442a878ecb99cd2b1a67e0..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..cc49227a0c7e13757f4863a9b7ace1eb56c3ce61 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl deleted file mode 100644 index 9d1319a24aba103fe956ef6298e3649efacc0b93..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..02c8ce8737bea57315cfff3b6b3b1a49eadacc6e GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 14 16:10:44 2016 From: python-checkins at python.org (donald.stufft) Date: Sun, 14 Aug 2016 20:10:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160814201044.68488.90628.4F254D32@psf.io> https://hg.python.org/cpython/rev/54c8d785bd39 changeset: 102657:54c8d785bd39 parent: 102653:9410dc027505 parent: 102656:2aa2fbb681d8 user: Donald Stufft date: Sun Aug 14 16:10:31 2016 -0400 summary: merge 3.5 files: Lib/ensurepip/__init__.py | 4 ++-- Lib/ensurepip/_bundled/pip-8.1.1-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl | Bin 5 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -8,9 +8,9 @@ __all__ = ["version", "bootstrap"] -_SETUPTOOLS_VERSION = "20.10.1" +_SETUPTOOLS_VERSION = "25.2.0" -_PIP_VERSION = "8.1.1" +_PIP_VERSION = "8.1.2" # pip currently requires ssl support, so we try to provide a nicer # error message when that is missing (http://bugs.python.org/issue19744) diff --git a/Lib/ensurepip/_bundled/pip-8.1.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.1.1-py2.py3-none-any.whl deleted file mode 100644 index 8632eb7af04c6337f0442a878ecb99cd2b1a67e0..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..cc49227a0c7e13757f4863a9b7ace1eb56c3ce61 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl deleted file mode 100644 index 9d1319a24aba103fe956ef6298e3649efacc0b93..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..02c8ce8737bea57315cfff3b6b3b1a49eadacc6e GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 14 16:10:44 2016 From: python-checkins at python.org (donald.stufft) Date: Sun, 14 Aug 2016 20:10:44 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4?= Message-ID: <20160814201044.68488.56896.798734A9@psf.io> https://hg.python.org/cpython/rev/2aa2fbb681d8 changeset: 102656:2aa2fbb681d8 branch: 3.5 parent: 102648:a277ab6bf66b parent: 102655:dc1bacd79e66 user: Donald Stufft date: Sun Aug 14 16:10:17 2016 -0400 summary: merge 3.4 files: Lib/ensurepip/__init__.py | 4 ++-- Lib/ensurepip/_bundled/pip-8.1.1-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl | Bin 5 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -8,9 +8,9 @@ __all__ = ["version", "bootstrap"] -_SETUPTOOLS_VERSION = "20.10.1" +_SETUPTOOLS_VERSION = "25.2.0" -_PIP_VERSION = "8.1.1" +_PIP_VERSION = "8.1.2" # pip currently requires ssl support, so we try to provide a nicer # error message when that is missing (http://bugs.python.org/issue19744) diff --git a/Lib/ensurepip/_bundled/pip-8.1.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.1.1-py2.py3-none-any.whl deleted file mode 100644 index 8632eb7af04c6337f0442a878ecb99cd2b1a67e0..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..cc49227a0c7e13757f4863a9b7ace1eb56c3ce61 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl deleted file mode 100644 index 9d1319a24aba103fe956ef6298e3649efacc0b93..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..02c8ce8737bea57315cfff3b6b3b1a49eadacc6e GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 14 21:21:28 2016 From: python-checkins at python.org (steven.daprano) Date: Mon, 15 Aug 2016 01:21:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Update_Misc/NEWS=2E?= Message-ID: <20160815012127.58504.62098.116CFEB2@psf.io> https://hg.python.org/cpython/rev/4b3899e734d5 changeset: 102658:4b3899e734d5 user: Steven D'Aprano date: Mon Aug 15 11:21:08 2016 +1000 summary: Update Misc/NEWS. files: Misc/NEWS | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -49,6 +49,12 @@ - Issue #27664: Add to concurrent.futures.thread.ThreadPoolExecutor() the ability to specify a thread name prefix. +- Issue #27181: Add geometric_mean and harmonic_mean to statistics module. + +- Issue #27573: code.interact now prints an message when exiting. + +- Issue #6422: Add autorange method to timeit.Timer objects. + - Issue #26750: unittest.mock.create_autospec() now works properly for subclasses of property() and other data descriptors. Removes the never publicly used, never documented unittest.mock.DescriptorTypes tuple. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 14 23:13:04 2016 From: python-checkins at python.org (nick.coghlan) Date: Mon, 15 Aug 2016 03:13:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326823=3A_Abbrevia?= =?utf-8?q?te_recursive_tracebacks?= Message-ID: <20160815031304.68654.71151.AA0D08A3@psf.io> https://hg.python.org/cpython/rev/5a2ff215e841 changeset: 102659:5a2ff215e841 user: Nick Coghlan date: Mon Aug 15 13:11:34 2016 +1000 summary: Issue #26823: Abbreviate recursive tracebacks Large sections of repeated lines in tracebacks are now abbreviated as "[Previous line repeated {count} more times]" by both the traceback module and the builtin traceback rendering. Patch by Emanuel Barry. files: Doc/library/traceback.rst | 15 ++ Doc/whatsnew/3.6.rst | 12 ++ Lib/test/test_traceback.py | 131 +++++++++++++++++++++++++ Lib/traceback.py | 23 ++++ Misc/NEWS | 9 + Python/traceback.c | 36 ++++++- 6 files changed, 222 insertions(+), 4 deletions(-) diff --git a/Doc/library/traceback.rst b/Doc/library/traceback.rst --- a/Doc/library/traceback.rst +++ b/Doc/library/traceback.rst @@ -291,6 +291,21 @@ of tuples. Each tuple should be a 4-tuple with filename, lineno, name, line as the elements. + .. method:: format() + + Returns a list of strings ready for printing. Each string in the + resulting list corresponds to a single frame from the stack. + Each string ends in a newline; the strings may contain internal + newlines as well, for those items with source text lines. + + For long sequences of the same frame and line, the first few + repetitions are shown, followed by a summary line stating the exact + number of further repetitions. + + .. versionchanged:: 3.6 + + Long sequences of repeated frames are now abbreviated. + :class:`FrameSummary` Objects ----------------------------- diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -438,6 +438,14 @@ (Contributed by Serhiy Storchaka in :issue:`22115`). +traceback +--------- + +The :meth:`~traceback.StackSummary.format` method now abbreviates long sequences +of repeated lines as ``"[Previous line repeated {count} more times]"``. +(Contributed by Emanuel Barry in :issue:`26823`.) + + typing ------ @@ -597,6 +605,10 @@ defined by empty names. (Contributed by Serhiy Storchaka in :issue:`26282`). +* ``PyTraceback_Print`` method now abbreviates long sequences of repeated lines + as ``"[Previous line repeated {count} more times]"``. + (Contributed by Emanuel Barry in :issue:`26823`.) + Deprecated ========== diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -303,6 +303,137 @@ ' traceback.print_stack()', ]) + # issue 26823 - Shrink recursive tracebacks + def _check_recursive_traceback_display(self, render_exc): + # Always show full diffs when this test fails + # Note that rearranging things may require adjusting + # the relative line numbers in the expected tracebacks + self.maxDiff = None + + # Check hitting the recursion limit + def f(): + f() + + with captured_output("stderr") as stderr_f: + try: + f() + except RecursionError as exc: + render_exc() + else: + self.fail("no recursion occurred") + + lineno_f = f.__code__.co_firstlineno + result_f = ( + 'Traceback (most recent call last):\n' + f' File "{__file__}", line {lineno_f+5}, in _check_recursive_traceback_display\n' + ' f()\n' + f' File "{__file__}", line {lineno_f+1}, in f\n' + ' f()\n' + f' File "{__file__}", line {lineno_f+1}, in f\n' + ' f()\n' + f' File "{__file__}", line {lineno_f+1}, in f\n' + ' f()\n' + # XXX: The following line changes depending on whether the tests + # are run through the interactive interpreter or with -m + # It also varies depending on the platform (stack size) + # Fortunately, we don't care about exactness here, so we use regex + r' \[Previous line repeated (\d+) more times\]' '\n' + 'RecursionError: maximum recursion depth exceeded\n' + ) + + expected = result_f.splitlines() + actual = stderr_f.getvalue().splitlines() + + # Check the output text matches expectations + # 2nd last line contains the repetition count + self.assertEqual(actual[:-2], expected[:-2]) + self.assertRegex(actual[-2], expected[-2]) + self.assertEqual(actual[-1], expected[-1]) + + # Check the recursion count is roughly as expected + rec_limit = sys.getrecursionlimit() + self.assertIn(int(re.search(r"\d+", actual[-2]).group()), range(rec_limit-50, rec_limit)) + + # Check a known (limited) number of recursive invocations + def g(count=10): + if count: + return g(count-1) + raise ValueError + + with captured_output("stderr") as stderr_g: + try: + g() + except ValueError as exc: + render_exc() + else: + self.fail("no value error was raised") + + lineno_g = g.__code__.co_firstlineno + result_g = ( + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' + ' return g(count-1)\n' + ' [Previous line repeated 6 more times]\n' + f' File "{__file__}", line {lineno_g+3}, in g\n' + ' raise ValueError\n' + 'ValueError\n' + ) + tb_line = ( + 'Traceback (most recent call last):\n' + f' File "{__file__}", line {lineno_g+7}, in _check_recursive_traceback_display\n' + ' g()\n' + ) + expected = (tb_line + result_g).splitlines() + actual = stderr_g.getvalue().splitlines() + self.assertEqual(actual, expected) + + # Check 2 different repetitive sections + def h(count=10): + if count: + return h(count-1) + g() + + with captured_output("stderr") as stderr_h: + try: + h() + except ValueError as exc: + render_exc() + else: + self.fail("no value error was raised") + + lineno_h = h.__code__.co_firstlineno + result_h = ( + 'Traceback (most recent call last):\n' + f' File "{__file__}", line {lineno_h+7}, in _check_recursive_traceback_display\n' + ' h()\n' + f' File "{__file__}", line {lineno_h+2}, in h\n' + ' return h(count-1)\n' + f' File "{__file__}", line {lineno_h+2}, in h\n' + ' return h(count-1)\n' + f' File "{__file__}", line {lineno_h+2}, in h\n' + ' return h(count-1)\n' + ' [Previous line repeated 6 more times]\n' + f' File "{__file__}", line {lineno_h+3}, in h\n' + ' g()\n' + ) + expected = (result_h + result_g).splitlines() + actual = stderr_h.getvalue().splitlines() + self.assertEqual(actual, expected) + + def test_recursive_traceback_python(self): + self._check_recursive_traceback_display(traceback.print_exc) + + @cpython_only + def test_recursive_traceback_cpython_internal(self): + from _testcapi import exception_print + def render_exc(): + exc_type, exc_value, exc_tb = sys.exc_info() + exception_print(exc_value) + self._check_recursive_traceback_display(render_exc) + def test_format_stack(self): def fmt(): return traceback.format_stack() diff --git a/Lib/traceback.py b/Lib/traceback.py --- a/Lib/traceback.py +++ b/Lib/traceback.py @@ -385,9 +385,30 @@ resulting list corresponds to a single frame from the stack. Each string ends in a newline; the strings may contain internal newlines as well, for those items with source text lines. + + For long sequences of the same frame and line, the first few + repetitions are shown, followed by a summary line stating the exact + number of further repetitions. """ result = [] + last_file = None + last_line = None + last_name = None + count = 0 for frame in self: + if (last_file is not None and last_file == frame.filename and + last_line is not None and last_line == frame.lineno and + last_name is not None and last_name == frame.name): + count += 1 + else: + if count > 3: + result.append(f' [Previous line repeated {count-3} more times]\n') + last_file = frame.filename + last_line = frame.lineno + last_name = frame.name + count = 0 + if count >= 3: + continue row = [] row.append(' File "{}", line {}, in {}\n'.format( frame.filename, frame.lineno, frame.name)) @@ -397,6 +418,8 @@ for name, value in sorted(frame.locals.items()): row.append(' {name} = {value}\n'.format(name=name, value=value)) result.append(''.join(row)) + if count > 3: + result.append(f' [Previous line repeated {count-3} more times]\n') return result diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #26823: Large sections of repeated lines in tracebacks are now + abbreviated as "[Previous line repeated {count} more times]" by the builtin + traceback rendering. Patch by Emanuel Barry. + - Issue #27574: Decreased an overhead of parsing keyword arguments in functions implemented with using Argument Clinic. @@ -46,6 +50,11 @@ Library ------- +- Issue #26823: traceback.StackSummary.format now abbreviates large sections of + repeated lines as "[Previous line repeated {count} more times]" (this change + then further affects other traceback display operations in the module). Patch + by Emanuel Barry. + - Issue #27664: Add to concurrent.futures.thread.ThreadPoolExecutor() the ability to specify a thread name prefix. diff --git a/Python/traceback.c b/Python/traceback.c --- a/Python/traceback.c +++ b/Python/traceback.c @@ -412,6 +412,11 @@ { int err = 0; long depth = 0; + PyObject *last_file = NULL; + int last_line = -1; + PyObject *last_name = NULL; + long cnt = 0; + PyObject *line; PyTracebackObject *tb1 = tb; while (tb1 != NULL) { depth++; @@ -419,16 +424,39 @@ } while (tb != NULL && err == 0) { if (depth <= limit) { - err = tb_displayline(f, - tb->tb_frame->f_code->co_filename, - tb->tb_lineno, - tb->tb_frame->f_code->co_name); + if (last_file != NULL && + tb->tb_frame->f_code->co_filename == last_file && + last_line != -1 && tb->tb_lineno == last_line && + last_name != NULL && + tb->tb_frame->f_code->co_name == last_name) { + cnt++; + } else { + if (cnt > 3) { + line = PyUnicode_FromFormat( + " [Previous line repeated %d more times]\n", cnt-3); + err = PyFile_WriteObject(line, f, Py_PRINT_RAW); + } + last_file = tb->tb_frame->f_code->co_filename; + last_line = tb->tb_lineno; + last_name = tb->tb_frame->f_code->co_name; + cnt = 0; + } + if (cnt < 3) + err = tb_displayline(f, + tb->tb_frame->f_code->co_filename, + tb->tb_lineno, + tb->tb_frame->f_code->co_name); } depth--; tb = tb->tb_next; if (err == 0) err = PyErr_CheckSignals(); } + if (cnt > 3) { + line = PyUnicode_FromFormat( + " [Previous line repeated %d more times]\n", cnt-3); + err = PyFile_WriteObject(line, f, Py_PRINT_RAW); + } return err; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 02:46:26 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 15 Aug 2016 06:46:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327704=3A_Optimize?= =?utf-8?q?d_creating_bytes_and_bytearray_from_byte-like_objects?= Message-ID: <20160815064625.76003.73702.BA4DA96E@psf.io> https://hg.python.org/cpython/rev/789a42401009 changeset: 102660:789a42401009 user: Serhiy Storchaka date: Mon Aug 15 09:46:07 2016 +0300 summary: Issue #27704: Optimized creating bytes and bytearray from byte-like objects and iterables. Speed up to 3 times for short objects. Original patch by Naoki Inada. files: Misc/NEWS | 4 ++++ Objects/bytearrayobject.c | 18 ++++++++---------- Objects/bytesobject.c | 18 ++++++++---------- 3 files changed, 20 insertions(+), 20 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #27704: Optimized creating bytes and bytearray from byte-like objects + and iterables. Speed up to 3 times for short objects. Original patch by + Naoki Inada. + - Issue #26823: Large sections of repeated lines in tracebacks are now abbreviated as "[Previous line repeated {count} more times]" by the builtin traceback rendering. Patch by Emanuel Barry. diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c --- a/Objects/bytearrayobject.c +++ b/Objects/bytearrayobject.c @@ -795,17 +795,15 @@ } /* Is it an int? */ - count = PyNumber_AsSsize_t(arg, PyExc_OverflowError); - if (count == -1 && PyErr_Occurred()) { - if (PyErr_ExceptionMatches(PyExc_OverflowError)) + if (PyIndex_Check(arg)) { + count = PyNumber_AsSsize_t(arg, PyExc_OverflowError); + if (count == -1 && PyErr_Occurred()) { return -1; - PyErr_Clear(); - } - else if (count < 0) { - PyErr_SetString(PyExc_ValueError, "negative count"); - return -1; - } - else { + } + if (count < 0) { + PyErr_SetString(PyExc_ValueError, "negative count"); + return -1; + } if (count > 0) { if (PyByteArray_Resize((PyObject *)self, count)) return -1; diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -2563,17 +2563,15 @@ return NULL; } /* Is it an integer? */ - size = PyNumber_AsSsize_t(x, PyExc_OverflowError); - if (size == -1 && PyErr_Occurred()) { - if (PyErr_ExceptionMatches(PyExc_OverflowError)) + if (PyIndex_Check(x)) { + size = PyNumber_AsSsize_t(x, PyExc_OverflowError); + if (size == -1 && PyErr_Occurred()) { return NULL; - PyErr_Clear(); - } - else if (size < 0) { - PyErr_SetString(PyExc_ValueError, "negative count"); - return NULL; - } - else { + } + if (size < 0) { + PyErr_SetString(PyExc_ValueError, "negative count"); + return NULL; + } new = _PyBytes_FromSize(size, 1); if (new == NULL) return NULL; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 03:06:32 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 15 Aug 2016 07:06:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2316764=3A_Support_?= =?utf-8?q?keyword_arguments_to_zlib=2Edecompress=28=29=2E__Patch_by?= Message-ID: <20160815070632.76210.92195.AA01E0B8@psf.io> https://hg.python.org/cpython/rev/a4101218364e changeset: 102661:a4101218364e user: Serhiy Storchaka date: Mon Aug 15 10:06:16 2016 +0300 summary: Issue #16764: Support keyword arguments to zlib.decompress(). Patch by Xiang Zhang. files: Doc/library/zlib.rst | 20 ++++++++----- Lib/test/test_zlib.py | 33 ++++++++++++++++++++-- Misc/NEWS | 3 ++ Modules/clinic/zlibmodule.c.h | 22 +++++++++------ Modules/zlibmodule.c | 8 ++-- 5 files changed, 61 insertions(+), 25 deletions(-) diff --git a/Doc/library/zlib.rst b/Doc/library/zlib.rst --- a/Doc/library/zlib.rst +++ b/Doc/library/zlib.rst @@ -129,7 +129,7 @@ platforms, use ``crc32(data) & 0xffffffff``. -.. function:: decompress(data[, wbits[, bufsize]]) +.. function:: decompress(data, wbits=MAX_WBITS, bufsize=DEF_BUF_SIZE) Decompresses the bytes in *data*, returning a bytes object containing the uncompressed data. The *wbits* parameter depends on @@ -164,14 +164,16 @@ When decompressing a stream, the window size must not be smaller than the size originally used to compress the stream; using a too-small value may result in an :exc:`error` exception. The default *wbits* value - is 15, which corresponds to the largest window size and requires a zlib - header and trailer to be included. + corresponds to the largest window size and requires a zlib header and + trailer to be included. *bufsize* is the initial size of the buffer used to hold decompressed data. If more space is required, the buffer size will be increased as needed, so you don't have to get this value exactly right; tuning it will only save a few calls - to :c:func:`malloc`. The default size is 16384. + to :c:func:`malloc`. + .. versionchanged:: 3.6 + *wbits* and *bufsize* can be used as keyword arguments. .. function:: decompressobj(wbits=15[, zdict]) @@ -257,7 +259,7 @@ .. versionadded:: 3.3 -.. method:: Decompress.decompress(data[, max_length]) +.. method:: Decompress.decompress(data, max_length=0) Decompress *data*, returning a bytes object containing the uncompressed data corresponding to at least part of the data in *string*. This data should be @@ -269,9 +271,11 @@ no longer than *max_length*. This may mean that not all of the compressed input can be processed; and unconsumed data will be stored in the attribute :attr:`unconsumed_tail`. This bytestring must be passed to a subsequent call to - :meth:`decompress` if decompression is to continue. If *max_length* is not - supplied then the whole input is decompressed, and :attr:`unconsumed_tail` is - empty. + :meth:`decompress` if decompression is to continue. If *max_length* is zero + then the whole input is decompressed, and :attr:`unconsumed_tail` is empty. + + .. versionchanged:: 3.6 + *max_length* can be used as a keyword argument. .. method:: Decompress.flush([length]) diff --git a/Lib/test/test_zlib.py b/Lib/test/test_zlib.py --- a/Lib/test/test_zlib.py +++ b/Lib/test/test_zlib.py @@ -169,6 +169,14 @@ self.assertEqual(zlib.decompress(x), HAMLET_SCENE) with self.assertRaises(TypeError): zlib.compress(data=HAMLET_SCENE, level=3) + self.assertEqual(zlib.decompress(x, + wbits=zlib.MAX_WBITS, + bufsize=zlib.DEF_BUF_SIZE), + HAMLET_SCENE) + with self.assertRaises(TypeError): + zlib.decompress(data=x, + wbits=zlib.MAX_WBITS, + bufsize=zlib.DEF_BUF_SIZE) def test_speech128(self): # compress more data @@ -240,6 +248,27 @@ self.assertIsInstance(dco.unconsumed_tail, bytes) self.assertIsInstance(dco.unused_data, bytes) + def test_keywords(self): + level = 2 + method = zlib.DEFLATED + wbits = -12 + memLevel = 9 + strategy = zlib.Z_FILTERED + co = zlib.compressobj(level=level, + method=method, + wbits=wbits, + memLevel=memLevel, + strategy=strategy, + zdict=b"") + do = zlib.decompressobj(wbits=wbits, zdict=b"") + with self.assertRaises(TypeError): + co.compress(data=HAMLET_SCENE) + with self.assertRaises(TypeError): + do.decompress(data=zlib.compress(HAMLET_SCENE)) + x = co.compress(HAMLET_SCENE) + co.flush() + y = do.decompress(x, max_length=len(HAMLET_SCENE)) + do.flush() + self.assertEqual(HAMLET_SCENE, y) + def test_compressoptions(self): # specify lots of options to compressobj() level = 2 @@ -255,10 +284,6 @@ y2 = dco.flush() self.assertEqual(HAMLET_SCENE, y1 + y2) - # keyword arguments should also be supported - zlib.compressobj(level=level, method=method, wbits=wbits, - memLevel=memLevel, strategy=strategy, zdict=b"") - def test_compressincremental(self): # compress object in steps, decompress object as one-shot data = HAMLET_SCENE * 128 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #16764: Support keyword arguments to zlib.decompress(). Patch by + Xiang Zhang. + - Issue #27704: Optimized creating bytes and bytearray from byte-like objects and iterables. Speed up to 3 times for short objects. Original patch by Naoki Inada. diff --git a/Modules/clinic/zlibmodule.c.h b/Modules/clinic/zlibmodule.c.h --- a/Modules/clinic/zlibmodule.c.h +++ b/Modules/clinic/zlibmodule.c.h @@ -44,7 +44,7 @@ } PyDoc_STRVAR(zlib_decompress__doc__, -"decompress($module, data, wbits=MAX_WBITS, bufsize=DEF_BUF_SIZE, /)\n" +"decompress($module, data, /, wbits=MAX_WBITS, bufsize=DEF_BUF_SIZE)\n" "--\n" "\n" "Returns a bytes object containing the uncompressed data.\n" @@ -57,21 +57,23 @@ " The initial output buffer size."); #define ZLIB_DECOMPRESS_METHODDEF \ - {"decompress", (PyCFunction)zlib_decompress, METH_VARARGS, zlib_decompress__doc__}, + {"decompress", (PyCFunction)zlib_decompress, METH_VARARGS|METH_KEYWORDS, zlib_decompress__doc__}, static PyObject * zlib_decompress_impl(PyObject *module, Py_buffer *data, int wbits, Py_ssize_t bufsize); static PyObject * -zlib_decompress(PyObject *module, PyObject *args) +zlib_decompress(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; + static const char * const _keywords[] = {"", "wbits", "bufsize", NULL}; + static _PyArg_Parser _parser = {"y*|iO&:decompress", _keywords, 0}; Py_buffer data = {NULL, NULL}; int wbits = MAX_WBITS; Py_ssize_t bufsize = DEF_BUF_SIZE; - if (!PyArg_ParseTuple(args, "y*|iO&:decompress", + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &data, &wbits, ssize_t_converter, &bufsize)) { goto exit; } @@ -228,7 +230,7 @@ } PyDoc_STRVAR(zlib_Decompress_decompress__doc__, -"decompress($self, data, max_length=0, /)\n" +"decompress($self, data, /, max_length=0)\n" "--\n" "\n" "Return a bytes object containing the decompressed version of the data.\n" @@ -245,20 +247,22 @@ "Call the flush() method to clear these buffers."); #define ZLIB_DECOMPRESS_DECOMPRESS_METHODDEF \ - {"decompress", (PyCFunction)zlib_Decompress_decompress, METH_VARARGS, zlib_Decompress_decompress__doc__}, + {"decompress", (PyCFunction)zlib_Decompress_decompress, METH_VARARGS|METH_KEYWORDS, zlib_Decompress_decompress__doc__}, static PyObject * zlib_Decompress_decompress_impl(compobject *self, Py_buffer *data, Py_ssize_t max_length); static PyObject * -zlib_Decompress_decompress(compobject *self, PyObject *args) +zlib_Decompress_decompress(compobject *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; + static const char * const _keywords[] = {"", "max_length", NULL}; + static _PyArg_Parser _parser = {"y*|O&:decompress", _keywords, 0}; Py_buffer data = {NULL, NULL}; Py_ssize_t max_length = 0; - if (!PyArg_ParseTuple(args, "y*|O&:decompress", + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &data, ssize_t_converter, &max_length)) { goto exit; } @@ -463,4 +467,4 @@ #ifndef ZLIB_COMPRESS_COPY_METHODDEF #define ZLIB_COMPRESS_COPY_METHODDEF #endif /* !defined(ZLIB_COMPRESS_COPY_METHODDEF) */ -/*[clinic end generated code: output=1fed251c15a9bffa input=a9049054013a1b77]*/ +/*[clinic end generated code: output=48911ef429b65903 input=a9049054013a1b77]*/ diff --git a/Modules/zlibmodule.c b/Modules/zlibmodule.c --- a/Modules/zlibmodule.c +++ b/Modules/zlibmodule.c @@ -318,11 +318,11 @@ data: Py_buffer Compressed data. + / wbits: int(c_default="MAX_WBITS") = MAX_WBITS The window buffer size and container format. bufsize: ssize_t(c_default="DEF_BUF_SIZE") = DEF_BUF_SIZE The initial output buffer size. - / Returns a bytes object containing the uncompressed data. [clinic start generated code]*/ @@ -330,7 +330,7 @@ static PyObject * zlib_decompress_impl(PyObject *module, Py_buffer *data, int wbits, Py_ssize_t bufsize) -/*[clinic end generated code: output=77c7e35111dc8c42 input=c13dd2c5696cd17f]*/ +/*[clinic end generated code: output=77c7e35111dc8c42 input=21960936208e9a5b]*/ { PyObject *RetVal = NULL; Byte *ibuf; @@ -750,11 +750,11 @@ data: Py_buffer The binary data to decompress. + / max_length: ssize_t = 0 The maximum allowable length of the decompressed data. Unconsumed input data will be stored in the unconsumed_tail attribute. - / Return a bytes object containing the decompressed version of the data. @@ -766,7 +766,7 @@ static PyObject * zlib_Decompress_decompress_impl(compobject *self, Py_buffer *data, Py_ssize_t max_length) -/*[clinic end generated code: output=6e5173c74e710352 input=d6de9b53c4566b8a]*/ +/*[clinic end generated code: output=6e5173c74e710352 input=b85a212a012b770a]*/ { int err = Z_OK; Py_ssize_t ibuflen, obuflen = DEF_BUF_SIZE, hard_limit; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 03:24:29 2016 From: python-checkins at python.org (ned.deily) Date: Mon, 15 Aug 2016 07:24:29 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzEwOTEw?= =?utf-8?q?=3A_Avoid_C++_compilation_errors_on_FreeBSD_and_OS_X=2E?= Message-ID: <20160815072429.585.5176.F01FAADB@psf.io> https://hg.python.org/cpython/rev/2f857ac9c7af changeset: 102662:2f857ac9c7af branch: 3.5 parent: 102656:2aa2fbb681d8 user: Ned Deily date: Mon Aug 15 03:07:26 2016 -0400 summary: Issue #10910: Avoid C++ compilation errors on FreeBSD and OS X. Patch by Ronald Oussoren. files: Include/pyport.h | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Include/pyport.h b/Include/pyport.h --- a/Include/pyport.h +++ b/Include/pyport.h @@ -688,6 +688,12 @@ #endif #ifdef _PY_PORT_CTYPE_UTF8_ISSUE +#ifndef __cplusplus + /* The workaround below is unsafe in C++ because + * the defines these symbols as real functions, + * with a slightly different signature. + * See issue #10910 + */ #include #include #undef isalnum @@ -705,6 +711,7 @@ #undef toupper #define toupper(c) towupper(btowc(c)) #endif +#endif /* Declarations for symbol visibility. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 03:24:29 2016 From: python-checkins at python.org (ned.deily) Date: Mon, 15 Aug 2016 07:24:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2310910=3A_merge_from_3=2E5?= Message-ID: <20160815072429.22111.36091.96FB8CD2@psf.io> https://hg.python.org/cpython/rev/5ca8790f1161 changeset: 102664:5ca8790f1161 parent: 102661:a4101218364e parent: 102663:27a99a722828 user: Ned Deily date: Mon Aug 15 03:23:23 2016 -0400 summary: Issue #10910: merge from 3.5 files: Include/pyport.h | 11 ++++++++++- Misc/NEWS | 4 ++++ 2 files changed, 14 insertions(+), 1 deletions(-) diff --git a/Include/pyport.h b/Include/pyport.h --- a/Include/pyport.h +++ b/Include/pyport.h @@ -677,7 +677,9 @@ #ifdef __FreeBSD__ #include -#if __FreeBSD_version > 500039 +#if (__FreeBSD_version >= 500040 && __FreeBSD_version < 602113) || \ + (__FreeBSD_version >= 700000 && __FreeBSD_version < 700054) || \ + (__FreeBSD_version >= 800000 && __FreeBSD_version < 800001) # define _PY_PORT_CTYPE_UTF8_ISSUE #endif #endif @@ -688,6 +690,12 @@ #endif #ifdef _PY_PORT_CTYPE_UTF8_ISSUE +#ifndef __cplusplus + /* The workaround below is unsafe in C++ because + * the defines these symbols as real functions, + * with a slightly different signature. + * See issue #10910 + */ #include #include #undef isalnum @@ -705,6 +713,7 @@ #undef toupper #define toupper(c) towupper(btowc(c)) #endif +#endif /* Declarations for symbol visibility. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -165,6 +165,10 @@ - Issue #26662: Set PYTHON_FOR_GEN in configure as the Python program to be used for file generation during the build. +- Issue #10910: Avoid C++ compilation errors on FreeBSD and OS X. + Also update FreedBSD version checks for the original ctype UTF-8 workaround. + + What's New in Python 3.6.0 alpha 3 ================================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 03:24:29 2016 From: python-checkins at python.org (ned.deily) Date: Mon, 15 Aug 2016 07:24:29 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzEwOTEw?= =?utf-8?q?=3A_Update_FreedBSD_version_checks_for_the_ctype_UTF-8_workarou?= =?utf-8?b?bmQu?= Message-ID: <20160815072429.62752.59951.D8CBEC92@psf.io> https://hg.python.org/cpython/rev/27a99a722828 changeset: 102663:27a99a722828 branch: 3.5 user: Ned Deily date: Mon Aug 15 03:08:18 2016 -0400 summary: Issue #10910: Update FreedBSD version checks for the ctype UTF-8 workaround. The original problem has been fixed in newer versions of FreeBSD. Patch by Dimitry Andric of the FreeBSD project. files: Include/pyport.h | 4 +++- Misc/NEWS | 4 ++++ 2 files changed, 7 insertions(+), 1 deletions(-) diff --git a/Include/pyport.h b/Include/pyport.h --- a/Include/pyport.h +++ b/Include/pyport.h @@ -677,7 +677,9 @@ #ifdef __FreeBSD__ #include -#if __FreeBSD_version > 500039 +#if (__FreeBSD_version >= 500040 && __FreeBSD_version < 602113) || \ + (__FreeBSD_version >= 700000 && __FreeBSD_version < 700054) || \ + (__FreeBSD_version >= 800000 && __FreeBSD_version < 800001) # define _PY_PORT_CTYPE_UTF8_ISSUE #endif #endif diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -175,6 +175,10 @@ - Issue #26662: Set PYTHON_FOR_GEN in configure as the Python program to be used for file generation during the build. +- Issue #10910: Avoid C++ compilation errors on FreeBSD and OS X. + Also update FreedBSD version checks for the original ctype UTF-8 workaround. + + What's New in Python 3.5.2? =========================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 03:24:35 2016 From: python-checkins at python.org (ned.deily) Date: Mon, 15 Aug 2016 07:24:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEwOTEw?= =?utf-8?q?=3A_Update_FreedBSD_version_checks_for_the_ctype_UTF-8_workarou?= =?utf-8?b?bmQu?= Message-ID: <20160815072435.13340.44059.9C011E45@psf.io> https://hg.python.org/cpython/rev/12a70477db03 changeset: 102666:12a70477db03 branch: 2.7 user: Ned Deily date: Mon Aug 15 03:04:36 2016 -0400 summary: Issue #10910: Update FreedBSD version checks for the ctype UTF-8 workaround. The original problem has been fixed in newer versions of FreeBSD. Patch by Dimitry Andric of the FreeBSD project. files: Include/pyport.h | 4 +++- Misc/NEWS | 7 +++++++ 2 files changed, 10 insertions(+), 1 deletions(-) diff --git a/Include/pyport.h b/Include/pyport.h --- a/Include/pyport.h +++ b/Include/pyport.h @@ -702,7 +702,9 @@ #ifdef __FreeBSD__ #include -#if __FreeBSD_version > 500039 +#if (__FreeBSD_version >= 500040 && __FreeBSD_version < 602113) || \ + (__FreeBSD_version >= 700000 && __FreeBSD_version < 700054) || \ + (__FreeBSD_version >= 800000 && __FreeBSD_version < 800001) # define _PY_PORT_CTYPE_UTF8_ISSUE #endif #endif diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -97,6 +97,13 @@ - Issue #27369: In test_pyexpat, avoid testing an error message detail that changed in Expat 2.2.0. +Build +----- + +- Issue #10910: Avoid C++ compilation errors on FreeBSD and OS X. + Also update FreedBSD version checks for the original ctype UTF-8 workaround. + + What's New in Python 2.7.12? ============================ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 03:24:35 2016 From: python-checkins at python.org (ned.deily) Date: Mon, 15 Aug 2016 07:24:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEwOTEw?= =?utf-8?q?=3A_Avoid_C++_compilation_errors_on_FreeBSD_and_OS_X=2E?= Message-ID: <20160815072434.67939.6357.75ADB10F@psf.io> https://hg.python.org/cpython/rev/e0ec3471cb09 changeset: 102665:e0ec3471cb09 branch: 2.7 parent: 102654:d568f66fa26c user: Ned Deily date: Mon Aug 15 02:59:31 2016 -0400 summary: Issue #10910: Avoid C++ compilation errors on FreeBSD and OS X. Patch by Ronald Oussoren. files: Include/pyport.h | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Include/pyport.h b/Include/pyport.h --- a/Include/pyport.h +++ b/Include/pyport.h @@ -713,6 +713,12 @@ #endif #ifdef _PY_PORT_CTYPE_UTF8_ISSUE +#ifndef __cplusplus + /* The workaround below is unsafe in C++ because + * the defines these symbols as real functions, + * with a slightly different signature. + * See issue #10910 + */ #include #include #undef isalnum @@ -730,6 +736,7 @@ #undef toupper #define toupper(c) towupper(btowc(c)) #endif +#endif /* Declarations for symbol visibility. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 12:15:52 2016 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 15 Aug 2016 16:15:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2312345=3A_Add_math?= =?utf-8?q?emathcal_constant_tau_to_math_and_cmath=2E?= Message-ID: <20160815161552.13490.16132.923D3D0A@psf.io> https://hg.python.org/cpython/rev/68f2d6098be1 changeset: 102667:68f2d6098be1 parent: 102664:5ca8790f1161 user: Guido van Rossum date: Mon Aug 15 09:12:52 2016 -0700 summary: Issue #12345: Add mathemathcal constant tau to math and cmath. Patch by Lisa Roach. See also PEP 628. files: Doc/library/cmath.rst | 6 ++++-- Doc/library/math.rst | 7 +++++++ Include/pymath.h | 6 ++++++ Lib/test/test_math.py | 1 + Misc/NEWS | 3 +++ Modules/cmathmodule.c | 1 + Modules/mathmodule.c | 1 + 7 files changed, 23 insertions(+), 2 deletions(-) diff --git a/Doc/library/cmath.rst b/Doc/library/cmath.rst --- a/Doc/library/cmath.rst +++ b/Doc/library/cmath.rst @@ -253,6 +253,10 @@ The mathematical constant *e*, as a float. +.. data:: tau + + The mathematical constant *?*, as a float. + .. index:: module: math Note that the selection of functions is similar, but not identical, to that in @@ -276,5 +280,3 @@ Kahan, W: Branch cuts for complex elementary functions; or, Much ado about nothing's sign bit. In Iserles, A., and Powell, M. (eds.), The state of the art in numerical analysis. Clarendon Press (1987) pp165-211. - - diff --git a/Doc/library/math.rst b/Doc/library/math.rst --- a/Doc/library/math.rst +++ b/Doc/library/math.rst @@ -426,6 +426,13 @@ The mathematical constant e = 2.718281..., to available precision. +.. data:: tau + + The mathematical constant ? = 6.283185..., to available precision. + Tau is a circle constant equal to 2?, the ratio of a circle's circumference to + its radius. To learn more about Tau, check out Vi Hart's video `Pi is (still) + Wrong `_, and start celebrating + `Tau day `_ by eating twice as much pie! .. data:: inf diff --git a/Include/pymath.h b/Include/pymath.h --- a/Include/pymath.h +++ b/Include/pymath.h @@ -55,6 +55,12 @@ #define Py_MATH_E 2.7182818284590452354 #endif +/* Tau (2pi) to 40 digits, taken from tauday.com/tau-digits. */ +#ifndef Py_MATH_TAU +#define Py_MATH_TAU 6.2831853071795864769252867665590057683943L +#endif + + /* On x86, Py_FORCE_DOUBLE forces a floating-point number out of an x87 FPU register and into a 64-bit memory location, rounding from extended precision to double precision in the process. On other platforms it does diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -196,6 +196,7 @@ def testConstants(self): self.ftest('pi', math.pi, 3.1415926) self.ftest('e', math.e, 2.7182818) + self.assertEqual(math.tau, 2*math.pi) def testAcos(self): self.assertRaises(TypeError, math.acos) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -57,6 +57,9 @@ Library ------- +- Issue #12345: Add mathemathcal constant tau to math and cmath. See also + PEP 628. + - Issue #26823: traceback.StackSummary.format now abbreviates large sections of repeated lines as "[Previous line repeated {count} more times]" (this change then further affects other traceback display operations in the module). Patch diff --git a/Modules/cmathmodule.c b/Modules/cmathmodule.c --- a/Modules/cmathmodule.c +++ b/Modules/cmathmodule.c @@ -1239,6 +1239,7 @@ PyModule_AddObject(m, "pi", PyFloat_FromDouble(Py_MATH_PI)); PyModule_AddObject(m, "e", PyFloat_FromDouble(Py_MATH_E)); + PyModule_AddObject(m, "tau", PyFloat_FromDouble(Py_MATH_TAU)); /* 2pi */ /* initialize special value tables */ diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -2144,6 +2144,7 @@ PyModule_AddObject(m, "pi", PyFloat_FromDouble(Py_MATH_PI)); PyModule_AddObject(m, "e", PyFloat_FromDouble(Py_MATH_E)); + PyModule_AddObject(m, "tau", PyFloat_FromDouble(Py_MATH_TAU)); /* 2pi */ PyModule_AddObject(m, "inf", PyFloat_FromDouble(m_inf())); #if !defined(PY_NO_SHORT_FLOAT_REPR) || defined(Py_NAN) PyModule_AddObject(m, "nan", PyFloat_FromDouble(m_nan())); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 14:41:54 2016 From: python-checkins at python.org (ned.deily) Date: Mon, 15 Aug 2016 18:41:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323968=3A_Make_OS_?= =?utf-8?q?X_installer_build_script_aware_of_renamed_platform?= Message-ID: <20160815184154.62752.55058.17554F79@psf.io> https://hg.python.org/cpython/rev/ddc4bdae5e41 changeset: 102668:ddc4bdae5e41 user: Ned Deily date: Mon Aug 15 14:37:14 2016 -0400 summary: Issue #23968: Make OS X installer build script aware of renamed platform directory and sysconfigdata file name. This is a workaround for 3.6.0a4 pending resolution of other #23968 items. files: Mac/BuildScript/build-installer.py | 48 +++++++++-------- 1 files changed, 26 insertions(+), 22 deletions(-) diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -1249,6 +1249,8 @@ LDVERSION = LDVERSION.replace('$(VERSION)', VERSION) LDVERSION = LDVERSION.replace('$(ABIFLAGS)', ABIFLAGS) config_suffix = '-' + LDVERSION + if getVersionMajorMinor() >= (3, 6): + config_suffix = config_suffix + '-darwin' else: config_suffix = '' # Python 2.x @@ -1274,7 +1276,7 @@ fp.write(data) fp.close() - # fix _sysconfigdata if it exists + # fix _sysconfigdata # # TODO: make this more robust! test_sysconfig_module of # distutils.tests.test_sysconfig.SysconfigTestCase tests that @@ -1288,28 +1290,30 @@ # _sysconfigdata.py). import pprint - path = os.path.join(path_to_lib, '_sysconfigdata.py') - if os.path.exists(path): - fp = open(path, 'r') - data = fp.read() - fp.close() - # create build_time_vars dict - exec(data) - vars = {} - for k, v in build_time_vars.items(): - if type(v) == type(''): - for p in (include_path, lib_path): - v = v.replace(' ' + p, '') - v = v.replace(p + ' ', '') - vars[k] = v + if getVersionMajorMinor() >= (3, 6): + path = os.path.join(path_to_lib, 'plat-darwin', '_sysconfigdata_m.py') + else: + path = os.path.join(path_to_lib, '_sysconfigdata.py') + fp = open(path, 'r') + data = fp.read() + fp.close() + # create build_time_vars dict + exec(data) + vars = {} + for k, v in build_time_vars.items(): + if type(v) == type(''): + for p in (include_path, lib_path): + v = v.replace(' ' + p, '') + v = v.replace(p + ' ', '') + vars[k] = v - fp = open(path, 'w') - # duplicated from sysconfig._generate_posix_vars() - fp.write('# system configuration generated and used by' - ' the sysconfig module\n') - fp.write('build_time_vars = ') - pprint.pprint(vars, stream=fp) - fp.close() + fp = open(path, 'w') + # duplicated from sysconfig._generate_posix_vars() + fp.write('# system configuration generated and used by' + ' the sysconfig module\n') + fp.write('build_time_vars = ') + pprint.pprint(vars, stream=fp) + fp.close() # Add symlinks in /usr/local/bin, using relative links usr_local_bin = os.path.join(rootDir, 'usr', 'local', 'bin') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 14:41:54 2016 From: python-checkins at python.org (ned.deily) Date: Mon, 15 Aug 2016 18:41:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327736=3A_Prevent_?= =?utf-8?q?segfault_after_interpreter_re-initialization_due?= Message-ID: <20160815184154.19515.28701.4FE000B4@psf.io> https://hg.python.org/cpython/rev/a95d98086621 changeset: 102669:a95d98086621 user: Ned Deily date: Mon Aug 15 14:40:38 2016 -0400 summary: Issue #27736: Prevent segfault after interpreter re-initialization due to ref count problem introduced in code for Issue #27038 in 3.6.0a3. Patch by Xiang Zhang. files: Misc/NEWS | 4 ++++ Modules/posixmodule.c | 2 ++ 2 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -57,6 +57,10 @@ Library ------- +- Issue #27736: Prevent segfault after interpreter re-initialization due + to ref count problem introduced in code for Issue #27038 in 3.6.0a3. + Patch by Xiang Zhang. + - Issue #12345: Add mathemathcal constant tau to math and cmath. See also PEP 628. diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -13355,6 +13355,8 @@ Py_DECREF(unicode); } PyModule_AddObject(m, "_have_functions", list); + + Py_INCREF((PyObject *) &DirEntryType); PyModule_AddObject(m, "DirEntry", (PyObject *)&DirEntryType); initialized = 1; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 18:08:28 2016 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 15 Aug 2016 22:08:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Add_docs_for_t?= =?utf-8?q?yping=2EAnyStr_and_typing=2EText=2E_By_Michael_Lee=2E?= Message-ID: <20160815220827.100354.10017.051C1D94@psf.io> https://hg.python.org/cpython/rev/d5872d0863c8 changeset: 102670:d5872d0863c8 branch: 3.5 parent: 102663:27a99a722828 user: Guido van Rossum date: Mon Aug 15 15:06:38 2016 -0700 summary: Add docs for typing.AnyStr and typing.Text. By Michael Lee. files: Doc/library/typing.rst | 27 +++++++++++++++++++++++++++ 1 files changed, 27 insertions(+), 0 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -647,6 +647,33 @@ yield start start += 1 +.. class:: AnyStr + + ``AnyStr`` is a type variable defined as + ``AnyStr = TypeVar('AnyStr', str, bytes)``. + + It is meant to be used for functions that may accept any kind of string + without allowing different kinds of strings to mix. For example:: + + def concat(a: AnyStr, b: AnyStr) -> AnyStr: + return a + b + + concat(u"foo", u"bar") # Ok, output has type 'unicode' + concat(b"foo", b"bar") # Ok, output has type 'bytes' + concat(u"foo", b"bar") # Error, cannot mix unicode and bytes + +.. class:: Text + + ``Text`` is an alias for ``str``. It is provided to supply a forward + compatible path for Python 2 code: in Python 2, ``Text`` is an alias for + ``unicode``. + + Use ``Text`` to indicate that a value must contain a unicode string in + a manner that is compatible with both Python 2 and Python 3:: + + def add_unicode_checkmark(text: Text) -> Text: + return text + u' \u2713' + .. class:: io Wrapper namespace for I/O stream types. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 18:08:29 2016 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 15 Aug 2016 22:08:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_typo_in_Misc/NEWS=2E?= Message-ID: <20160815220829.79407.50558.CE38CE34@psf.io> https://hg.python.org/cpython/rev/247ecbae325c changeset: 102671:247ecbae325c parent: 102669:a95d98086621 user: Guido van Rossum date: Mon Aug 15 15:07:25 2016 -0700 summary: Fix typo in Misc/NEWS. files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -61,7 +61,7 @@ to ref count problem introduced in code for Issue #27038 in 3.6.0a3. Patch by Xiang Zhang. -- Issue #12345: Add mathemathcal constant tau to math and cmath. See also +- Issue #12345: Add mathemathical constant tau to math and cmath. See also PEP 628. - Issue #26823: traceback.StackSummary.format now abbreviates large sections of -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 18:08:29 2016 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 15 Aug 2016 22:08:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Add_docs_for_typing=2EAnyStr_and_typing=2EText=2E_By_Mic?= =?utf-8?b?aGFlbCBMZWUuIChNZXJnZSAzLjUtPjMuNik=?= Message-ID: <20160815220829.100241.46677.F6A76DED@psf.io> https://hg.python.org/cpython/rev/705416c5909e changeset: 102672:705416c5909e parent: 102671:247ecbae325c parent: 102670:d5872d0863c8 user: Guido van Rossum date: Mon Aug 15 15:08:11 2016 -0700 summary: Add docs for typing.AnyStr and typing.Text. By Michael Lee. (Merge 3.5->3.6) files: Doc/library/typing.rst | 27 +++++++++++++++++++++++++++ 1 files changed, 27 insertions(+), 0 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -652,6 +652,33 @@ yield start start += 1 +.. class:: AnyStr + + ``AnyStr`` is a type variable defined as + ``AnyStr = TypeVar('AnyStr', str, bytes)``. + + It is meant to be used for functions that may accept any kind of string + without allowing different kinds of strings to mix. For example:: + + def concat(a: AnyStr, b: AnyStr) -> AnyStr: + return a + b + + concat(u"foo", u"bar") # Ok, output has type 'unicode' + concat(b"foo", b"bar") # Ok, output has type 'bytes' + concat(u"foo", b"bar") # Error, cannot mix unicode and bytes + +.. class:: Text + + ``Text`` is an alias for ``str``. It is provided to supply a forward + compatible path for Python 2 code: in Python 2, ``Text`` is an alias for + ``unicode``. + + Use ``Text`` to indicate that a value must contain a unicode string in + a manner that is compatible with both Python 2 and Python 3:: + + def add_unicode_checkmark(text: Text) -> Text: + return text + u' \u2713' + .. class:: io Wrapper namespace for I/O stream types. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 20:58:33 2016 From: python-checkins at python.org (nick.coghlan) Date: Tue, 16 Aug 2016 00:58:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326823=3A_fix_trac?= =?utf-8?q?eback_abbreviation_docs?= Message-ID: <20160816005833.100200.45825.7E3D2A95@psf.io> https://hg.python.org/cpython/rev/86d062edb6ab changeset: 102673:86d062edb6ab user: Nick Coghlan date: Tue Aug 16 10:58:14 2016 +1000 summary: Issue #26823: fix traceback abbreviation docs - be clear builtin traceback display was also updated - show example output in What's New - fix versionadded markup files: Doc/library/traceback.rst | 5 ++--- Doc/whatsnew/3.6.rst | 26 +++++++++++++++++++++++--- 2 files changed, 25 insertions(+), 6 deletions(-) diff --git a/Doc/library/traceback.rst b/Doc/library/traceback.rst --- a/Doc/library/traceback.rst +++ b/Doc/library/traceback.rst @@ -302,9 +302,8 @@ repetitions are shown, followed by a summary line stating the exact number of further repetitions. - .. versionchanged:: 3.6 - - Long sequences of repeated frames are now abbreviated. + .. versionchanged:: 3.6 + Long sequences of repeated frames are now abbreviated. :class:`FrameSummary` Objects diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -207,7 +207,12 @@ Other Language Changes ====================== -* None yet. +Some smaller changes made to the core Python language are: + +* Long sequences of repeated traceback lines are now abbreviated as + ``"[Previous line repeated {count} more times]"`` (see + :ref:`py36-traceback` for an example). + (Contributed by Emanuel Barry in :issue:`26823`.) New Modules @@ -438,11 +443,26 @@ (Contributed by Serhiy Storchaka in :issue:`22115`). +.. _py36-traceback: + traceback --------- -The :meth:`~traceback.StackSummary.format` method now abbreviates long sequences -of repeated lines as ``"[Previous line repeated {count} more times]"``. +Both the traceback module and the interpreter's builtin exception display now +abbreviate long sequences of repeated lines in tracebacks as shown in the +following example:: + + >>> def f(): f() + ... + >>> f() + Traceback (most recent call last): + File "", line 1, in + File "", line 1, in f + File "", line 1, in f + File "", line 1, in f + [Previous line repeated 995 more times] + RecursionError: maximum recursion depth exceeded + (Contributed by Emanuel Barry in :issue:`26823`.) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 21:58:54 2016 From: python-checkins at python.org (steve.dower) Date: Tue, 16 Aug 2016 01:58:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Adds_missing_file_to_insta?= =?utf-8?q?ller=2E?= Message-ID: <20160816015854.17350.50992.35056FD4@psf.io> https://hg.python.org/cpython/rev/d277870a6012 changeset: 102674:d277870a6012 user: Steve Dower date: Mon Aug 15 18:58:29 2016 -0700 summary: Adds missing file to installer. files: Tools/msi/exe/exe_reg.wxs | 25 +++++++++++++++++++++++++ 1 files changed, 25 insertions(+), 0 deletions(-) diff --git a/Tools/msi/exe/exe_reg.wxs b/Tools/msi/exe/exe_reg.wxs new file mode 100644 --- /dev/null +++ b/Tools/msi/exe/exe_reg.wxs @@ -0,0 +1,25 @@ +? + + + + + + + + + + + + + + + + + + + + + + + + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 22:41:58 2016 From: python-checkins at python.org (ned.deily) Date: Tue, 16 Aug 2016 02:41:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Added_tag_v3=2E6=2E0a4_for?= =?utf-8?q?_changeset_017cf260936b?= Message-ID: <20160816024158.12826.67433.23226B47@psf.io> https://hg.python.org/cpython/rev/689556b29b76 changeset: 102677:689556b29b76 user: Ned Deily date: Mon Aug 15 16:21:52 2016 -0400 summary: Added tag v3.6.0a4 for changeset 017cf260936b files: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -169,3 +169,4 @@ 5896da372fb044e38595fb74495de1e1e7c8fb3c v3.6.0a1 37889342355223e2fc1438de3dc7ffcd625c60f7 v3.6.0a2 f3edf13dc339b8942ae6b309771ab197dd8ce6fa v3.6.0a3 +017cf260936b444788c9b671d195b7bfd83dbd25 v3.6.0a4 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 22:41:58 2016 From: python-checkins at python.org (ned.deily) Date: Tue, 16 Aug 2016 02:41:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Start_3=2E6=2E0bb1?= Message-ID: <20160816024158.17588.7440.F0385B54@psf.io> https://hg.python.org/cpython/rev/fb147ca3a550 changeset: 102678:fb147ca3a550 user: Ned Deily date: Mon Aug 15 22:32:43 2016 -0400 summary: Start 3.6.0bb1 files: Include/patchlevel.h | 2 +- Misc/NEWS | 12 ++++++++++++ 2 files changed, 13 insertions(+), 1 deletions(-) diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -23,7 +23,7 @@ #define PY_RELEASE_SERIAL 4 /* Version as a string */ -#define PY_VERSION "3.6.0a4" +#define PY_VERSION "3.6.0a4+" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -2,6 +2,18 @@ Python News +++++++++++ +What's New in Python 3.6.0 beta 1 +================================= + +*Release date: XXXX-XX-XX* + +Core and Builtins +----------------- + +Library +------- + + What's New in Python 3.6.0 alpha 4 ================================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 22:41:58 2016 From: python-checkins at python.org (ned.deily) Date: Tue, 16 Aug 2016 02:41:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Version_bump_for_3=2E6=2E0?= =?utf-8?q?a4?= Message-ID: <20160816024158.17289.7160.DF458C71@psf.io> https://hg.python.org/cpython/rev/017cf260936b changeset: 102676:017cf260936b tag: v3.6.0a4 user: Ned Deily date: Mon Aug 15 16:21:29 2016 -0400 summary: Version bump for 3.6.0a4 files: Include/patchlevel.h | 4 ++-- Misc/NEWS | 2 +- README | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -20,10 +20,10 @@ #define PY_MINOR_VERSION 6 #define PY_MICRO_VERSION 0 #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_ALPHA -#define PY_RELEASE_SERIAL 3 +#define PY_RELEASE_SERIAL 4 /* Version as a string */ -#define PY_VERSION "3.6.0a3+" +#define PY_VERSION "3.6.0a4" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -5,7 +5,7 @@ What's New in Python 3.6.0 alpha 4 ================================== -*Release date: XXXX-XX-XX* +*Release date: 2016-08-15* Core and Builtins ----------------- diff --git a/README b/README --- a/README +++ b/README @@ -1,4 +1,4 @@ -This is Python version 3.6.0 alpha 3 +This is Python version 3.6.0 alpha 4 ==================================== Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 22:41:58 2016 From: python-checkins at python.org (ned.deily) Date: Tue, 16 Aug 2016 02:41:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Update_pydoc_topics_for_3?= =?utf-8?b?LjYuMGE0?= Message-ID: <20160816024158.24097.6865.A9AE17EA@psf.io> https://hg.python.org/cpython/rev/d93831918887 changeset: 102675:d93831918887 parent: 102669:a95d98086621 user: Ned Deily date: Mon Aug 15 16:12:59 2016 -0400 summary: Update pydoc topics for 3.6.0a4 files: Lib/pydoc_data/topics.py | 132 +++++++++++++++++++++----- 1 files changed, 105 insertions(+), 27 deletions(-) diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py --- a/Lib/pydoc_data/topics.py +++ b/Lib/pydoc_data/topics.py @@ -1,5 +1,5 @@ # -*- coding: utf-8 -*- -# Autogenerated by Sphinx on Mon Jul 11 15:30:24 2016 +# Autogenerated by Sphinx on Mon Aug 15 16:11:20 2016 topics = {'assert': '\n' 'The "assert" statement\n' '**********************\n' @@ -569,6 +569,14 @@ '*instance* of the\n' ' owner class.\n' '\n' + 'object.__set_name__(self, owner, name)\n' + '\n' + ' Called at the time the owning class *owner* is ' + 'created. The\n' + ' descriptor has been assigned to *name*.\n' + '\n' + ' New in version 3.6.\n' + '\n' 'The attribute "__objclass__" is interpreted by the ' '"inspect" module as\n' 'specifying the class where this object was defined ' @@ -1338,13 +1346,12 @@ '\n' 'A class definition is an executable statement. The inheritance ' 'list\n' - 'usually gives a list of base classes (see Customizing class ' - 'creation\n' - 'for more advanced uses), so each item in the list should evaluate ' - 'to a\n' - 'class object which allows subclassing. Classes without an ' - 'inheritance\n' - 'list inherit, by default, from the base class "object"; hence,\n' + 'usually gives a list of base classes (see Metaclasses for more\n' + 'advanced uses), so each item in the list should evaluate to a ' + 'class\n' + 'object which allows subclassing. Classes without an inheritance ' + 'list\n' + 'inherit, by default, from the base class "object"; hence,\n' '\n' ' class Foo:\n' ' pass\n' @@ -1377,16 +1384,14 @@ ' @f2\n' ' class Foo: pass\n' '\n' - 'is equivalent to\n' + 'is roughly equivalent to\n' '\n' ' class Foo: pass\n' ' Foo = f1(arg)(f2(Foo))\n' '\n' 'The evaluation rules for the decorator expressions are the same as ' 'for\n' - 'function decorators. The result must be a class object, which is ' - 'then\n' - 'bound to the class name.\n' + 'function decorators. The result is then bound to the class name.\n' '\n' "**Programmer's note:** Variables defined in the class definition " 'are\n' @@ -2312,11 +2317,15 @@ ' @f2\n' ' def func(): pass\n' '\n' - 'is equivalent to\n' + 'is roughly equivalent to\n' '\n' ' def func(): pass\n' ' func = f1(arg)(f2(func))\n' '\n' + 'except that the original function is not temporarily bound to ' + 'the name\n' + '"func".\n' + '\n' 'When one or more *parameters* have the form *parameter* "="\n' '*expression*, the function is said to have "default parameter ' 'values."\n' @@ -2440,13 +2449,12 @@ '\n' 'A class definition is an executable statement. The inheritance ' 'list\n' - 'usually gives a list of base classes (see Customizing class ' - 'creation\n' - 'for more advanced uses), so each item in the list should ' - 'evaluate to a\n' - 'class object which allows subclassing. Classes without an ' - 'inheritance\n' - 'list inherit, by default, from the base class "object"; hence,\n' + 'usually gives a list of base classes (see Metaclasses for more\n' + 'advanced uses), so each item in the list should evaluate to a ' + 'class\n' + 'object which allows subclassing. Classes without an inheritance ' + 'list\n' + 'inherit, by default, from the base class "object"; hence,\n' '\n' ' class Foo:\n' ' pass\n' @@ -2482,16 +2490,15 @@ ' @f2\n' ' class Foo: pass\n' '\n' - 'is equivalent to\n' + 'is roughly equivalent to\n' '\n' ' class Foo: pass\n' ' Foo = f1(arg)(f2(Foo))\n' '\n' 'The evaluation rules for the decorator expressions are the same ' 'as for\n' - 'function decorators. The result must be a class object, which ' - 'is then\n' - 'bound to the class name.\n' + 'function decorators. The result is then bound to the class ' + 'name.\n' '\n' "**Programmer's note:** Variables defined in the class definition " 'are\n' @@ -3776,7 +3783,7 @@ '\n' 'interact\n' '\n' - ' Start an interative interpreter (using the "code" module) ' + ' Start an interactive interpreter (using the "code" module) ' 'whose\n' ' global namespace contains all the (global and local) names ' 'found in\n' @@ -5296,11 +5303,15 @@ ' @f2\n' ' def func(): pass\n' '\n' - 'is equivalent to\n' + 'is roughly equivalent to\n' '\n' ' def func(): pass\n' ' func = f1(arg)(f2(func))\n' '\n' + 'except that the original function is not temporarily bound to ' + 'the name\n' + '"func".\n' + '\n' 'When one or more *parameters* have the form *parameter* "="\n' '*expression*, the function is said to have "default parameter ' 'values."\n' @@ -6032,7 +6043,7 @@ 'expression"\n' 'yields a function object. The unnamed object behaves like a ' 'function\n' - 'object defined with\n' + 'object defined with:\n' '\n' ' def (arguments):\n' ' return expression\n' @@ -7964,6 +7975,14 @@ 'of the\n' ' owner class.\n' '\n' + 'object.__set_name__(self, owner, name)\n' + '\n' + ' Called at the time the owning class *owner* is created. ' + 'The\n' + ' descriptor has been assigned to *name*.\n' + '\n' + ' New in version 3.6.\n' + '\n' 'The attribute "__objclass__" is interpreted by the "inspect" ' 'module as\n' 'specifying the class where this object was defined (setting ' @@ -8188,6 +8207,65 @@ 'Customizing class creation\n' '==========================\n' '\n' + 'Whenever a class inherits from another class, ' + '*__init_subclass__* is\n' + 'called on that class. This way, it is possible to write ' + 'classes which\n' + 'change the behavior of subclasses. This is closely related ' + 'to class\n' + 'decorators, but where class decorators only affect the ' + 'specific class\n' + 'they\'re applied to, "__init_subclass__" solely applies to ' + 'future\n' + 'subclasses of the class defining the method.\n' + '\n' + 'classmethod object.__init_subclass__(cls)\n' + '\n' + ' This method is called whenever the containing class is ' + 'subclassed.\n' + ' *cls* is then the new subclass. If defined as a normal ' + 'instance\n' + ' method, this method is implicitly converted to a class ' + 'method.\n' + '\n' + ' Keyword arguments which are given to a new class are ' + 'passed to the\n' + ' parent\'s class "__init_subclass__". For compatibility ' + 'with other\n' + ' classes using "__init_subclass__", one should take out ' + 'the needed\n' + ' keyword arguments and pass the others over to the base ' + 'class, as\n' + ' in:\n' + '\n' + ' class Philosopher:\n' + ' def __init_subclass__(cls, default_name, ' + '**kwargs):\n' + ' super().__init_subclass__(**kwargs)\n' + ' cls.default_name = default_name\n' + '\n' + ' class AustralianPhilosopher(Philosopher, ' + 'default_name="Bruce"):\n' + ' pass\n' + '\n' + ' The default implementation "object.__init_subclass__" ' + 'does nothing,\n' + ' but raises an error if it is called with any arguments.\n' + '\n' + ' Note: The metaclass hint "metaclass" is consumed by the ' + 'rest of\n' + ' the type machinery, and is never passed to ' + '"__init_subclass__"\n' + ' implementations. The actual metaclass (rather than the ' + 'explicit\n' + ' hint) can be accessed as "type(cls)".\n' + '\n' + ' New in version 3.6.\n' + '\n' + '\n' + 'Metaclasses\n' + '-----------\n' + '\n' 'By default, classes are constructed using "type()". The ' 'class body is\n' 'executed in a new namespace and the class name is bound ' -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 15 22:41:59 2016 From: python-checkins at python.org (ned.deily) Date: Tue, 16 Aug 2016 02:41:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_releasing_branch?= Message-ID: <20160816024159.99632.59033.5F6429EE@psf.io> https://hg.python.org/cpython/rev/4f324dce8adb changeset: 102679:4f324dce8adb parent: 102674:d277870a6012 parent: 102678:fb147ca3a550 user: Ned Deily date: Mon Aug 15 22:41:17 2016 -0400 summary: Merge from releasing branch files: .hgtags | 1 + Include/patchlevel.h | 4 +- Lib/pydoc_data/topics.py | 132 +++++++++++++++++++++----- Misc/NEWS | 14 ++- README | 2 +- 5 files changed, 122 insertions(+), 31 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -169,3 +169,4 @@ 5896da372fb044e38595fb74495de1e1e7c8fb3c v3.6.0a1 37889342355223e2fc1438de3dc7ffcd625c60f7 v3.6.0a2 f3edf13dc339b8942ae6b309771ab197dd8ce6fa v3.6.0a3 +017cf260936b444788c9b671d195b7bfd83dbd25 v3.6.0a4 diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -20,10 +20,10 @@ #define PY_MINOR_VERSION 6 #define PY_MICRO_VERSION 0 #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_ALPHA -#define PY_RELEASE_SERIAL 3 +#define PY_RELEASE_SERIAL 4 /* Version as a string */ -#define PY_VERSION "3.6.0a3+" +#define PY_VERSION "3.6.0a4+" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py --- a/Lib/pydoc_data/topics.py +++ b/Lib/pydoc_data/topics.py @@ -1,5 +1,5 @@ # -*- coding: utf-8 -*- -# Autogenerated by Sphinx on Mon Jul 11 15:30:24 2016 +# Autogenerated by Sphinx on Mon Aug 15 16:11:20 2016 topics = {'assert': '\n' 'The "assert" statement\n' '**********************\n' @@ -569,6 +569,14 @@ '*instance* of the\n' ' owner class.\n' '\n' + 'object.__set_name__(self, owner, name)\n' + '\n' + ' Called at the time the owning class *owner* is ' + 'created. The\n' + ' descriptor has been assigned to *name*.\n' + '\n' + ' New in version 3.6.\n' + '\n' 'The attribute "__objclass__" is interpreted by the ' '"inspect" module as\n' 'specifying the class where this object was defined ' @@ -1338,13 +1346,12 @@ '\n' 'A class definition is an executable statement. The inheritance ' 'list\n' - 'usually gives a list of base classes (see Customizing class ' - 'creation\n' - 'for more advanced uses), so each item in the list should evaluate ' - 'to a\n' - 'class object which allows subclassing. Classes without an ' - 'inheritance\n' - 'list inherit, by default, from the base class "object"; hence,\n' + 'usually gives a list of base classes (see Metaclasses for more\n' + 'advanced uses), so each item in the list should evaluate to a ' + 'class\n' + 'object which allows subclassing. Classes without an inheritance ' + 'list\n' + 'inherit, by default, from the base class "object"; hence,\n' '\n' ' class Foo:\n' ' pass\n' @@ -1377,16 +1384,14 @@ ' @f2\n' ' class Foo: pass\n' '\n' - 'is equivalent to\n' + 'is roughly equivalent to\n' '\n' ' class Foo: pass\n' ' Foo = f1(arg)(f2(Foo))\n' '\n' 'The evaluation rules for the decorator expressions are the same as ' 'for\n' - 'function decorators. The result must be a class object, which is ' - 'then\n' - 'bound to the class name.\n' + 'function decorators. The result is then bound to the class name.\n' '\n' "**Programmer's note:** Variables defined in the class definition " 'are\n' @@ -2312,11 +2317,15 @@ ' @f2\n' ' def func(): pass\n' '\n' - 'is equivalent to\n' + 'is roughly equivalent to\n' '\n' ' def func(): pass\n' ' func = f1(arg)(f2(func))\n' '\n' + 'except that the original function is not temporarily bound to ' + 'the name\n' + '"func".\n' + '\n' 'When one or more *parameters* have the form *parameter* "="\n' '*expression*, the function is said to have "default parameter ' 'values."\n' @@ -2440,13 +2449,12 @@ '\n' 'A class definition is an executable statement. The inheritance ' 'list\n' - 'usually gives a list of base classes (see Customizing class ' - 'creation\n' - 'for more advanced uses), so each item in the list should ' - 'evaluate to a\n' - 'class object which allows subclassing. Classes without an ' - 'inheritance\n' - 'list inherit, by default, from the base class "object"; hence,\n' + 'usually gives a list of base classes (see Metaclasses for more\n' + 'advanced uses), so each item in the list should evaluate to a ' + 'class\n' + 'object which allows subclassing. Classes without an inheritance ' + 'list\n' + 'inherit, by default, from the base class "object"; hence,\n' '\n' ' class Foo:\n' ' pass\n' @@ -2482,16 +2490,15 @@ ' @f2\n' ' class Foo: pass\n' '\n' - 'is equivalent to\n' + 'is roughly equivalent to\n' '\n' ' class Foo: pass\n' ' Foo = f1(arg)(f2(Foo))\n' '\n' 'The evaluation rules for the decorator expressions are the same ' 'as for\n' - 'function decorators. The result must be a class object, which ' - 'is then\n' - 'bound to the class name.\n' + 'function decorators. The result is then bound to the class ' + 'name.\n' '\n' "**Programmer's note:** Variables defined in the class definition " 'are\n' @@ -3776,7 +3783,7 @@ '\n' 'interact\n' '\n' - ' Start an interative interpreter (using the "code" module) ' + ' Start an interactive interpreter (using the "code" module) ' 'whose\n' ' global namespace contains all the (global and local) names ' 'found in\n' @@ -5296,11 +5303,15 @@ ' @f2\n' ' def func(): pass\n' '\n' - 'is equivalent to\n' + 'is roughly equivalent to\n' '\n' ' def func(): pass\n' ' func = f1(arg)(f2(func))\n' '\n' + 'except that the original function is not temporarily bound to ' + 'the name\n' + '"func".\n' + '\n' 'When one or more *parameters* have the form *parameter* "="\n' '*expression*, the function is said to have "default parameter ' 'values."\n' @@ -6032,7 +6043,7 @@ 'expression"\n' 'yields a function object. The unnamed object behaves like a ' 'function\n' - 'object defined with\n' + 'object defined with:\n' '\n' ' def (arguments):\n' ' return expression\n' @@ -7964,6 +7975,14 @@ 'of the\n' ' owner class.\n' '\n' + 'object.__set_name__(self, owner, name)\n' + '\n' + ' Called at the time the owning class *owner* is created. ' + 'The\n' + ' descriptor has been assigned to *name*.\n' + '\n' + ' New in version 3.6.\n' + '\n' 'The attribute "__objclass__" is interpreted by the "inspect" ' 'module as\n' 'specifying the class where this object was defined (setting ' @@ -8188,6 +8207,65 @@ 'Customizing class creation\n' '==========================\n' '\n' + 'Whenever a class inherits from another class, ' + '*__init_subclass__* is\n' + 'called on that class. This way, it is possible to write ' + 'classes which\n' + 'change the behavior of subclasses. This is closely related ' + 'to class\n' + 'decorators, but where class decorators only affect the ' + 'specific class\n' + 'they\'re applied to, "__init_subclass__" solely applies to ' + 'future\n' + 'subclasses of the class defining the method.\n' + '\n' + 'classmethod object.__init_subclass__(cls)\n' + '\n' + ' This method is called whenever the containing class is ' + 'subclassed.\n' + ' *cls* is then the new subclass. If defined as a normal ' + 'instance\n' + ' method, this method is implicitly converted to a class ' + 'method.\n' + '\n' + ' Keyword arguments which are given to a new class are ' + 'passed to the\n' + ' parent\'s class "__init_subclass__". For compatibility ' + 'with other\n' + ' classes using "__init_subclass__", one should take out ' + 'the needed\n' + ' keyword arguments and pass the others over to the base ' + 'class, as\n' + ' in:\n' + '\n' + ' class Philosopher:\n' + ' def __init_subclass__(cls, default_name, ' + '**kwargs):\n' + ' super().__init_subclass__(**kwargs)\n' + ' cls.default_name = default_name\n' + '\n' + ' class AustralianPhilosopher(Philosopher, ' + 'default_name="Bruce"):\n' + ' pass\n' + '\n' + ' The default implementation "object.__init_subclass__" ' + 'does nothing,\n' + ' but raises an error if it is called with any arguments.\n' + '\n' + ' Note: The metaclass hint "metaclass" is consumed by the ' + 'rest of\n' + ' the type machinery, and is never passed to ' + '"__init_subclass__"\n' + ' implementations. The actual metaclass (rather than the ' + 'explicit\n' + ' hint) can be accessed as "type(cls)".\n' + '\n' + ' New in version 3.6.\n' + '\n' + '\n' + 'Metaclasses\n' + '-----------\n' + '\n' 'By default, classes are constructed using "type()". The ' 'class body is\n' 'executed in a new namespace and the class name is bound ' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -2,10 +2,22 @@ Python News +++++++++++ +What's New in Python 3.6.0 beta 1 +================================= + +*Release date: XXXX-XX-XX* + +Core and Builtins +----------------- + +Library +------- + + What's New in Python 3.6.0 alpha 4 ================================== -*Release date: XXXX-XX-XX* +*Release date: 2016-08-15* Core and Builtins ----------------- diff --git a/README b/README --- a/README +++ b/README @@ -1,4 +1,4 @@ -This is Python version 3.6.0 alpha 3 +This is Python version 3.6.0 alpha 4 ==================================== Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 00:10:29 2016 From: python-checkins at python.org (terry.reedy) Date: Tue, 16 Aug 2016 04:10:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327611=2C_=2324137?= =?utf-8?q?=3A_Only_change_tkinter_when_easily_restored=2E?= Message-ID: <20160816041028.100200.55148.0A534287@psf.io> https://hg.python.org/cpython/rev/a6a248479b66 changeset: 102680:a6a248479b66 user: Terry Jan Reedy date: Tue Aug 16 00:10:14 2016 -0400 summary: Issue #27611, #24137: Only change tkinter when easily restored. files: Lib/idlelib/pyshell.py | 6 ++++-- Lib/test/test_idle.py | 18 +++++++++--------- 2 files changed, 13 insertions(+), 11 deletions(-) diff --git a/Lib/idlelib/pyshell.py b/Lib/idlelib/pyshell.py --- a/Lib/idlelib/pyshell.py +++ b/Lib/idlelib/pyshell.py @@ -30,6 +30,7 @@ from code import InteractiveInterpreter from platform import python_version, system +from idlelib import testing from idlelib.editor import EditorWindow, fixwordbreaks from idlelib.filelist import FileList from idlelib.colorizer import ColorDelegator @@ -1448,8 +1449,9 @@ enable_edit = enable_edit or edit_start enable_shell = enable_shell or not enable_edit - # Setup root. - if use_subprocess: # Don't break user code run in IDLE process + # Setup root. Don't break user code run in IDLE process. + # Don't change environment when testing. + if use_subprocess and not testing: NoDefaultRoot() root = Tk(className="Idle") root.withdraw() diff --git a/Lib/test/test_idle.py b/Lib/test/test_idle.py --- a/Lib/test/test_idle.py +++ b/Lib/test/test_idle.py @@ -1,23 +1,23 @@ import unittest from test.support import import_module -# Skip test if _thread or _tkinter wasn't built, or idlelib is missing, -# or if tcl/tk version before 8.5, which is needed for ttk widgets. - +# Skip test if _thread or _tkinter wasn't built, if idlelib is missing, +# or if tcl/tk is not the 8.5+ needed for ttk widgets. import_module('threading') # imported by PyShell, imports _thread tk = import_module('tkinter') # imports _tkinter if tk.TkVersion < 8.5: raise unittest.SkipTest("IDLE requires tk 8.5 or later.") -tk.NoDefaultRoot() idlelib = import_module('idlelib') -idlelib.testing = True # Avoid locale-changed test error -# Without test_main present, test.libregrtest.runtest.runtest_inner -# calls (line 173) unittest.TestLoader().loadTestsFromModule(module) -# which calls load_tests() if it finds it. (Unittest.main does the same.) +# Before test imports, tell IDLE to avoid changing the environment. +idlelib.testing = True + +# unittest.main and test.libregrtest.runtest.runtest_inner +# call load_tests, when present, to discover tests to run. from idlelib.idle_test import load_tests if __name__ == '__main__': - unittest.main(verbosity=2, exit=False) + tk.NoDefaultRoot() + unittest.main(exit=False) tk._support_default_root = 1 tk._default_root = None -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 00:18:38 2016 From: python-checkins at python.org (ned.deily) Date: Tue, 16 Aug 2016 04:18:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327736=3A_Improve_?= =?utf-8?q?the_existing_embedded_interpreter_init/fini_test?= Message-ID: <20160816041838.24038.86094.2607AA4B@psf.io> https://hg.python.org/cpython/rev/4f69626fd923 changeset: 102681:4f69626fd923 user: Ned Deily date: Tue Aug 16 00:17:42 2016 -0400 summary: Issue #27736: Improve the existing embedded interpreter init/fini test by increasing the number of iterations. That appears sufficient to expose the ref count problem fixed in this issue. Patch suggested by Xiang Zhang files: Programs/_testembed.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Programs/_testembed.c b/Programs/_testembed.c --- a/Programs/_testembed.c +++ b/Programs/_testembed.c @@ -41,7 +41,7 @@ #endif int i, j; - for (i=0; i<3; i++) { + for (i=0; i<15; i++) { printf("--- Pass %d ---\n", i); _testembed_Py_Initialize(); mainstate = PyThreadState_Get(); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 00:44:20 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 16 Aug 2016 04:44:20 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4?= Message-ID: <20160816044420.17344.51285.6F9054FB@psf.io> https://hg.python.org/cpython/rev/096e800e6834 changeset: 102684:096e800e6834 branch: 3.5 parent: 102670:d5872d0863c8 parent: 102682:1de897915974 user: Benjamin Peterson date: Mon Aug 15 21:43:57 2016 -0700 summary: merge 3.4 files: Lib/test/test_curses.py | 2 ++ Misc/NEWS | 4 ++-- Modules/_cursesmodule.c | 8 ++++++++ 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py --- a/Lib/test/test_curses.py +++ b/Lib/test/test_curses.py @@ -190,6 +190,8 @@ self.assertRaises(ValueError, stdscr.getstr, -400) self.assertRaises(ValueError, stdscr.getstr, 2, 3, -400) + self.assertRaises(ValueError, stdscr.instr, -2) + self.assertRaises(ValueError, stdscr.instr, 2, 3, -2) def test_module_funcs(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,8 +37,8 @@ - Issue #26750: unittest.mock.create_autospec() now works properly for subclasses of property() and other data descriptors. -- In the curses module, raise an error if window.getstr() is passed a negative - value. +- In the curses module, raise an error if window.getstr() or window.instr() is + passed a negative value. - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -1393,6 +1393,10 @@ case 1: if (!PyArg_ParseTuple(args,"i;n", &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } rtn2 = winnstr(self->win, rtn, Py_MIN(n, 1023)); break; case 2: @@ -1403,6 +1407,10 @@ case 3: if (!PyArg_ParseTuple(args, "iii;y,x,n", &y, &x, &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } rtn2 = mvwinnstr(self->win, y, x, rtn, Py_MIN(n,1023)); break; default: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 00:44:19 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 16 Aug 2016 04:44:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_fail_when_nega?= =?utf-8?q?tive_values_are_passed_to_instr=28=29?= Message-ID: <20160816044419.99537.86910.D641D9C4@psf.io> https://hg.python.org/cpython/rev/1de897915974 changeset: 102682:1de897915974 branch: 3.4 parent: 102655:dc1bacd79e66 user: Benjamin Peterson date: Mon Aug 15 21:40:14 2016 -0700 summary: fail when negative values are passed to instr() files: Lib/test/test_curses.py | 2 ++ Misc/NEWS | 4 ++-- Modules/_cursesmodule.c | 8 ++++++++ 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py --- a/Lib/test/test_curses.py +++ b/Lib/test/test_curses.py @@ -165,6 +165,8 @@ self.assertRaises(ValueError, stdscr.getstr, -400) self.assertRaises(ValueError, stdscr.getstr, 2, 3, -400) + self.assertRaises(ValueError, stdscr.instr, -2) + self.assertRaises(ValueError, stdscr.instr, 2, 3, -2) def test_module_funcs(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,8 +13,8 @@ Library ------- -- In the curses module, raise an error if window.getstr() is passed a negative - value. +- In the curses module, raise an error if window.getstr() or window.instr() is + passed a negative value. - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -1456,6 +1456,10 @@ case 1: if (!PyArg_ParseTuple(args,"i;n", &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } rtn2 = winnstr(self->win, rtn, Py_MIN(n, 1023)); break; case 2: @@ -1466,6 +1470,10 @@ case 3: if (!PyArg_ParseTuple(args, "iii;y,x,n", &y, &x, &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } rtn2 = mvwinnstr(self->win, y, x, rtn, Py_MIN(n,1023)); break; default: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 00:44:20 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 16 Aug 2016 04:44:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160816044420.19409.2293.2AFEC4D0@psf.io> https://hg.python.org/cpython/rev/0052694ec55d changeset: 102685:0052694ec55d parent: 102681:4f69626fd923 parent: 102684:096e800e6834 user: Benjamin Peterson date: Mon Aug 15 21:44:06 2016 -0700 summary: merge 3.5 files: Lib/test/test_curses.py | 2 ++ Misc/NEWS | 4 ++-- Modules/_cursesmodule.c | 8 ++++++++ 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py --- a/Lib/test/test_curses.py +++ b/Lib/test/test_curses.py @@ -190,6 +190,8 @@ self.assertRaises(ValueError, stdscr.getstr, -400) self.assertRaises(ValueError, stdscr.getstr, 2, 3, -400) + self.assertRaises(ValueError, stdscr.instr, -2) + self.assertRaises(ValueError, stdscr.instr, 2, 3, -2) def test_module_funcs(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -102,8 +102,8 @@ - Issue #27661: Added tzinfo keyword argument to datetime.combine. -- In the curses module, raise an error if window.getstr() is passed a negative - value. +- In the curses module, raise an error if window.getstr() or window.instr() is + passed a negative value. - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -1393,6 +1393,10 @@ case 1: if (!PyArg_ParseTuple(args,"i;n", &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } rtn2 = winnstr(self->win, rtn, Py_MIN(n, 1023)); break; case 2: @@ -1403,6 +1407,10 @@ case 3: if (!PyArg_ParseTuple(args, "iii;y,x,n", &y, &x, &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } rtn2 = mvwinnstr(self->win, y, x, rtn, Py_MIN(n,1023)); break; default: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 00:44:20 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 16 Aug 2016 04:44:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_fail_when_nega?= =?utf-8?q?tive_values_are_passed_to_instr=28=29?= Message-ID: <20160816044420.19009.15446.65B78B84@psf.io> https://hg.python.org/cpython/rev/85b35300f200 changeset: 102683:85b35300f200 branch: 2.7 parent: 102666:12a70477db03 user: Benjamin Peterson date: Mon Aug 15 21:40:14 2016 -0700 summary: fail when negative values are passed to instr() files: Lib/test/test_curses.py | 2 ++ Misc/NEWS | 4 ++-- Modules/_cursesmodule.c | 8 ++++++++ 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py --- a/Lib/test/test_curses.py +++ b/Lib/test/test_curses.py @@ -187,6 +187,8 @@ self.assertRaises(ValueError, stdscr.getstr, -400) self.assertRaises(ValueError, stdscr.getstr, 2, 3, -400) + self.assertRaises(ValueError, stdscr.instr, -2) + self.assertRaises(ValueError, stdscr.instr, 2, 3, -2) def test_module_funcs(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -31,8 +31,8 @@ - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. -- In the curses module, raise an error if window.getstr() is passed a negative - value. +- In the curses module, raise an error if window.getstr() or window.instr() is + passed a negative value. - Issue #27758: Fix possible integer overflow in the _csv module for large record lengths. diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -1095,6 +1095,10 @@ case 1: if (!PyArg_ParseTuple(args,"i;n", &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } rtn2 = winnstr(self->win,rtn,MIN(n,1023)); break; case 2: @@ -1105,6 +1109,10 @@ case 3: if (!PyArg_ParseTuple(args, "iii;y,x,n", &y, &x, &n)) return NULL; + if (n < 0) { + PyErr_SetString(PyExc_ValueError, "'n' must be nonnegative"); + return NULL; + } rtn2 = mvwinnstr(self->win, y, x, rtn, MIN(n,1023)); break; default: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 00:56:23 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 16 Aug 2016 04:56:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_fix_corner_cas?= =?utf-8?q?es_in_the_management_of_server=5Fhostname_=28closes_=2327773=29?= Message-ID: <20160816045623.23896.5924.8642592D@psf.io> https://hg.python.org/cpython/rev/98c86d5a6655 changeset: 102686:98c86d5a6655 branch: 3.5 parent: 102684:096e800e6834 user: Benjamin Peterson date: Mon Aug 15 21:55:37 2016 -0700 summary: fix corner cases in the management of server_hostname (closes #27773) files: Misc/NEWS | 2 ++ Modules/_ssl.c | 10 ++++------ 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -34,6 +34,8 @@ Library ------- +- Issue #27773: Correct some memory management errors server_hostname in _ssl.wrap_socket(). + - Issue #26750: unittest.mock.create_autospec() now works properly for subclasses of property() and other data descriptors. diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -487,7 +487,6 @@ { PySSLSocket *self; SSL_CTX *ctx = sslctx->ctx; - PyObject *hostname; long mode; self = PyObject_New(PySSLSocket, &PySSLSocket_Type); @@ -501,16 +500,16 @@ self->shutdown_seen_zero = 0; self->handshake_done = 0; self->owner = NULL; + self->server_hostname = NULL; if (server_hostname != NULL) { - hostname = PyUnicode_Decode(server_hostname, strlen(server_hostname), - "idna", "strict"); + PyObject *hostname = PyUnicode_Decode(server_hostname, strlen(server_hostname), + "idna", "strict"); if (hostname == NULL) { Py_DECREF(self); return NULL; } self->server_hostname = hostname; - } else - self->server_hostname = NULL; + } Py_INCREF(sslctx); @@ -563,7 +562,6 @@ self->Socket = PyWeakref_NewRef((PyObject *) sock, NULL); if (self->Socket == NULL) { Py_DECREF(self); - Py_XDECREF(self->server_hostname); return NULL; } } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 00:56:23 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 16 Aug 2016 04:56:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjc3NzMp?= Message-ID: <20160816045623.12748.55719.FFF6614C@psf.io> https://hg.python.org/cpython/rev/a8cd67e80ed3 changeset: 102687:a8cd67e80ed3 parent: 102685:0052694ec55d parent: 102686:98c86d5a6655 user: Benjamin Peterson date: Mon Aug 15 21:56:11 2016 -0700 summary: merge 3.5 (#27773) files: Misc/NEWS | 2 ++ Modules/_ssl.c | 10 ++++------ 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -90,6 +90,8 @@ - Issue #6422: Add autorange method to timeit.Timer objects. +- Issue #27773: Correct some memory management errors server_hostname in _ssl.wrap_socket(). + - Issue #26750: unittest.mock.create_autospec() now works properly for subclasses of property() and other data descriptors. Removes the never publicly used, never documented unittest.mock.DescriptorTypes tuple. diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -487,7 +487,6 @@ { PySSLSocket *self; SSL_CTX *ctx = sslctx->ctx; - PyObject *hostname; long mode; self = PyObject_New(PySSLSocket, &PySSLSocket_Type); @@ -501,16 +500,16 @@ self->shutdown_seen_zero = 0; self->handshake_done = 0; self->owner = NULL; + self->server_hostname = NULL; if (server_hostname != NULL) { - hostname = PyUnicode_Decode(server_hostname, strlen(server_hostname), - "idna", "strict"); + PyObject *hostname = PyUnicode_Decode(server_hostname, strlen(server_hostname), + "idna", "strict"); if (hostname == NULL) { Py_DECREF(self); return NULL; } self->server_hostname = hostname; - } else - self->server_hostname = NULL; + } Py_INCREF(sslctx); @@ -563,7 +562,6 @@ self->Socket = PyWeakref_NewRef((PyObject *) sock, NULL); if (self->Socket == NULL) { Py_DECREF(self); - Py_XDECREF(self->server_hostname); return NULL; } } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 01:05:28 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 16 Aug 2016 05:05:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_do_not_decref_?= =?utf-8?q?value_borrowed_from_list_=28closes_=2327774=29?= Message-ID: <20160816050528.25057.7501.BAC72BFE@psf.io> https://hg.python.org/cpython/rev/cbf2a05648b3 changeset: 102689:cbf2a05648b3 branch: 3.3 parent: 102646:54c74212db91 user: Benjamin Peterson date: Mon Aug 15 22:01:41 2016 -0700 summary: do not decref value borrowed from list (closes #27774) files: Misc/NEWS | 2 ++ Modules/_sre.c | 4 +--- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,8 @@ Library ------- +- Issue #27774: Fix possible Py_DECREF on unowned object in _sre. + - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. - Issue #27758: Fix possible integer overflow in the _csv module for large record diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -3401,10 +3401,8 @@ if (!key) goto failed; value = match_getslice(self, key, def); - if (!value) { - Py_DECREF(key); + if (!value) goto failed; - } status = PyDict_SetItem(result, key, value); Py_DECREF(value); if (status < 0) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 01:05:28 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 16 Aug 2016 05:05:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_do_not_decref_?= =?utf-8?q?value_borrowed_from_list_=28closes_=2327774=29?= Message-ID: <20160816050528.12620.41063.8452D128@psf.io> https://hg.python.org/cpython/rev/4ca84a3e37d7 changeset: 102688:4ca84a3e37d7 branch: 2.7 parent: 102683:85b35300f200 user: Benjamin Peterson date: Mon Aug 15 22:01:41 2016 -0700 summary: do not decref value borrowed from list (closes #27774) files: Misc/NEWS | 2 ++ Modules/_sre.c | 4 +--- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,8 @@ Library ------- +- Issue #27774: Fix possible Py_DECREF on unowned object in _sre. + - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. - In the curses module, raise an error if window.getstr() or window.instr() is diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -3438,10 +3438,8 @@ if (!key) goto failed; value = match_getslice(self, key, def); - if (!value) { - Py_DECREF(key); + if (!value) goto failed; - } status = PyDict_SetItem(result, key, value); Py_DECREF(value); if (status < 0) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 01:05:28 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 16 Aug 2016 05:05:28 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_merge_3=2E3_=28=2327774=29?= Message-ID: <20160816050528.28553.92469.05CB396C@psf.io> https://hg.python.org/cpython/rev/2e404ac88e0e changeset: 102690:2e404ac88e0e branch: 3.4 parent: 102682:1de897915974 parent: 102689:cbf2a05648b3 user: Benjamin Peterson date: Mon Aug 15 22:03:44 2016 -0700 summary: merge 3.3 (#27774) files: Misc/NEWS | 2 ++ Modules/_sre.c | 4 +--- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -16,6 +16,8 @@ - In the curses module, raise an error if window.getstr() or window.instr() is passed a negative value. +- Issue #27774: Fix possible Py_DECREF on unowned object in _sre. + - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. - Issue #27758: Fix possible integer overflow in the _csv module for large record diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -2137,10 +2137,8 @@ if (!key) goto failed; value = match_getslice(self, key, def); - if (!value) { - Py_DECREF(key); + if (!value) goto failed; - } status = PyDict_SetItem(result, key, value); Py_DECREF(value); if (status < 0) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 01:05:28 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 16 Aug 2016 05:05:28 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4_=28=2327774=29?= Message-ID: <20160816050528.100516.42258.19DC2031@psf.io> https://hg.python.org/cpython/rev/424cb9482974 changeset: 102691:424cb9482974 branch: 3.5 parent: 102686:98c86d5a6655 parent: 102690:2e404ac88e0e user: Benjamin Peterson date: Mon Aug 15 22:05:06 2016 -0700 summary: merge 3.4 (#27774) files: Misc/NEWS | 2 ++ Modules/_sre.c | 4 +--- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -42,6 +42,8 @@ - In the curses module, raise an error if window.getstr() or window.instr() is passed a negative value. +- Issue #27774: Fix possible Py_DECREF on unowned object in _sre. + - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. - Issue #27758: Fix possible integer overflow in the _csv module for large record diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -2196,10 +2196,8 @@ if (!key) goto failed; value = match_getslice(self, key, default_value); - if (!value) { - Py_DECREF(key); + if (!value) goto failed; - } status = PyDict_SetItem(result, key, value); Py_DECREF(value); if (status < 0) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 01:05:34 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 16 Aug 2016 05:05:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjc3NzQp?= Message-ID: <20160816050534.100144.50342.352378B4@psf.io> https://hg.python.org/cpython/rev/64b0e0a29874 changeset: 102692:64b0e0a29874 parent: 102687:a8cd67e80ed3 parent: 102691:424cb9482974 user: Benjamin Peterson date: Mon Aug 15 22:05:16 2016 -0700 summary: merge 3.5 (#27774) files: Misc/NEWS | 2 ++ Modules/_sre.c | 4 +--- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -107,6 +107,8 @@ - In the curses module, raise an error if window.getstr() or window.instr() is passed a negative value. +- Issue #27774: Fix possible Py_DECREF on unowned object in _sre. + - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. - Issue #27758: Fix possible integer overflow in the _csv module for large record diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -2194,10 +2194,8 @@ if (!key) goto failed; value = match_getslice(self, key, default_value); - if (!value) { - Py_DECREF(key); + if (!value) goto failed; - } status = PyDict_SetItem(result, key, value); Py_DECREF(value); if (status < 0) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 01:13:09 2016 From: python-checkins at python.org (georg.brandl) Date: Tue, 16 Aug 2016 05:13:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_versionadded_tags_for_?= =?utf-8?b?KGMpbWF0aC50YXUu?= Message-ID: <20160816051309.85070.56137.A05B51F8@psf.io> https://hg.python.org/cpython/rev/5c8e54d47221 changeset: 102693:5c8e54d47221 user: Georg Brandl date: Tue Aug 16 07:08:46 2016 +0200 summary: Add versionadded tags for (c)math.tau. files: Doc/library/cmath.rst | 2 ++ Doc/library/math.rst | 2 ++ 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/library/cmath.rst b/Doc/library/cmath.rst --- a/Doc/library/cmath.rst +++ b/Doc/library/cmath.rst @@ -257,6 +257,8 @@ The mathematical constant *?*, as a float. + .. versionadded:: 3.6 + .. index:: module: math Note that the selection of functions is similar, but not identical, to that in diff --git a/Doc/library/math.rst b/Doc/library/math.rst --- a/Doc/library/math.rst +++ b/Doc/library/math.rst @@ -434,6 +434,8 @@ Wrong `_, and start celebrating `Tau day `_ by eating twice as much pie! + .. versionadded:: 3.6 + .. data:: inf A floating-point positive infinity. (For negative infinity, use -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 01:44:49 2016 From: python-checkins at python.org (terry.reedy) Date: Tue, 16 Aug 2016 05:44:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NjEx?= =?utf-8?q?=3A_Don=27t_import_volatile_attribute=2E?= Message-ID: <20160816054449.22286.89232.EADE6AD2@psf.io> https://hg.python.org/cpython/rev/a42933335897 changeset: 102695:a42933335897 branch: 3.5 parent: 102691:424cb9482974 user: Terry Jan Reedy date: Tue Aug 16 01:44:12 2016 -0400 summary: Issue #27611: Don't import volatile attribute. files: Lib/tkinter/tix.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Lib/tkinter/tix.py b/Lib/tkinter/tix.py --- a/Lib/tkinter/tix.py +++ b/Lib/tkinter/tix.py @@ -26,8 +26,10 @@ # appreciate the advantages. # +import os +import tkinter from tkinter import * -from tkinter import _cnfmerge, _default_root +from tkinter import _cnfmerge # WARNING - TkVersion is a limited precision floating point number if TkVersion < 3.999: @@ -72,7 +74,6 @@ # BEWARE - this is implemented by copying some code from the Widget class # in Tkinter (to override Widget initialization) and is therefore # liable to break. -import tkinter, os # Could probably add this to Tkinter.Misc class tixCommand: @@ -476,7 +477,7 @@ (multiple) Display Items""" def __init__(self, itemtype, cnf={}, **kw): - master = _default_root # global from Tkinter + master = tkinter._default_root # global from Tkinter if not master and 'refwindow' in cnf: master=cnf['refwindow'] elif not master and 'refwindow' in kw: master= kw['refwindow'] elif not master: raise RuntimeError("Too early to create display style: no root window") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 01:44:49 2016 From: python-checkins at python.org (terry.reedy) Date: Tue, 16 Aug 2016 05:44:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3NjEx?= =?utf-8?q?=3A_Don=27t_import_volatile_attribute=2E?= Message-ID: <20160816054449.24002.1371.47777DBC@psf.io> https://hg.python.org/cpython/rev/cddd633b959f changeset: 102694:cddd633b959f branch: 2.7 parent: 102688:4ca84a3e37d7 user: Terry Jan Reedy date: Tue Aug 16 01:44:06 2016 -0400 summary: Issue #27611: Don't import volatile attribute. files: Lib/lib-tk/Tix.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Lib/lib-tk/Tix.py b/Lib/lib-tk/Tix.py --- a/Lib/lib-tk/Tix.py +++ b/Lib/lib-tk/Tix.py @@ -26,8 +26,10 @@ # appreciate the advantages. # +import os +import Tkinter from Tkinter import * -from Tkinter import _flatten, _cnfmerge, _default_root +from Tkinter import _flatten, _cnfmerge # WARNING - TkVersion is a limited precision floating point number if TkVersion < 3.999: @@ -72,7 +74,6 @@ # BEWARE - this is implemented by copying some code from the Widget class # in Tkinter (to override Widget initialization) and is therefore # liable to break. -import Tkinter, os # Could probably add this to Tkinter.Misc class tixCommand: @@ -476,7 +477,7 @@ (multiple) Display Items""" def __init__(self, itemtype, cnf={}, **kw): - master = _default_root # global from Tkinter + master = Tkinter._default_root if not master and 'refwindow' in cnf: master=cnf['refwindow'] elif not master and 'refwindow' in kw: master= kw['refwindow'] elif not master: raise RuntimeError, "Too early to create display style: no root window" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 01:44:49 2016 From: python-checkins at python.org (terry.reedy) Date: Tue, 16 Aug 2016 05:44:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20160816054449.85070.11521.EE5C0234@psf.io> https://hg.python.org/cpython/rev/965a6a5539f2 changeset: 102696:965a6a5539f2 parent: 102693:5c8e54d47221 parent: 102695:a42933335897 user: Terry Jan Reedy date: Tue Aug 16 01:44:31 2016 -0400 summary: Merge with 3.5 files: Lib/tkinter/tix.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Lib/tkinter/tix.py b/Lib/tkinter/tix.py --- a/Lib/tkinter/tix.py +++ b/Lib/tkinter/tix.py @@ -26,8 +26,10 @@ # appreciate the advantages. # +import os +import tkinter from tkinter import * -from tkinter import _cnfmerge, _default_root +from tkinter import _cnfmerge import _tkinter # If this fails your Python may not be configured for Tk @@ -68,7 +70,6 @@ # BEWARE - this is implemented by copying some code from the Widget class # in Tkinter (to override Widget initialization) and is therefore # liable to break. -import tkinter, os # Could probably add this to Tkinter.Misc class tixCommand: @@ -472,7 +473,7 @@ (multiple) Display Items""" def __init__(self, itemtype, cnf={}, **kw): - master = _default_root # global from Tkinter + master = tkinter._default_root # global from Tkinter if not master and 'refwindow' in cnf: master=cnf['refwindow'] elif not master and 'refwindow' in kw: master= kw['refwindow'] elif not master: raise RuntimeError("Too early to create display style: no root window") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 02:56:38 2016 From: python-checkins at python.org (gregory.p.smith) Date: Tue, 16 Aug 2016 06:56:38 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2NzUw?= =?utf-8?q?=3A_use_inspect=2Eisdatadescriptor_instead_of_our_own?= Message-ID: <20160816065638.28741.26833.ACAAD9DE@psf.io> https://hg.python.org/cpython/rev/3ff02380b1bf changeset: 102697:3ff02380b1bf branch: 3.5 parent: 102695:a42933335897 user: Gregory P. Smith date: Mon Aug 15 23:23:40 2016 -0700 summary: Issue #26750: use inspect.isdatadescriptor instead of our own _is_data_descriptor(). files: Lib/unittest/mock.py | 8 +------- 1 files changed, 1 insertions(+), 7 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -72,12 +72,6 @@ ) -def _is_data_descriptor(obj): - # Data descriptors are Properties, slots, getsets and C data members. - return ((hasattr(obj, '__set__') or hasattr(obj, '__del__')) and - hasattr(obj, '__get__')) - - def _get_signature_object(func, as_instance, eat_self): """ Given an arbitrary, possibly callable object, try to create a suitable @@ -2138,7 +2132,7 @@ _kwargs.update(kwargs) Klass = MagicMock - if _is_data_descriptor(spec): + if inspect.isdatadescriptor(spec): # descriptors don't have a spec # because we don't know what type they return _kwargs = {} -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 02:56:39 2016 From: python-checkins at python.org (gregory.p.smith) Date: Tue, 16 Aug 2016 06:56:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326750=3A_use_inspect=2Eisdatadescriptor_instead?= =?utf-8?q?_of_our_own?= Message-ID: <20160816065639.100354.20292.8EED5313@psf.io> https://hg.python.org/cpython/rev/d51a66622266 changeset: 102698:d51a66622266 parent: 102696:965a6a5539f2 parent: 102697:3ff02380b1bf user: Gregory P. Smith date: Mon Aug 15 23:56:32 2016 -0700 summary: Issue #26750: use inspect.isdatadescriptor instead of our own _is_data_descriptor(). files: Lib/unittest/mock.py | 8 +------- 1 files changed, 1 insertions(+), 7 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -60,12 +60,6 @@ ) -def _is_data_descriptor(obj): - # Data descriptors are Properties, slots, getsets and C data members. - return ((hasattr(obj, '__set__') or hasattr(obj, '__del__')) and - hasattr(obj, '__get__')) - - def _get_signature_object(func, as_instance, eat_self): """ Given an arbitrary, possibly callable object, try to create a suitable @@ -2149,7 +2143,7 @@ _kwargs.update(kwargs) Klass = MagicMock - if _is_data_descriptor(spec): + if inspect.isdatadescriptor(spec): # descriptors don't have a spec # because we don't know what type they return _kwargs = {} -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 09:24:59 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 16 Aug 2016 13:24:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327776=3A_Cleanup_?= =?utf-8?q?random=2Ec?= Message-ID: <20160816132459.100341.60653.5044A98F@psf.io> https://hg.python.org/cpython/rev/980e2c781810 changeset: 102699:980e2c781810 user: Victor Stinner date: Tue Aug 16 15:23:58 2016 +0200 summary: Issue #27776: Cleanup random.c * Add pyurandom() helper function to factorize the code * don't call Py_FatalError() in helper functions, but only in _PyRandom_Init() if pyurandom() failed, to uniformize the code files: Python/random.c | 133 ++++++++++++++++++++--------------- 1 files changed, 76 insertions(+), 57 deletions(-) diff --git a/Python/random.c b/Python/random.c --- a/Python/random.c +++ b/Python/random.c @@ -39,10 +39,9 @@ return 0; error: - if (raise) + if (raise) { PyErr_SetFromWindowsErr(0); - else - Py_FatalError("Failed to initialize Windows random API (CryptoGen)"); + } return -1; } @@ -55,8 +54,9 @@ if (hCryptProv == 0) { - if (win32_urandom_init(raise) == -1) + if (win32_urandom_init(raise) == -1) { return -1; + } } while (size > 0) @@ -65,11 +65,9 @@ if (!CryptGenRandom(hCryptProv, (DWORD)chunk, buffer)) { /* CryptGenRandom() failed */ - if (raise) + if (raise) { PyErr_SetFromWindowsErr(0); - else - Py_FatalError("Failed to initialized the randomized hash " - "secret using CryptoGen)"); + } return -1; } buffer += chunk; @@ -86,29 +84,28 @@ /* Fill buffer with size pseudo-random bytes generated by getentropy(). Return 0 on success, or raise an exception and return -1 on error. - If fatal is nonzero, call Py_FatalError() instead of raising an exception - on error. */ + If raise is zero, don't raise an exception on error. */ static int -py_getentropy(unsigned char *buffer, Py_ssize_t size, int fatal) +py_getentropy(char *buffer, Py_ssize_t size, int raise) { while (size > 0) { Py_ssize_t len = Py_MIN(size, 256); int res; - if (!fatal) { + if (raise) { Py_BEGIN_ALLOW_THREADS res = getentropy(buffer, len); Py_END_ALLOW_THREADS - - if (res < 0) { - PyErr_SetFromErrno(PyExc_OSError); - return -1; - } } else { res = getentropy(buffer, len); - if (res < 0) - Py_FatalError("getentropy() failed"); + } + + if (res < 0) { + if (raise) { + PyErr_SetFromErrno(PyExc_OSError); + } + return -1; } buffer += len; @@ -195,18 +192,15 @@ if (errno == EINTR) { if (PyErr_CheckSignals()) { - if (!raise) - Py_FatalError("getrandom() interrupted by a signal"); return -1; } /* retry getrandom() */ continue; } - if (raise) + if (raise) { PyErr_SetFromErrno(PyExc_OSError); - else - Py_FatalError("getrandom() failed"); + } return -1; } @@ -225,9 +219,9 @@ /* Read size bytes from /dev/urandom into buffer. - Call Py_FatalError() on error. */ -static void -dev_urandom_noraise(unsigned char *buffer, Py_ssize_t size) + Return 0 success, or return -1 on error. */ +static int +dev_urandom_noraise(char *buffer, Py_ssize_t size) { int fd; Py_ssize_t n; @@ -235,31 +229,35 @@ assert (0 < size); #ifdef PY_GETRANDOM - if (py_getrandom(buffer, size, 0) == 1) - return; + if (py_getrandom(buffer, size, 0) == 1) { + return 0; + } /* getrandom() is not supported by the running kernel, fall back * on reading /dev/urandom */ #endif fd = _Py_open_noraise("/dev/urandom", O_RDONLY); - if (fd < 0) - Py_FatalError("Failed to open /dev/urandom"); + if (fd < 0) { + return -1; + } while (0 < size) { do { n = read(fd, buffer, (size_t)size); } while (n < 0 && errno == EINTR); - if (n <= 0) - { + + if (n <= 0) { /* stop on error or if read(size) returned 0 */ - Py_FatalError("Failed to read bytes from /dev/urandom"); - break; + return -1; } + buffer += n; size -= n; } close(fd); + + return 0; } /* Read size bytes from /dev/urandom into buffer. @@ -379,6 +377,40 @@ } } +/* If raise is zero: + * - Don't raise exceptions on error + * - Don't call PyErr_CheckSignals() on EINTR (retry directly the interrupted + * syscall) + * - Don't release the GIL to call syscalls. */ +static int +pyurandom(void *buffer, Py_ssize_t size, int raise) +{ + if (size < 0) { + if (raise) { + PyErr_Format(PyExc_ValueError, + "negative argument not allowed"); + } + return -1; + } + + if (size == 0) { + return 0; + } + +#ifdef MS_WINDOWS + return win32_urandom((unsigned char *)buffer, size, raise); +#elif defined(PY_GETENTROPY) + return py_getentropy(buffer, size, raise); +#else + if (raise) { + return dev_urandom_python(buffer, size); + } + else { + return dev_urandom_noraise(buffer, size); + } +#endif +} + /* Fill buffer with size pseudo-random bytes from the operating system random number generator (RNG). It is suitable for most cryptographic purposes except long living private keys for asymmetric encryption. @@ -387,21 +419,7 @@ int _PyOS_URandom(void *buffer, Py_ssize_t size) { - if (size < 0) { - PyErr_Format(PyExc_ValueError, - "negative argument not allowed"); - return -1; - } - if (size == 0) - return 0; - -#ifdef MS_WINDOWS - return win32_urandom((unsigned char *)buffer, size, 1); -#elif defined(PY_GETENTROPY) - return py_getentropy(buffer, size, 0); -#else - return dev_urandom_python((char*)buffer, size); -#endif + return pyurandom(buffer, size, 1); } void @@ -442,13 +460,14 @@ } } else { -#ifdef MS_WINDOWS - (void)win32_urandom(secret, secret_size, 0); -#elif defined(PY_GETENTROPY) - (void)py_getentropy(secret, secret_size, 1); -#else - dev_urandom_noraise(secret, secret_size); -#endif + int res; + + /* _PyRandom_Init() is called very early in the Python initialization + * and so exceptions cannot be used. */ + res = pyurandom(secret, secret_size, 0); + if (res < 0) { + Py_FatalError("failed to get random numbers to initialize Python"); + } } } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 09:25:00 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 16 Aug 2016 13:25:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327776=3A_=5FPyRan?= =?utf-8?q?dom=5FInit=28=29_doesn=27t_call_PyErr=5FCheckSignals=28=29_anym?= =?utf-8?q?ore?= Message-ID: <20160816132459.24115.40236.6E5F5356@psf.io> https://hg.python.org/cpython/rev/265644bad99e changeset: 102700:265644bad99e user: Victor Stinner date: Tue Aug 16 15:19:09 2016 +0200 summary: Issue #27776: _PyRandom_Init() doesn't call PyErr_CheckSignals() anymore Modify py_getrandom() to not call PyErr_CheckSignals() if raise is zero. _PyRandom_Init() is called very early in the Python initialization, so it's safer to not call PyErr_CheckSignals(). files: Python/random.c | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Python/random.c b/Python/random.c --- a/Python/random.c +++ b/Python/random.c @@ -191,10 +191,13 @@ } if (errno == EINTR) { - if (PyErr_CheckSignals()) { - return -1; + if (raise) { + if (PyErr_CheckSignals()) { + return -1; + } } - /* retry getrandom() */ + + /* retry getrandom() if it was interrupted by a signal */ continue; } -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Tue Aug 16 11:07:11 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 16 Aug 2016 16:07:11 +0100 Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7 2016-08-16 Message-ID: Results for project Python 2.7, build date 2016-08-16 02:45:36 +0000 commit: 12a70477db03 previous commit: d568f66fa26c revision date: 2016-08-15 07:04:36 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.12% 0.41% 5.03% 5.54% :-) pybench 0.23% -0.12% 5.64% 4.31% :-( regex_v8 0.64% 0.12% -2.35% 11.00% :-) nbody 0.07% 0.20% 8.83% 3.15% :-) json_dump_v2 0.34% 0.04% 2.52% 10.33% :-| normal_startup 0.52% -0.15% -0.60% 2.21% :-) ssbench 0.24% 0.16% 2.62% 1.68% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-08-16/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Tue Aug 16 11:08:20 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 16 Aug 2016 16:08:20 +0100 Subject: [Python-checkins] BAD Benchmark Results for Python Default 2016-08-16 Message-ID: Results for project Python default, build date 2016-08-16 02:00:24 +0000 commit: d277870a6012 previous commit: 4b3899e734d5 revision date: 2016-08-16 01:58:29 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.53% 0.46% 11.66% 15.08% :-) pybench 0.15% 0.36% 5.82% 6.33% :-( regex_v8 2.63% -2.66% -4.14% 6.10% :-| nbody 0.11% 0.06% 1.79% 6.41% :-( json_dump_v2 0.36% -0.42% -2.26% 12.15% :-| normal_startup 0.89% 0.04% 1.25% 5.60% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-default-2016-08-16/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Tue Aug 16 12:47:54 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 16 Aug 2016 16:47:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327776=3A_dev=5Fur?= =?utf-8?q?andom=28raise=3D0=29_now_closes_the_file_descriptor_on_error?= Message-ID: <20160816164754.85153.39476.D1BC1C24@psf.io> https://hg.python.org/cpython/rev/ad141164c792 changeset: 102702:ad141164c792 user: Victor Stinner date: Tue Aug 16 18:27:44 2016 +0200 summary: Issue #27776: dev_urandom(raise=0) now closes the file descriptor on error files: Python/random.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Python/random.c b/Python/random.c --- a/Python/random.c +++ b/Python/random.c @@ -331,6 +331,7 @@ if (n <= 0) { /* stop on error or if read(size) returned 0 */ + close(fd); return -1; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 12:47:55 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 16 Aug 2016 16:47:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327776=3A_Cleanup_?= =?utf-8?q?random=2Ec?= Message-ID: <20160816164754.100241.15769.96138D97@psf.io> https://hg.python.org/cpython/rev/86d0d74bc2e1 changeset: 102701:86d0d74bc2e1 user: Victor Stinner date: Tue Aug 16 18:46:38 2016 +0200 summary: Issue #27776: Cleanup random.c Merge dev_urandom_python() and dev_urandom_noraise() functions to reduce code duplication. files: Python/random.c | 233 +++++++++++++++++------------------ 1 files changed, 111 insertions(+), 122 deletions(-) diff --git a/Python/random.c b/Python/random.c --- a/Python/random.c +++ b/Python/random.c @@ -77,7 +77,7 @@ } /* Issue #25003: Don't use getentropy() on Solaris (available since - * Solaris 11.3), it is blocking whereas os.urandom() should not block. */ + Solaris 11.3), it is blocking whereas os.urandom() should not block. */ #elif defined(HAVE_GETENTROPY) && !defined(sun) #define PY_GETENTROPY 1 @@ -119,25 +119,32 @@ #if defined(HAVE_GETRANDOM) || defined(HAVE_GETRANDOM_SYSCALL) #define PY_GETRANDOM 1 +/* Call getrandom() + - Return 1 on success + - Return 0 if getrandom() syscall is not available (fails with ENOSYS). + - Raise an exception (if raise is non-zero) and return -1 on error: + getrandom() failed with EINTR and the Python signal handler raised an + exception, or getrandom() failed with a different error. */ static int py_getrandom(void *buffer, Py_ssize_t size, int raise) { /* Is getrandom() supported by the running kernel? - * Need Linux kernel 3.17 or newer, or Solaris 11.3 or newer */ + Need Linux kernel 3.17 or newer, or Solaris 11.3 or newer */ static int getrandom_works = 1; /* getrandom() on Linux will block if called before the kernel has - * initialized the urandom entropy pool. This will cause Python - * to hang on startup if called very early in the boot process - - * see https://bugs.python.org/issue26839. To avoid this, use the - * GRND_NONBLOCK flag. */ + initialized the urandom entropy pool. This will cause Python + to hang on startup if called very early in the boot process - + see https://bugs.python.org/issue26839. To avoid this, use the + GRND_NONBLOCK flag. */ const int flags = GRND_NONBLOCK; char *dest; long n; - if (!getrandom_works) + if (!getrandom_works) { return 0; + } dest = buffer; while (0 < size) { @@ -161,8 +168,8 @@ } #else /* On Linux, use the syscall() function because the GNU libc doesn't - * expose the Linux getrandom() syscall yet. See: - * https://sourceware.org/bugzilla/show_bug.cgi?id=17252 */ + expose the Linux getrandom() syscall yet. See: + https://sourceware.org/bugzilla/show_bug.cgi?id=17252 */ if (raise) { Py_BEGIN_ALLOW_THREADS n = syscall(SYS_getrandom, dest, n, flags); @@ -180,12 +187,12 @@ } if (errno == EAGAIN) { /* If we failed with EAGAIN, the entropy pool was - * uninitialized. In this case, we return failure to fall - * back to reading from /dev/urandom. - * - * Note: In this case the data read will not be random so - * should not be used for cryptographic purposes. Retaining - * the existing semantics for practical purposes. */ + uninitialized. In this case, we return failure to fall + back to reading from /dev/urandom. + + Note: In this case the data read will not be random so + should not be used for cryptographic purposes. Retaining + the existing semantics for practical purposes. */ getrandom_works = 0; return 0; } @@ -221,130 +228,117 @@ } urandom_cache = { -1 }; -/* Read size bytes from /dev/urandom into buffer. - Return 0 success, or return -1 on error. */ +/* Read 'size' random bytes from getrandom(). Fall back on reading from + /dev/urandom if getrandom() is not available. + + Return 0 on success. Raise an exception (if raise is non-zero) and return -1 + on error. */ static int -dev_urandom_noraise(char *buffer, Py_ssize_t size) +dev_urandom(char *buffer, Py_ssize_t size, int raise) { int fd; Py_ssize_t n; - - assert (0 < size); - -#ifdef PY_GETRANDOM - if (py_getrandom(buffer, size, 0) == 1) { - return 0; - } - /* getrandom() is not supported by the running kernel, fall back - * on reading /dev/urandom */ -#endif - - fd = _Py_open_noraise("/dev/urandom", O_RDONLY); - if (fd < 0) { - return -1; - } - - while (0 < size) - { - do { - n = read(fd, buffer, (size_t)size); - } while (n < 0 && errno == EINTR); - - if (n <= 0) { - /* stop on error or if read(size) returned 0 */ - return -1; - } - - buffer += n; - size -= n; - } - close(fd); - - return 0; -} - -/* Read size bytes from /dev/urandom into buffer. - Return 0 on success, raise an exception and return -1 on error. */ -static int -dev_urandom_python(char *buffer, Py_ssize_t size) -{ - int fd; - Py_ssize_t n; - struct _Py_stat_struct st; #ifdef PY_GETRANDOM int res; #endif - if (size <= 0) - return 0; + assert(size > 0); #ifdef PY_GETRANDOM - res = py_getrandom(buffer, size, 1); - if (res < 0) + res = py_getrandom(buffer, size, raise); + if (res < 0) { return -1; - if (res == 1) + } + if (res == 1) { return 0; + } /* getrandom() is not supported by the running kernel, fall back - * on reading /dev/urandom */ + on reading /dev/urandom */ #endif - if (urandom_cache.fd >= 0) { - /* Does the fd point to the same thing as before? (issue #21207) */ - if (_Py_fstat_noraise(urandom_cache.fd, &st) - || st.st_dev != urandom_cache.st_dev - || st.st_ino != urandom_cache.st_ino) { - /* Something changed: forget the cached fd (but don't close it, - since it probably points to something important for some - third-party code). */ - urandom_cache.fd = -1; + + if (raise) { + struct _Py_stat_struct st; + + if (urandom_cache.fd >= 0) { + /* Does the fd point to the same thing as before? (issue #21207) */ + if (_Py_fstat_noraise(urandom_cache.fd, &st) + || st.st_dev != urandom_cache.st_dev + || st.st_ino != urandom_cache.st_ino) { + /* Something changed: forget the cached fd (but don't close it, + since it probably points to something important for some + third-party code). */ + urandom_cache.fd = -1; + } } - } - if (urandom_cache.fd >= 0) - fd = urandom_cache.fd; - else { - fd = _Py_open("/dev/urandom", O_RDONLY); - if (fd < 0) { - if (errno == ENOENT || errno == ENXIO || - errno == ENODEV || errno == EACCES) - PyErr_SetString(PyExc_NotImplementedError, - "/dev/urandom (or equivalent) not found"); - /* otherwise, keep the OSError exception raised by _Py_open() */ - return -1; - } - if (urandom_cache.fd >= 0) { - /* urandom_fd was initialized by another thread while we were - not holding the GIL, keep it. */ - close(fd); + if (urandom_cache.fd >= 0) fd = urandom_cache.fd; - } else { - if (_Py_fstat(fd, &st)) { - close(fd); + fd = _Py_open("/dev/urandom", O_RDONLY); + if (fd < 0) { + if (errno == ENOENT || errno == ENXIO || + errno == ENODEV || errno == EACCES) + PyErr_SetString(PyExc_NotImplementedError, + "/dev/urandom (or equivalent) not found"); + /* otherwise, keep the OSError exception raised by _Py_open() */ return -1; } + if (urandom_cache.fd >= 0) { + /* urandom_fd was initialized by another thread while we were + not holding the GIL, keep it. */ + close(fd); + fd = urandom_cache.fd; + } else { - urandom_cache.fd = fd; - urandom_cache.st_dev = st.st_dev; - urandom_cache.st_ino = st.st_ino; + if (_Py_fstat(fd, &st)) { + close(fd); + return -1; + } + else { + urandom_cache.fd = fd; + urandom_cache.st_dev = st.st_dev; + urandom_cache.st_ino = st.st_ino; + } } } + + do { + n = _Py_read(fd, buffer, (size_t)size); + if (n == -1) + return -1; + if (n == 0) { + PyErr_Format(PyExc_RuntimeError, + "Failed to read %zi bytes from /dev/urandom", + size); + return -1; + } + + buffer += n; + size -= n; + } while (0 < size); } - - do { - n = _Py_read(fd, buffer, (size_t)size); - if (n == -1) - return -1; - if (n == 0) { - PyErr_Format(PyExc_RuntimeError, - "Failed to read %zi bytes from /dev/urandom", - size); + else { + fd = _Py_open_noraise("/dev/urandom", O_RDONLY); + if (fd < 0) { return -1; } - buffer += n; - size -= n; - } while (0 < size); + while (0 < size) + { + do { + n = read(fd, buffer, (size_t)size); + } while (n < 0 && errno == EINTR); + if (n <= 0) { + /* stop on error or if read(size) returned 0 */ + return -1; + } + + buffer += n; + size -= n; + } + close(fd); + } return 0; } @@ -381,10 +375,10 @@ } /* If raise is zero: - * - Don't raise exceptions on error - * - Don't call PyErr_CheckSignals() on EINTR (retry directly the interrupted - * syscall) - * - Don't release the GIL to call syscalls. */ + - Don't raise exceptions on error + - Don't call PyErr_CheckSignals() on EINTR (retry directly the interrupted + syscall) + - Don't release the GIL to call syscalls. */ static int pyurandom(void *buffer, Py_ssize_t size, int raise) { @@ -405,12 +399,7 @@ #elif defined(PY_GETENTROPY) return py_getentropy(buffer, size, raise); #else - if (raise) { - return dev_urandom_python(buffer, size); - } - else { - return dev_urandom_noraise(buffer, size); - } + return dev_urandom(buffer, size, raise); #endif } @@ -466,7 +455,7 @@ int res; /* _PyRandom_Init() is called very early in the Python initialization - * and so exceptions cannot be used. */ + and so exceptions cannot be used (use raise=0). */ res = pyurandom(secret, secret_size, 0); if (res < 0) { Py_FatalError("failed to get random numbers to initialize Python"); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 13:56:06 2016 From: python-checkins at python.org (raymond.hettinger) Date: Tue, 16 Aug 2016 17:56:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325628=3A__Make_na?= =?utf-8?q?medtuple_=22rename=22_and_=22verbose=22_parameters_keyword-only?= =?utf-8?q?=2E?= Message-ID: <20160816175604.24990.71521.57639D6D@psf.io> https://hg.python.org/cpython/rev/f086d6c9d9f6 changeset: 102703:f086d6c9d9f6 user: Raymond Hettinger date: Tue Aug 16 10:55:43 2016 -0700 summary: Issue #25628: Make namedtuple "rename" and "verbose" parameters keyword-only. files: Doc/library/collections.rst | 8 ++++++-- Lib/collections/__init__.py | 2 +- Lib/test/test_collections.py | 12 ++++++++++++ Misc/NEWS | 3 +++ 4 files changed, 22 insertions(+), 3 deletions(-) diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -763,7 +763,7 @@ self-documenting code. They can be used wherever regular tuples are used, and they add the ability to access fields by name instead of position index. -.. function:: namedtuple(typename, field_names, verbose=False, rename=False) +.. function:: namedtuple(typename, field_names, *, verbose=False, rename=False) Returns a new tuple subclass named *typename*. The new subclass is used to create tuple-like objects that have fields accessible by attribute lookup as @@ -799,7 +799,11 @@ a namedtuple. .. versionchanged:: 3.1 - Added support for *rename*. + Added support for *rename*. + + .. versionchanged:: 3.6 + The *verbose* and *rename* parameters became + :ref:`keyword-only arguments `. .. doctest:: diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -353,7 +353,7 @@ {name} = _property(_itemgetter({index:d}), doc='Alias for field number {index:d}') ''' -def namedtuple(typename, field_names, verbose=False, rename=False): +def namedtuple(typename, field_names, *, verbose=False, rename=False): """Returns a new subclass of tuple with named fields. >>> Point = namedtuple('Point', ['x', 'y']) diff --git a/Lib/test/test_collections.py b/Lib/test/test_collections.py --- a/Lib/test/test_collections.py +++ b/Lib/test/test_collections.py @@ -412,6 +412,18 @@ self.assertEqual(NTColor._fields, ('red', 'green', 'blue')) globals().pop('NTColor', None) # clean-up after this test + def test_keyword_only_arguments(self): + # See issue 25628 + with support.captured_stdout() as template: + NT = namedtuple('NT', ['x', 'y'], verbose=True) + self.assertIn('class NT', NT._source) + with self.assertRaises(TypeError): + NT = namedtuple('NT', ['x', 'y'], True) + + NT = namedtuple('NT', ['abc', 'def'], rename=True) + self.assertEqual(NT._fields, ('abc', '_1')) + with self.assertRaises(TypeError): + NT = namedtuple('NT', ['abc', 'def'], False, True) def test_namedtuple_subclass_issue_24931(self): class Point(namedtuple('_Point', ['x', 'y'])): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -73,6 +73,9 @@ to ref count problem introduced in code for Issue #27038 in 3.6.0a3. Patch by Xiang Zhang. +- Issue #25628: The *verbose* and *rename* parameters for collections.namedtuple + are now keyword-only. + - Issue #12345: Add mathemathical constant tau to math and cmath. See also PEP 628. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 13:59:27 2016 From: python-checkins at python.org (raymond.hettinger) Date: Tue, 16 Aug 2016 17:59:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge?= Message-ID: <20160816175926.23914.56598.2E0E1C6F@psf.io> https://hg.python.org/cpython/rev/61403b862793 changeset: 102705:61403b862793 parent: 102703:f086d6c9d9f6 parent: 102704:bcf863c44494 user: Raymond Hettinger date: Tue Aug 16 10:59:21 2016 -0700 summary: merge files: Doc/library/collections.rst | 5 ----- 1 files changed, 0 insertions(+), 5 deletions(-) diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -793,11 +793,6 @@ Named tuple instances do not have per-instance dictionaries, so they are lightweight and require no more memory than regular tuples. - For simple uses, where the only requirement is to be able to refer to a set - of values by name using attribute-style access, the - :class:`types.SimpleNamespace` type can be a suitable alternative to using - a namedtuple. - .. versionchanged:: 3.1 Added support for *rename*. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 13:59:27 2016 From: python-checkins at python.org (raymond.hettinger) Date: Tue, 16 Aug 2016 17:59:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Remove_duplica?= =?utf-8?q?te_of_the_entry_in_the_seealso_section?= Message-ID: <20160816175926.79302.9272.0518BC8F@psf.io> https://hg.python.org/cpython/rev/bcf863c44494 changeset: 102704:bcf863c44494 branch: 3.5 parent: 102697:3ff02380b1bf user: Raymond Hettinger date: Tue Aug 16 10:59:04 2016 -0700 summary: Remove duplicate of the entry in the seealso section files: Doc/library/collections.rst | 5 ----- 1 files changed, 0 insertions(+), 5 deletions(-) diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -793,11 +793,6 @@ Named tuple instances do not have per-instance dictionaries, so they are lightweight and require no more memory than regular tuples. - For simple uses, where the only requirement is to be able to refer to a set - of values by name using attribute-style access, the - :class:`types.SimpleNamespace` type can be a suitable alternative to using - a namedtuple. - .. versionchanged:: 3.1 Added support for *rename*. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 15:29:44 2016 From: python-checkins at python.org (ethan.furman) Date: Tue, 16 Aug 2016 19:29:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_fix_hyperlink?= Message-ID: <20160816192944.23827.59505.DC19EFAD@psf.io> https://hg.python.org/cpython/rev/9d6e0178d210 changeset: 102706:9d6e0178d210 user: Ethan Furman date: Tue Aug 16 12:29:31 2016 -0700 summary: fix hyperlink files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -150,7 +150,7 @@ - Issue 27512: Fix a segfault when os.fspath() called a an __fspath__() method that raised an exception. Patch by Xiang Zhang. -- Issue 26988: Add AutoEnum. +- Issue #26988: Add AutoEnum. Tests ----- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 16:11:11 2016 From: python-checkins at python.org (brett.cannon) Date: Tue, 16 Aug 2016 20:11:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_a_missing_=23?= Message-ID: <20160816201111.79339.37424.6A1F8015@psf.io> https://hg.python.org/cpython/rev/27013b1e9f46 changeset: 102707:27013b1e9f46 user: Brett Cannon date: Tue Aug 16 13:11:00 2016 -0700 summary: Add a missing # files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -147,7 +147,7 @@ - Issue #27522: Avoid an unintentional reference cycle in email.feedparser. -- Issue 27512: Fix a segfault when os.fspath() called a an __fspath__() method +- Issue #27512: Fix a segfault when os.fspath() called a an __fspath__() method that raised an exception. Patch by Xiang Zhang. - Issue #26988: Add AutoEnum. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 16:13:21 2016 From: python-checkins at python.org (raymond.hettinger) Date: Tue, 16 Aug 2016 20:13:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_cross-reference_to_typ?= =?utf-8?q?ing=2ENamedTuple=2E__Doctest_two_more_examples=2E?= Message-ID: <20160816201321.99475.5308.A9328245@psf.io> https://hg.python.org/cpython/rev/e7682f193c96 changeset: 102708:e7682f193c96 user: Raymond Hettinger date: Tue Aug 16 13:13:17 2016 -0700 summary: Add cross-reference to typing.NamedTuple. Doctest two more examples. files: Doc/library/collections.rst | 14 ++++++++++---- 1 files changed, 10 insertions(+), 4 deletions(-) diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -850,7 +850,9 @@ .. method:: somenamedtuple._asdict() Return a new :class:`OrderedDict` which maps field names to their corresponding - values:: + values: + + .. doctest:: >>> p = Point(x=11, y=22) >>> p._asdict() @@ -912,7 +914,9 @@ functionality with a subclass. Here is how to add a calculated field and a fixed-width print format: - >>> class Point(namedtuple('Point', 'x y')): +.. doctest:: + + >>> class Point(namedtuple('Point', ['x', 'y'])): ... __slots__ = () ... @property ... def hypot(self): @@ -963,8 +967,10 @@ constructor that is convenient for use cases where named tuples are being subclassed. - * :meth:`types.SimpleNamespace` for a mutable namespace based on an underlying - dictionary instead of a tuple. + * See :meth:`types.SimpleNamespace` for a mutable namespace based on an + underlying dictionary instead of a tuple. + + * See :meth:`typing.NamedTuple` for a way to add type hints for named tuples. :class:`OrderedDict` objects -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 16:22:55 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 16 Aug 2016 20:22:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327181=3A_Skip_tes?= =?utf-8?q?ts_known_to_fail_until_a_fix_is_found?= Message-ID: <20160816202255.79509.23175.3C923D9A@psf.io> https://hg.python.org/cpython/rev/54288b160243 changeset: 102709:54288b160243 user: Victor Stinner date: Tue Aug 16 22:22:21 2016 +0200 summary: Issue #27181: Skip tests known to fail until a fix is found files: Lib/test/test_statistics.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_statistics.py b/Lib/test/test_statistics.py --- a/Lib/test/test_statistics.py +++ b/Lib/test/test_statistics.py @@ -1053,6 +1053,7 @@ self.assertRaises(TypeError, statistics._product, [Decimal(1), 1.0]) + at unittest.skipIf(True, "FIXME: tests known to fail, see issue #27181") class Test_Nth_Root(NumericTestCase): """Test the functionality of the private _nth_root function.""" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 17:48:57 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 16 Aug 2016 21:48:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327128=3A_Cleanup_?= =?utf-8?q?=5FPyEval=5FEvalCodeWithName=28=29?= Message-ID: <20160816214857.25038.3027.C760122D@psf.io> https://hg.python.org/cpython/rev/288ec55f1912 changeset: 102710:288ec55f1912 user: Victor Stinner date: Tue Aug 16 23:40:29 2016 +0200 summary: Issue #27128: Cleanup _PyEval_EvalCodeWithName() * Add comments * Add empty lines for readability * PEP 7 style for if block * Remove useless assert(globals != NULL); (globals is tested a few lines before) files: Python/ceval.c | 54 ++++++++++++++++++++++++++++++------- 1 files changed, 43 insertions(+), 11 deletions(-) diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -3793,12 +3793,13 @@ PyFrameObject *f; PyObject *retval = NULL; PyObject **fastlocals, **freevars; - PyThreadState *tstate = PyThreadState_GET(); + PyThreadState *tstate; PyObject *x, *u; int total_args = co->co_argcount + co->co_kwonlyargcount; - int i; - int n = argcount; - PyObject *kwdict = NULL; + int i, n; + PyObject *kwdict; + + assert((kwcount == 0) || (kws != NULL)); if (globals == NULL) { PyErr_SetString(PyExc_SystemError, @@ -3806,36 +3807,50 @@ return NULL; } + /* Create the frame */ + tstate = PyThreadState_GET(); assert(tstate != NULL); - assert(globals != NULL); f = PyFrame_New(tstate, co, globals, locals); - if (f == NULL) + if (f == NULL) { return NULL; - + } fastlocals = f->f_localsplus; freevars = f->f_localsplus + co->co_nlocals; - /* Parse arguments. */ + /* Create a dictionary for keyword parameters (**kwags) */ if (co->co_flags & CO_VARKEYWORDS) { kwdict = PyDict_New(); if (kwdict == NULL) goto fail; i = total_args; - if (co->co_flags & CO_VARARGS) + if (co->co_flags & CO_VARARGS) { i++; + } SETLOCAL(i, kwdict); } - if (argcount > co->co_argcount) + else { + kwdict = NULL; + } + + /* Copy positional arguments into local variables */ + if (argcount > co->co_argcount) { n = co->co_argcount; + } + else { + n = argcount; + } for (i = 0; i < n; i++) { x = args[i]; Py_INCREF(x); SETLOCAL(i, x); } + + /* Pack other positional arguments into the *args argument */ if (co->co_flags & CO_VARARGS) { u = PyTuple_New(argcount - n); - if (u == NULL) + if (u == NULL) { goto fail; + } SETLOCAL(total_args, u); for (i = n; i < argcount; i++) { x = args[i]; @@ -3843,17 +3858,21 @@ PyTuple_SET_ITEM(u, i-n, x); } } + + /* Handle keyword arguments (passed as an array of (key, value)) */ for (i = 0; i < kwcount; i++) { PyObject **co_varnames; PyObject *keyword = kws[2*i]; PyObject *value = kws[2*i + 1]; int j; + if (keyword == NULL || !PyUnicode_Check(keyword)) { PyErr_Format(PyExc_TypeError, "%U() keywords must be strings", co->co_name); goto fail; } + /* Speed hack: do raw pointer compares. As names are normally interned this should almost always hit. */ co_varnames = ((PyTupleObject *)(co->co_varnames))->ob_item; @@ -3862,6 +3881,7 @@ if (nm == keyword) goto kw_found; } + /* Slow fallback, just in case */ for (j = 0; j < total_args; j++) { PyObject *nm = co_varnames[j]; @@ -3872,6 +3892,7 @@ else if (cmp < 0) goto fail; } + if (j >= total_args && kwdict == NULL) { PyErr_Format(PyExc_TypeError, "%U() got an unexpected " @@ -3880,10 +3901,12 @@ keyword); goto fail; } + if (PyDict_SetItem(kwdict, keyword, value) == -1) { goto fail; } continue; + kw_found: if (GETLOCAL(j) != NULL) { PyErr_Format(PyExc_TypeError, @@ -3896,10 +3919,14 @@ Py_INCREF(value); SETLOCAL(j, value); } + + /* Check the number of positional arguments */ if (argcount > co->co_argcount && !(co->co_flags & CO_VARARGS)) { too_many_positional(co, argcount, defcount, fastlocals); goto fail; } + + /* Add missing positional arguments (copy default values from defs) */ if (argcount < co->co_argcount) { int m = co->co_argcount - defcount; int missing = 0; @@ -3922,6 +3949,8 @@ } } } + + /* Add missing keyword arguments (copy default values from kwdefs) */ if (co->co_kwonlyargcount > 0) { int missing = 0; for (i = co->co_argcount; i < total_args; i++) { @@ -3964,12 +3993,15 @@ goto fail; SETLOCAL(co->co_nlocals + i, c); } + + /* Copy closure variables to free variables */ for (i = 0; i < PyTuple_GET_SIZE(co->co_freevars); ++i) { PyObject *o = PyTuple_GET_ITEM(closure, i); Py_INCREF(o); freevars[PyTuple_GET_SIZE(co->co_cellvars) + i] = o; } + /* Handle generator/coroutine */ if (co->co_flags & (CO_GENERATOR | CO_COROUTINE)) { PyObject *gen; PyObject *coro_wrapper = tstate->coroutine_wrapper; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 16 17:48:57 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 16 Aug 2016 21:48:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Use_Py=5Fssize=5Ft_in_=5FP?= =?utf-8?q?yEval=5FEvalCodeWithName=28=29?= Message-ID: <20160816214857.28840.94041.54D96338@psf.io> https://hg.python.org/cpython/rev/e615718a6455 changeset: 102711:e615718a6455 user: Victor Stinner date: Tue Aug 16 23:39:42 2016 +0200 summary: Use Py_ssize_t in _PyEval_EvalCodeWithName() Issue #27128, #18295: replace int type with Py_ssize_t for index variables used for positional arguments. It should help to avoid integer overflow and help to emit better machine code for "i++" (no trap needed for overflow). Make also the total_args variable constant. files: Python/ceval.c | 18 ++++++++++-------- 1 files changed, 10 insertions(+), 8 deletions(-) diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -3795,8 +3795,8 @@ PyObject **fastlocals, **freevars; PyThreadState *tstate; PyObject *x, *u; - int total_args = co->co_argcount + co->co_kwonlyargcount; - int i, n; + const Py_ssize_t total_args = co->co_argcount + co->co_kwonlyargcount; + Py_ssize_t i, n; PyObject *kwdict; assert((kwcount == 0) || (kws != NULL)); @@ -3864,7 +3864,7 @@ PyObject **co_varnames; PyObject *keyword = kws[2*i]; PyObject *value = kws[2*i + 1]; - int j; + Py_ssize_t j; if (keyword == NULL || !PyUnicode_Check(keyword)) { PyErr_Format(PyExc_TypeError, @@ -3928,11 +3928,13 @@ /* Add missing positional arguments (copy default values from defs) */ if (argcount < co->co_argcount) { - int m = co->co_argcount - defcount; - int missing = 0; - for (i = argcount; i < m; i++) - if (GETLOCAL(i) == NULL) + Py_ssize_t m = co->co_argcount - defcount; + Py_ssize_t missing = 0; + for (i = argcount; i < m; i++) { + if (GETLOCAL(i) == NULL) { missing++; + } + } if (missing) { missing_arguments(co, missing, defcount, fastlocals); goto fail; @@ -3952,7 +3954,7 @@ /* Add missing keyword arguments (copy default values from kwdefs) */ if (co->co_kwonlyargcount > 0) { - int missing = 0; + Py_ssize_t missing = 0; for (i = co->co_argcount; i < total_args; i++) { PyObject *name; if (GETLOCAL(i) != NULL) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 02:37:56 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 17 Aug 2016 06:37:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_rearrange_meth?= =?utf-8?q?odcaller=5Fnew_so_that_the_main_error_case_does_not_cause?= Message-ID: <20160817063756.23360.62488.EE1D5301@psf.io> https://hg.python.org/cpython/rev/8e3b9bf917a7 changeset: 102713:8e3b9bf917a7 branch: 3.3 parent: 102689:cbf2a05648b3 user: Benjamin Peterson date: Tue Aug 16 23:35:35 2016 -0700 summary: rearrange methodcaller_new so that the main error case does not cause uninitialized memory usage (closes #27783) files: Misc/NEWS | 2 ++ Modules/operator.c | 15 +++++++-------- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,8 @@ Library ------- +- Issue #27783: Fix possible usage of uninitialized memory in operator.methodcaller. + - Issue #27774: Fix possible Py_DECREF on unowned object in _sre. - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. diff --git a/Modules/operator.c b/Modules/operator.c --- a/Modules/operator.c +++ b/Modules/operator.c @@ -776,7 +776,7 @@ methodcaller_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { methodcallerobject *mc; - PyObject *name, *newargs; + PyObject *name; if (PyTuple_GET_SIZE(args) < 1) { PyErr_SetString(PyExc_TypeError, "methodcaller needs at least " @@ -789,13 +789,6 @@ if (mc == NULL) return NULL; - newargs = PyTuple_GetSlice(args, 1, PyTuple_GET_SIZE(args)); - if (newargs == NULL) { - Py_DECREF(mc); - return NULL; - } - mc->args = newargs; - name = PyTuple_GET_ITEM(args, 0); Py_INCREF(name); mc->name = name; @@ -803,6 +796,12 @@ Py_XINCREF(kwds); mc->kwds = kwds; + mc->args = PyTuple_GetSlice(args, 1, PyTuple_GET_SIZE(args)); + if (mc->args == NULL) { + Py_DECREF(mc); + return NULL; + } + PyObject_GC_Track(mc); return (PyObject *)mc; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 02:37:57 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 17 Aug 2016 06:37:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_merge_3=2E3_=28=2327783=29?= Message-ID: <20160817063756.23731.31502.8D8D6F04@psf.io> https://hg.python.org/cpython/rev/675e20c38fda changeset: 102714:675e20c38fda branch: 3.4 parent: 102690:2e404ac88e0e parent: 102713:8e3b9bf917a7 user: Benjamin Peterson date: Tue Aug 16 23:36:20 2016 -0700 summary: merge 3.3 (#27783) files: Misc/NEWS | 2 ++ Modules/_operator.c | 15 +++++++-------- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -16,6 +16,8 @@ - In the curses module, raise an error if window.getstr() or window.instr() is passed a negative value. +- Issue #27783: Fix possible usage of uninitialized memory in operator.methodcaller. + - Issue #27774: Fix possible Py_DECREF on unowned object in _sre. - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. diff --git a/Modules/_operator.c b/Modules/_operator.c --- a/Modules/_operator.c +++ b/Modules/_operator.c @@ -801,7 +801,7 @@ methodcaller_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { methodcallerobject *mc; - PyObject *name, *newargs; + PyObject *name; if (PyTuple_GET_SIZE(args) < 1) { PyErr_SetString(PyExc_TypeError, "methodcaller needs at least " @@ -814,13 +814,6 @@ if (mc == NULL) return NULL; - newargs = PyTuple_GetSlice(args, 1, PyTuple_GET_SIZE(args)); - if (newargs == NULL) { - Py_DECREF(mc); - return NULL; - } - mc->args = newargs; - name = PyTuple_GET_ITEM(args, 0); Py_INCREF(name); mc->name = name; @@ -828,6 +821,12 @@ Py_XINCREF(kwds); mc->kwds = kwds; + mc->args = PyTuple_GetSlice(args, 1, PyTuple_GET_SIZE(args)); + if (mc->args == NULL) { + Py_DECREF(mc); + return NULL; + } + PyObject_GC_Track(mc); return (PyObject *)mc; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 02:37:57 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 17 Aug 2016 06:37:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjc3ODMp?= Message-ID: <20160817063757.28109.47479.18FCE36F@psf.io> https://hg.python.org/cpython/rev/0f0a040d45b2 changeset: 102716:0f0a040d45b2 parent: 102711:e615718a6455 parent: 102715:d1b93ce7dad8 user: Benjamin Peterson date: Tue Aug 16 23:37:42 2016 -0700 summary: merge 3.5 (#27783) files: Misc/NEWS | 2 ++ Modules/_operator.c | 16 ++++++++-------- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -110,6 +110,8 @@ - In the curses module, raise an error if window.getstr() or window.instr() is passed a negative value. +- Issue #27783: Fix possible usage of uninitialized memory in operator.methodcaller. + - Issue #27774: Fix possible Py_DECREF on unowned object in _sre. - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. diff --git a/Modules/_operator.c b/Modules/_operator.c --- a/Modules/_operator.c +++ b/Modules/_operator.c @@ -931,7 +931,7 @@ methodcaller_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { methodcallerobject *mc; - PyObject *name, *newargs; + PyObject *name; if (PyTuple_GET_SIZE(args) < 1) { PyErr_SetString(PyExc_TypeError, "methodcaller needs at least " @@ -951,13 +951,7 @@ if (mc == NULL) return NULL; - newargs = PyTuple_GetSlice(args, 1, PyTuple_GET_SIZE(args)); - if (newargs == NULL) { - Py_DECREF(mc); - return NULL; - } - mc->args = newargs; - + name = PyTuple_GET_ITEM(args, 0); Py_INCREF(name); PyUnicode_InternInPlace(&name); mc->name = name; @@ -965,6 +959,12 @@ Py_XINCREF(kwds); mc->kwds = kwds; + mc->args = PyTuple_GetSlice(args, 1, PyTuple_GET_SIZE(args)); + if (mc->args == NULL) { + Py_DECREF(mc); + return NULL; + } + PyObject_GC_Track(mc); return (PyObject *)mc; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 02:37:57 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 17 Aug 2016 06:37:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4_=28=2327783=29?= Message-ID: <20160817063757.23348.7725.F323B0A4@psf.io> https://hg.python.org/cpython/rev/d1b93ce7dad8 changeset: 102715:d1b93ce7dad8 branch: 3.5 parent: 102704:bcf863c44494 parent: 102714:675e20c38fda user: Benjamin Peterson date: Tue Aug 16 23:37:33 2016 -0700 summary: merge 3.4 (#27783) files: Misc/NEWS | 2 ++ Modules/_operator.c | 16 ++++++++-------- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -42,6 +42,8 @@ - In the curses module, raise an error if window.getstr() or window.instr() is passed a negative value. +- Issue #27783: Fix possible usage of uninitialized memory in operator.methodcaller. + - Issue #27774: Fix possible Py_DECREF on unowned object in _sre. - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. diff --git a/Modules/_operator.c b/Modules/_operator.c --- a/Modules/_operator.c +++ b/Modules/_operator.c @@ -931,7 +931,7 @@ methodcaller_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { methodcallerobject *mc; - PyObject *name, *newargs; + PyObject *name; if (PyTuple_GET_SIZE(args) < 1) { PyErr_SetString(PyExc_TypeError, "methodcaller needs at least " @@ -951,13 +951,7 @@ if (mc == NULL) return NULL; - newargs = PyTuple_GetSlice(args, 1, PyTuple_GET_SIZE(args)); - if (newargs == NULL) { - Py_DECREF(mc); - return NULL; - } - mc->args = newargs; - + name = PyTuple_GET_ITEM(args, 0); Py_INCREF(name); PyUnicode_InternInPlace(&name); mc->name = name; @@ -965,6 +959,12 @@ Py_XINCREF(kwds); mc->kwds = kwds; + mc->args = PyTuple_GetSlice(args, 1, PyTuple_GET_SIZE(args)); + if (mc->args == NULL) { + Py_DECREF(mc); + return NULL; + } + PyObject_GC_Track(mc); return (PyObject *)mc; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 02:37:57 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 17 Aug 2016 06:37:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_rearrange_meth?= =?utf-8?q?odcaller=5Fnew_so_that_the_main_error_case_does_not_cause?= Message-ID: <20160817063756.78577.23271.F3EEBA57@psf.io> https://hg.python.org/cpython/rev/11a9bca71528 changeset: 102712:11a9bca71528 branch: 2.7 parent: 102694:cddd633b959f user: Benjamin Peterson date: Tue Aug 16 23:35:35 2016 -0700 summary: rearrange methodcaller_new so that the main error case does not cause uninitialized memory usage (closes #27783) files: Misc/NEWS | 2 ++ Modules/operator.c | 15 +++++++-------- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,8 @@ Library ------- +- Issue #27783: Fix possible usage of uninitialized memory in operator.methodcaller. + - Issue #27774: Fix possible Py_DECREF on unowned object in _sre. - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. diff --git a/Modules/operator.c b/Modules/operator.c --- a/Modules/operator.c +++ b/Modules/operator.c @@ -788,7 +788,7 @@ methodcaller_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { methodcallerobject *mc; - PyObject *name, *newargs; + PyObject *name; if (PyTuple_GET_SIZE(args) < 1) { PyErr_SetString(PyExc_TypeError, "methodcaller needs at least " @@ -801,13 +801,6 @@ if (mc == NULL) return NULL; - newargs = PyTuple_GetSlice(args, 1, PyTuple_GET_SIZE(args)); - if (newargs == NULL) { - Py_DECREF(mc); - return NULL; - } - mc->args = newargs; - name = PyTuple_GET_ITEM(args, 0); Py_INCREF(name); mc->name = name; @@ -815,6 +808,12 @@ Py_XINCREF(kwds); mc->kwds = kwds; + mc->args = PyTuple_GetSlice(args, 1, PyTuple_GET_SIZE(args)); + if (mc->args == NULL) { + Py_DECREF(mc); + return NULL; + } + PyObject_GC_Track(mc); return (PyObject *)mc; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 03:47:08 2016 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 17 Aug 2016 07:47:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Minor_readability_tweak?= Message-ID: <20160817074702.78476.54201.90BCC11D@psf.io> https://hg.python.org/cpython/rev/576128ef551b changeset: 102717:576128ef551b user: Raymond Hettinger date: Wed Aug 17 00:46:48 2016 -0700 summary: Minor readability tweak files: Lib/collections/__init__.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -393,7 +393,7 @@ field_names[index] = '_%d' % index seen.add(name) for name in [typename] + field_names: - if type(name) != str: + if type(name) is not str: raise TypeError('Type names and field names must be strings') if not name.isidentifier(): raise ValueError('Type names and field names must be valid ' -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 06:25:04 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 10:25:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_typo_in_test=5Ftime=2E?= =?utf-8?q?py?= Message-ID: <20160817102503.11934.5932.35581BC1@psf.io> https://hg.python.org/cpython/rev/4b3a9bb1994b changeset: 102718:4b3a9bb1994b user: Victor Stinner date: Wed Aug 17 11:07:21 2016 +0200 summary: Fix typo in test_time.py files: Lib/test/test_time.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -729,7 +729,7 @@ for seconds in (_testcapi.INT_MIN, _testcapi.INT_MAX): ns_timestamps.append(seconds * SEC_TO_NS) if use_float: - # numbers with an extract representation in IEEE 754 (base 2) + # numbers with an exact representation in IEEE 754 (base 2) for pow2 in (3, 7, 10, 15): ns = 2.0 ** (-pow2) ns_timestamps.extend((-ns, ns)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 06:25:04 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 10:25:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Tests=3A_add_--slowest_opt?= =?utf-8?q?ion_to_buildbots?= Message-ID: <20160817102503.23216.28559.FC6E9229@psf.io> https://hg.python.org/cpython/rev/f12b226620e6 changeset: 102720:f12b226620e6 user: Victor Stinner date: Wed Aug 17 11:27:40 2016 +0200 summary: Tests: add --slowest option to buildbots Display the top 10 slowest tests. files: Makefile.pre.in | 2 +- Tools/buildbot/test.bat | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1018,7 +1018,7 @@ - at if which pybuildbot.identify >/dev/null 2>&1; then \ pybuildbot.identify "CC='$(CC)'" "CXX='$(CXX)'"; \ fi - $(TESTRUNNER) -j 1 -u all -W --timeout=$(TESTTIMEOUT) $(TESTOPTS) + $(TESTRUNNER) -j 1 -u all -W --slowest --timeout=$(TESTTIMEOUT) $(TESTOPTS) QUICKTESTOPTS= $(TESTOPTS) -x test_subprocess test_io test_lib2to3 \ test_multibytecodec test_urllib2_localnet test_itertools \ diff --git a/Tools/buildbot/test.bat b/Tools/buildbot/test.bat --- a/Tools/buildbot/test.bat +++ b/Tools/buildbot/test.bat @@ -16,4 +16,4 @@ if NOT "%1"=="" (set regrtest_args=%regrtest_args% %1) & shift & goto CheckOpts echo on -call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW --timeout=900 %regrtest_args% +call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW --slowest --timeout=900 %regrtest_args% -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 06:25:04 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 10:25:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_regrtest=3A_nicer_output_f?= =?utf-8?q?or_durations?= Message-ID: <20160817102503.23360.46145.72F37DFF@psf.io> https://hg.python.org/cpython/rev/61d19015921c changeset: 102721:61d19015921c user: Victor Stinner date: Wed Aug 17 12:22:52 2016 +0200 summary: regrtest: nicer output for durations Use milliseconds and minutes units, not only seconds. files: Lib/test/libregrtest/main.py | 31 ++++++++++++++--------- Lib/test/test_regrtest.py | 4 +- 2 files changed, 21 insertions(+), 14 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -34,6 +34,16 @@ TEMPDIR = os.path.abspath(TEMPDIR) +def format_duration(seconds): + if seconds < 1.0: + return '%.0f ms' % (seconds * 1e3) + if seconds < 60.0: + return '%.0f sec' % seconds + + minutes, seconds = divmod(seconds, 60.0) + return '%.0f min %.0f sec' % (minutes, seconds) + + class Regrtest: """Execute a test suite. @@ -107,14 +117,6 @@ self.skipped.append(test) self.resource_denieds.append(test) - def time_delta(self, ceil=False): - seconds = time.monotonic() - self.start_time - if ceil: - seconds = math.ceil(seconds) - else: - seconds = int(seconds) - return datetime.timedelta(seconds=seconds) - def display_progress(self, test_index, test): if self.ns.quiet: return @@ -122,12 +124,14 @@ fmt = "{time} [{test_index:{count_width}}{test_count}/{nbad}] {test_name}" else: fmt = "{time} [{test_index:{count_width}}{test_count}] {test_name}" + test_time = time.monotonic() - self.start_time + test_time = datetime.timedelta(seconds=int(test_time)) line = fmt.format(count_width=self.test_count_width, test_index=test_index, test_count=self.test_count, nbad=len(self.bad), test_name=test, - time=self.time_delta()) + time=test_time) print(line, flush=True) def parse_args(self, kwargs): @@ -286,9 +290,10 @@ if self.ns.print_slow: self.test_times.sort(reverse=True) + print() print("10 slowest tests:") for time, test in self.test_times[:10]: - print("%s: %.1fs" % (test, time)) + print("- %s: %s" % (test, format_duration(time))) if self.bad: print(count(len(self.bad), "test"), "failed:") @@ -342,7 +347,7 @@ previous_test = format_test_result(test, result[0]) test_time = time.monotonic() - start_time if test_time >= PROGRESS_MIN_TIME: - previous_test = "%s in %.0f sec" % (previous_test, test_time) + previous_test = "%s in %s" % (previous_test, format_duration(test_time)) elif result[0] == PASSED: # be quiet: say nothing if the test passed shortly previous_test = None @@ -418,7 +423,9 @@ r.write_results(show_missing=True, summary=True, coverdir=self.ns.coverdir) - print("Total duration: %s" % self.time_delta(ceil=True)) + print() + duration = time.monotonic() - self.start_time + print("Total duration: %s" % format_duration(duration)) if self.ns.runleaks: os.system("leaks %d" % os.getpid()) diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -660,13 +660,13 @@ output = self.run_tests(test, exitcode=1) self.check_executed_tests(output, test, omitted=test) - def test_slow(self): + def test_slowest(self): # test --slowest tests = [self.create_test() for index in range(3)] output = self.run_tests("--slowest", *tests) self.check_executed_tests(output, tests) regex = ('10 slowest tests:\n' - '(?:%s: [0-9]+\.[0-9]+s\n){%s}' + '(?:- %s: .*\n){%s}' % (self.TESTNAME_REGEX, len(tests))) self.check_line(output, regex) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 06:25:05 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 10:25:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_regrtest=3A_rename_--slow_?= =?utf-8?q?option_to_--slowest?= Message-ID: <20160817102503.86069.36539.5A5BD035@psf.io> https://hg.python.org/cpython/rev/57aacfa9749c changeset: 102719:57aacfa9749c user: Victor Stinner date: Wed Aug 17 11:25:43 2016 +0200 summary: regrtest: rename --slow option to --slowest Thanks to optparse, --slow syntax still works ;-) files: Lib/test/libregrtest/cmdline.py | 2 +- Lib/test/test_regrtest.py | 12 ++++++------ 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Lib/test/libregrtest/cmdline.py b/Lib/test/libregrtest/cmdline.py --- a/Lib/test/libregrtest/cmdline.py +++ b/Lib/test/libregrtest/cmdline.py @@ -168,7 +168,7 @@ help='display test output on failure') group.add_argument('-q', '--quiet', action='store_true', help='no output unless one or more tests fail') - group.add_argument('-o', '--slow', action='store_true', dest='print_slow', + group.add_argument('-o', '--slowest', action='store_true', dest='print_slow', help='print the slowest 10 tests') group.add_argument('--header', action='store_true', help='print header with interpreter info') diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -109,7 +109,7 @@ self.assertEqual(ns.verbose, 0) def test_slow(self): - for opt in '-o', '--slow': + for opt in '-o', '--slowest': with self.subTest(opt=opt): ns = libregrtest._parse_args([opt]) self.assertTrue(ns.print_slow) @@ -661,9 +661,9 @@ self.check_executed_tests(output, test, omitted=test) def test_slow(self): - # test --slow + # test --slowest tests = [self.create_test() for index in range(3)] - output = self.run_tests("--slow", *tests) + output = self.run_tests("--slowest", *tests) self.check_executed_tests(output, tests) regex = ('10 slowest tests:\n' '(?:%s: [0-9]+\.[0-9]+s\n){%s}' @@ -671,15 +671,15 @@ self.check_line(output, regex) def test_slow_interrupted(self): - # Issue #25373: test --slow with an interrupted test + # Issue #25373: test --slowest with an interrupted test code = TEST_INTERRUPTED test = self.create_test("sigint", code=code) for multiprocessing in (False, True): if multiprocessing: - args = ("--slow", "-j2", test) + args = ("--slowest", "-j2", test) else: - args = ("--slow", test) + args = ("--slowest", test) output = self.run_tests(*args, exitcode=1) self.check_executed_tests(output, test, omitted=test) regex = ('10 slowest tests:\n') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 06:45:33 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 10:45:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_=28script=5Fhelper=29?= Message-ID: <20160817104533.23198.47052.8C5A5A2A@psf.io> https://hg.python.org/cpython/rev/2f5c37f4814d changeset: 102723:2f5c37f4814d parent: 102721:61d19015921c parent: 102722:242c89ab09fe user: Victor Stinner date: Wed Aug 17 12:38:35 2016 +0200 summary: Merge 3.5 (script_helper) files: Lib/test/support/script_helper.py | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Lib/test/support/script_helper.py b/Lib/test/support/script_helper.py --- a/Lib/test/support/script_helper.py +++ b/Lib/test/support/script_helper.py @@ -83,16 +83,16 @@ env = {} env.update(env_vars) cmd_line.extend(args) - p = subprocess.Popen(cmd_line, stdin=subprocess.PIPE, + proc = subprocess.Popen(cmd_line, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env) - try: - out, err = p.communicate() - finally: - subprocess._cleanup() - p.stdout.close() - p.stderr.close() - rc = p.returncode + with proc: + try: + out, err = proc.communicate() + finally: + proc.kill() + subprocess._cleanup() + rc = proc.returncode err = strip_python_stderr(err) return _PythonRunResult(rc, out, err), cmd_line -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 06:46:51 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 10:46:51 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogc2NyaXB0X2hlbHBl?= =?utf-8?q?r=3A_kill_the_subprocess_on_error?= Message-ID: <20160817104533.92516.13327.E11551C6@psf.io> https://hg.python.org/cpython/rev/242c89ab09fe changeset: 102722:242c89ab09fe branch: 3.5 parent: 102715:d1b93ce7dad8 user: Victor Stinner date: Wed Aug 17 12:29:58 2016 +0200 summary: script_helper: kill the subprocess on error If Popen.communicate() raises an exception, kill the child process to not leave a running child process in background and maybe create a zombi process. This change fixes a ResourceWarning in Python 3.6 when unit tests are interrupted by CTRL+c. files: Lib/test/support/script_helper.py | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Lib/test/support/script_helper.py b/Lib/test/support/script_helper.py --- a/Lib/test/support/script_helper.py +++ b/Lib/test/support/script_helper.py @@ -83,16 +83,16 @@ env = {} env.update(env_vars) cmd_line.extend(args) - p = subprocess.Popen(cmd_line, stdin=subprocess.PIPE, + proc = subprocess.Popen(cmd_line, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env) - try: - out, err = p.communicate() - finally: - subprocess._cleanup() - p.stdout.close() - p.stderr.close() - rc = p.returncode + with proc: + try: + out, err = proc.communicate() + finally: + proc.kill() + subprocess._cleanup() + rc = proc.returncode err = strip_python_stderr(err) return _PythonRunResult(rc, out, err), cmd_line -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 07:53:04 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 11:53:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=22make_tags=22=3A_remove_?= =?utf-8?q?-t_option_of_ctags?= Message-ID: <20160817115304.85988.60554.CA7D14FF@psf.io> https://hg.python.org/cpython/rev/8455902b3b8e changeset: 102724:8455902b3b8e user: Victor Stinner date: Wed Aug 17 13:51:52 2016 +0200 summary: "make tags": remove -t option of ctags The option was kept for backward compatibility, but it was completly removed recently. Patch written by St?phane Wirtel. files: Makefile.pre.in | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1563,8 +1563,8 @@ # Create a tags file for vi tags:: cd $(srcdir); \ - ctags -w -t Include/*.h; \ - for i in $(SRCDIRS); do ctags -w -t -a $$i/*.[ch]; \ + ctags -w Include/*.h; \ + for i in $(SRCDIRS); do ctags -w -a $$i/*.[ch]; \ done; \ sort -o tags tags -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 07:59:21 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 11:59:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_=22make_tags=22=3A_set?= =?utf-8?q?_locale_to_C_to_call_sort?= Message-ID: <20160817115921.86127.79786.41F2D833@psf.io> https://hg.python.org/cpython/rev/cbe87a9a0cf2 changeset: 102725:cbe87a9a0cf2 user: Victor Stinner date: Wed Aug 17 13:58:12 2016 +0200 summary: Fix "make tags": set locale to C to call sort vim expects that the tags file is sorted using english collation, so it fails if the locale is french for example. Use LC_ALL=C to force english sorting order. Issue #27726. files: Makefile.pre.in | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1566,7 +1566,7 @@ ctags -w Include/*.h; \ for i in $(SRCDIRS); do ctags -w -a $$i/*.[ch]; \ done; \ - sort -o tags tags + LC_ALL=C sort -o tags tags # Create a tags file for GNU Emacs TAGS:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 08:04:33 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 12:04:33 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NzI2?= =?utf-8?q?=3A_Fix_=22make_tags=22?= Message-ID: <20160817120432.78476.61306.362D92AD@psf.io> https://hg.python.org/cpython/rev/ed95e9ca2699 changeset: 102726:ed95e9ca2699 branch: 3.5 parent: 102722:242c89ab09fe user: Victor Stinner date: Wed Aug 17 14:00:58 2016 +0200 summary: Issue #27726: Fix "make tags" * Memove -t option of ctags. The option was kept for backward compatibility, but it was completly removed recently. Patch written by St?phane Wirtel. * Set locale to C to call sort. vim expects that the tags file is sorted using english collation, so it fails if the locale is french for example. Use LC_ALL=C to force english sorting order. . files: Makefile.pre.in | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1548,10 +1548,10 @@ # Create a tags file for vi tags:: cd $(srcdir); \ - ctags -w -t Include/*.h; \ - for i in $(SRCDIRS); do ctags -w -t -a $$i/*.[ch]; \ + ctags -w Include/*.h; \ + for i in $(SRCDIRS); do ctags -w -a $$i/*.[ch]; \ done; \ - sort -o tags tags + LC_ALL=C sort -o tags tags # Create a tags file for GNU Emacs TAGS:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 08:04:42 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 12:04:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Null_merge?= Message-ID: <20160817120432.16330.24007.85859501@psf.io> https://hg.python.org/cpython/rev/2fce793ec0a2 changeset: 102727:2fce793ec0a2 parent: 102725:cbe87a9a0cf2 parent: 102726:ed95e9ca2699 user: Victor Stinner date: Wed Aug 17 14:04:19 2016 +0200 summary: Null merge files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 10:14:18 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 14:14:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3Njk4?= =?utf-8?q?=3A_Add_socketpair_to_socket=2E=5F=5Fall=5F=5F_on_Windows?= Message-ID: <20160817141415.28198.63805.8F33B99A@psf.io> https://hg.python.org/cpython/rev/ac2bc921169c changeset: 102728:ac2bc921169c branch: 3.5 parent: 102726:ed95e9ca2699 user: Victor Stinner date: Wed Aug 17 14:40:08 2016 +0200 summary: Issue #27698: Add socketpair to socket.__all__ on Windows files: Lib/socket.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/socket.py b/Lib/socket.py --- a/Lib/socket.py +++ b/Lib/socket.py @@ -519,6 +519,7 @@ finally: lsock.close() return (ssock, csock) + __all__.append("socketpair") socketpair.__doc__ = """socketpair([family[, type[, proto]]]) -> (socket object, socket object) Create a pair of socket objects from the sockets returned by the platform -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 10:14:18 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 14:14:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_regrtest=3A_add_newlines_i?= =?utf-8?q?n_output_for_readability?= Message-ID: <20160817141416.23552.84065.AD3A0E0A@psf.io> https://hg.python.org/cpython/rev/e5fb95b9a596 changeset: 102730:e5fb95b9a596 user: Victor Stinner date: Wed Aug 17 15:42:21 2016 +0200 summary: regrtest: add newlines in output for readability files: Lib/test/libregrtest/main.py | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -296,15 +296,18 @@ print("- %s: %s" % (test, format_duration(time))) if self.bad: + print() print(count(len(self.bad), "test"), "failed:") printlist(self.bad) if self.environment_changed: + print() print("{} altered the execution environment:".format( count(len(self.environment_changed), "test"))) printlist(self.environment_changed) if self.skipped and not self.ns.quiet: + print() print(count(len(self.skipped), "test"), "skipped:") printlist(self.skipped) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 10:14:18 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 14:14:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_regrtest=3A_set_interrupte?= =?utf-8?q?d_to_True_if_re-run_is_interrupted?= Message-ID: <20160817141418.72579.19615.2D70F623@psf.io> https://hg.python.org/cpython/rev/11571aa83ea1 changeset: 102731:11571aa83ea1 user: Victor Stinner date: Wed Aug 17 16:00:12 2016 +0200 summary: regrtest: set interrupted to True if re-run is interrupted files: Lib/test/libregrtest/main.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -254,6 +254,7 @@ self.ns.verbose = True ok = runtest(self.ns, test) except KeyboardInterrupt: + self.interrupted = True # print a newline separate from the ^C print() break @@ -341,8 +342,8 @@ try: result = runtest(self.ns, test) except KeyboardInterrupt: + self.interrupted = True self.accumulate_result(test, (INTERRUPTED, None)) - self.interrupted = True break else: self.accumulate_result(test, result) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 10:14:19 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 14:14:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_regrtest=3A_add_a_summary_?= =?utf-8?q?of_the_summary=2C_=22Result=3A_xxx=22?= Message-ID: <20160817141418.85992.83499.A954F758@psf.io> https://hg.python.org/cpython/rev/14a0ba0a71d2 changeset: 102732:14a0ba0a71d2 user: Victor Stinner date: Wed Aug 17 16:12:16 2016 +0200 summary: regrtest: add a summary of the summary, "Result: xxx" It's sometimes hard to check quickly if tests succeeded, failed or something bad happened. I added a final "Result: xxx" line which summarizes all outputs into a single line, written at the end (it should always be the last line of the output). files: Lib/test/libregrtest/main.py | 8 ++++++++ Lib/test/test_regrtest.py | 21 +++++++++++++++++---- 2 files changed, 25 insertions(+), 4 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -431,6 +431,14 @@ duration = time.monotonic() - self.start_time print("Total duration: %s" % format_duration(duration)) + if self.bad: + result = "FAILURE" + elif self.interrupted: + result = "INTERRUPTED" + else: + result = "SUCCESS" + print("Result: %s" % result) + if self.ns.runleaks: os.system("leaks %d" % os.getpid()) diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -349,7 +349,7 @@ return list(match.group(1) for match in parser) def check_executed_tests(self, output, tests, skipped=(), failed=(), - omitted=(), randomize=False): + omitted=(), randomize=False, interrupted=False): if isinstance(tests, str): tests = [tests] if isinstance(skipped, str): @@ -398,6 +398,17 @@ regex = 'All %s' % regex self.check_line(output, regex) + if interrupted: + self.check_line(output, 'Test suite interrupted by signal SIGINT.') + + if nfailed: + result = 'FAILURE' + elif interrupted: + result = 'INTERRUPTED' + else: + result = 'SUCCESS' + self.check_line(output, 'Result: %s' % result) + def parse_random_seed(self, output): match = self.regex_search(r'Using random seed ([0-9]+)', output) randseed = int(match.group(1)) @@ -658,7 +669,8 @@ code = TEST_INTERRUPTED test = self.create_test('sigint', code=code) output = self.run_tests(test, exitcode=1) - self.check_executed_tests(output, test, omitted=test) + self.check_executed_tests(output, test, omitted=test, + interrupted=True) def test_slowest(self): # test --slowest @@ -681,10 +693,11 @@ else: args = ("--slowest", test) output = self.run_tests(*args, exitcode=1) - self.check_executed_tests(output, test, omitted=test) + self.check_executed_tests(output, test, + omitted=test, interrupted=True) + regex = ('10 slowest tests:\n') self.check_line(output, regex) - self.check_line(output, 'Test suite interrupted by signal SIGINT.') def test_coverage(self): # test --coverage -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 10:14:20 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 14:14:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChzb2NrZXQuX19hbGxfXyk=?= Message-ID: <20160817141416.16227.16152.7853E434@psf.io> https://hg.python.org/cpython/rev/83398d643f34 changeset: 102729:83398d643f34 parent: 102727:2fce793ec0a2 parent: 102728:ac2bc921169c user: Victor Stinner date: Wed Aug 17 14:40:45 2016 +0200 summary: Merge 3.5 (socket.__all__) files: Lib/socket.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/socket.py b/Lib/socket.py --- a/Lib/socket.py +++ b/Lib/socket.py @@ -519,6 +519,7 @@ finally: lsock.close() return (ssock, csock) + __all__.append("socketpair") socketpair.__doc__ = """socketpair([family[, type[, proto]]]) -> (socket object, socket object) Create a pair of socket objects from the sockets returned by the platform -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 10:52:56 2016 From: python-checkins at python.org (zach.ware) Date: Wed, 17 Aug 2016 14:52:56 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogVXNlIHN5cy52ZXJz?= =?utf-8?q?ion=5Finfo=2C_not_sys=2Eversion=2E?= Message-ID: <20160817145255.78713.30408.27832C86@psf.io> https://hg.python.org/cpython/rev/df3a907d94cd changeset: 102733:df3a907d94cd branch: 3.5 parent: 102728:ac2bc921169c user: Zachary Ware date: Wed Aug 17 09:51:20 2016 -0500 summary: Use sys.version_info, not sys.version. sys.version[0] gives a string, which fails > comparison with 2. Reported by Arne Maximilian Richter on docs@ files: Doc/howto/pyporting.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/howto/pyporting.rst b/Doc/howto/pyporting.rst --- a/Doc/howto/pyporting.rst +++ b/Doc/howto/pyporting.rst @@ -299,7 +299,7 @@ import sys - if sys.version[0] == 3: + if sys.version_info[0] == 3: from importlib import abc else: from importlib2 import abc @@ -311,7 +311,7 @@ import sys - if sys.version[0] > 2: + if sys.version_info[0] > 2: from importlib import abc else: from importlib2 import abc -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 10:52:56 2016 From: python-checkins at python.org (zach.ware) Date: Wed, 17 Aug 2016 14:52:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20160817145255.28369.82162.C08E30B9@psf.io> https://hg.python.org/cpython/rev/1aeb52c635ad changeset: 102734:1aeb52c635ad parent: 102732:14a0ba0a71d2 parent: 102733:df3a907d94cd user: Zachary Ware date: Wed Aug 17 09:52:32 2016 -0500 summary: Merge with 3.5 files: Doc/howto/pyporting.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/howto/pyporting.rst b/Doc/howto/pyporting.rst --- a/Doc/howto/pyporting.rst +++ b/Doc/howto/pyporting.rst @@ -299,7 +299,7 @@ import sys - if sys.version[0] == 3: + if sys.version_info[0] == 3: from importlib import abc else: from importlib2 import abc @@ -311,7 +311,7 @@ import sys - if sys.version[0] > 2: + if sys.version_info[0] > 2: from importlib import abc else: from importlib2 import abc -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 11:20:38 2016 From: python-checkins at python.org (vinay.sajip) Date: Wed, 17 Aug 2016 15:20:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=239998=3A_Allowed_?= =?utf-8?q?find=5Flibrary_to_search_additional_locations_for_libraries=2E?= Message-ID: <20160817152035.11551.71407.875D68D8@psf.io> https://hg.python.org/cpython/rev/385181e809bc changeset: 102735:385181e809bc user: Vinay Sajip date: Wed Aug 17 16:20:07 2016 +0100 summary: Closes #9998: Allowed find_library to search additional locations for libraries. files: Doc/library/ctypes.rst | 17 ++++++-- Lib/ctypes/test/test_find.py | 43 ++++++++++++++++++++++++ Lib/ctypes/util.py | 26 +++++++++++++- Misc/NEWS | 2 + 4 files changed, 82 insertions(+), 6 deletions(-) diff --git a/Doc/library/ctypes.rst b/Doc/library/ctypes.rst --- a/Doc/library/ctypes.rst +++ b/Doc/library/ctypes.rst @@ -1239,9 +1239,10 @@ compiling/linking a program, and when the program is run. The purpose of the :func:`find_library` function is to locate a library in a way -similar to what the compiler does (on platforms with several versions of a -shared library the most recent should be loaded), while the ctypes library -loaders act like when a program is run, and call the runtime loader directly. +similar to what the compiler or runtime loader does (on platforms with several +versions of a shared library the most recent should be loaded), while the ctypes +library loaders act like when a program is run, and call the runtime loader +directly. The :mod:`ctypes.util` module provides a function which can help to determine the library to load. @@ -1259,8 +1260,14 @@ The exact functionality is system dependent. On Linux, :func:`find_library` tries to run external programs -(``/sbin/ldconfig``, ``gcc``, and ``objdump``) to find the library file. It -returns the filename of the library file. Here are some examples:: +(``/sbin/ldconfig``, ``gcc``, ``objdump`` and ``ld``) to find the library file. +It returns the filename of the library file. + +.. versionchanged:: 3.6 + On Linux, the value of the environment variable ``LD_LIBRARY_PATH`` is used + when searching for libraries, if a library cannot be found by any other means. + +Here are some examples:: >>> from ctypes.util import find_library >>> find_library("m") diff --git a/Lib/ctypes/test/test_find.py b/Lib/ctypes/test/test_find.py --- a/Lib/ctypes/test/test_find.py +++ b/Lib/ctypes/test/test_find.py @@ -69,6 +69,49 @@ self.assertFalse(os.path.lexists(test.support.TESTFN)) self.assertIsNone(result) + + at unittest.skipUnless(sys.platform.startswith('linux'), + 'Test only valid for Linux') +class LibPathFindTest(unittest.TestCase): + def test_find_on_libpath(self): + import subprocess + import tempfile + + try: + p = subprocess.Popen(['gcc', '--version'], stdout=subprocess.PIPE, + stderr=subprocess.DEVNULL) + out, _ = p.communicate() + except OSError: + raise unittest.SkipTest('gcc, needed for test, not available') + with tempfile.TemporaryDirectory() as d: + # create an empty temporary file + srcname = os.path.join(d, 'dummy.c') + libname = 'py_ctypes_test_dummy' + dstname = os.path.join(d, 'lib%s.so' % libname) + with open(srcname, 'w') as f: + pass + self.assertTrue(os.path.exists(srcname)) + # compile the file to a shared library + cmd = ['gcc', '-o', dstname, '--shared', + '-Wl,-soname,lib%s.so' % libname, srcname] + out = subprocess.check_output(cmd) + self.assertTrue(os.path.exists(dstname)) + # now check that the .so can't be found (since not in + # LD_LIBRARY_PATH) + self.assertIsNone(find_library(libname)) + # now add the location to LD_LIBRARY_PATH + with test.support.EnvironmentVarGuard() as env: + KEY = 'LD_LIBRARY_PATH' + if KEY not in env: + v = d + else: + v = '%s:%s' % (env[KEY], d) + env.set(KEY, v) + # now check that the .so can be found (since in + # LD_LIBRARY_PATH) + self.assertEqual(find_library(libname), 'lib%s.so' % libname) + + # On platforms where the default shared library suffix is '.so', # at least some libraries can be loaded as attributes of the cdll # object, since ctypes now tries loading the lib again diff --git a/Lib/ctypes/util.py b/Lib/ctypes/util.py --- a/Lib/ctypes/util.py +++ b/Lib/ctypes/util.py @@ -285,8 +285,32 @@ except OSError: pass + def _findLib_ld(name): + # See issue #9998 for why this is needed + expr = r'[^\(\)\s]*lib%s\.[^\(\)\s]*' % re.escape(name) + cmd = ['ld', '-t'] + libpath = os.environ.get('LD_LIBRARY_PATH') + if libpath: + for d in libpath.split(':'): + cmd.extend(['-L', d]) + cmd.extend(['-o', os.devnull, '-l%s' % name]) + result = None + try: + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + universal_newlines=True) + out, _ = p.communicate() + res = re.search(expr, os.fsdecode(out)) + if res: + result = res.group(0) + except Exception as e: + pass # result will be None + return result + def find_library(name): - return _findSoname_ldconfig(name) or _get_soname(_findLib_gcc(name)) + # See issue #9998 + return _findSoname_ldconfig(name) or \ + _get_soname(_findLib_gcc(name) or _findLib_ld(name)) ################################################################ # test code diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,6 +13,8 @@ Library ------- +- Issue #9998: On Linux, ctypes.util.find_library now looks in LD_LIBRARY_PATH + for shared libraries. What's New in Python 3.6.0 alpha 4 ================================== -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Wed Aug 17 12:05:41 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 17 Aug 2016 17:05:41 +0100 Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7 2016-08-17 Message-ID: <9415cc3d-eba4-45b2-a0fa-2db42a07ac53@irsmsx106.ger.corp.intel.com> Results for project Python 2.7, build date 2016-08-17 02:45:38 +0000 commit: cddd633b959f previous commit: 12a70477db03 revision date: 2016-08-16 05:44:06 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.18% -0.40% 4.65% 3.00% :-) pybench 0.21% 0.14% 5.77% 4.45% :-( regex_v8 0.64% 0.19% -2.15% 10.69% :-) nbody 0.11% 0.03% 8.85% 2.12% :-) json_dump_v2 0.30% -0.52% 2.02% 11.48% :-| normal_startup 0.65% -0.00% -0.61% 1.82% :-) ssbench 0.20% -0.31% 2.31% 2.02% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-08-17/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Wed Aug 17 12:07:04 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 17 Aug 2016 17:07:04 +0100 Subject: [Python-checkins] GOOD Benchmark Results for Python Default 2016-08-17 Message-ID: Results for project Python default, build date 2016-08-17 02:00:05 +0000 commit: e615718a6455 previous commit: d277870a6012 revision date: 2016-08-16 21:39:42 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.29% -0.24% 11.45% 15.55% :-) pybench 3.89% -0.12% 5.71% 7.53% :-( regex_v8 2.64% 1.03% -3.07% 3.08% :-) nbody 0.92% 0.80% 2.58% 4.90% :-| json_dump_v2 0.28% 0.36% -1.89% 13.55% :-| normal_startup 0.40% -0.46% 1.06% 6.29% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-08-17/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Wed Aug 17 13:49:01 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 17 Aug 2016 17:49:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327786=3A_Simplify?= =?utf-8?b?IHhfc3ViKCk=?= Message-ID: <20160817174900.28258.5164.A7B33171@psf.io> https://hg.python.org/cpython/rev/be9dc240bf28 changeset: 102736:be9dc240bf28 user: Victor Stinner date: Wed Aug 17 19:48:33 2016 +0200 summary: Issue #27786: Simplify x_sub() The z variable is known to be a fresh number which cannot be shared, Py_SIZE() can be used directly to negate the number. files: Objects/longobject.c | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -3000,9 +3000,7 @@ } assert(borrow == 0); if (sign < 0) { - _PyLong_Negate(&z); - if (z == NULL) - return NULL; + Py_SIZE(z) = -Py_SIZE(z); } return long_normalize(z); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 17:19:02 2016 From: python-checkins at python.org (ned.deily) Date: Wed, 17 Aug 2016 21:19:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327594=3A_Prevent_?= =?utf-8?q?assertion_error_when_running_test=5Fast_with_coverage?= Message-ID: <20160817211902.28226.44728.63AA7FFE@psf.io> https://hg.python.org/cpython/rev/1bf307f42a6b changeset: 102737:1bf307f42a6b user: Ned Deily date: Wed Aug 17 17:18:33 2016 -0400 summary: Issue #27594: Prevent assertion error when running test_ast with coverage enabled: ensure code object has a valid first line number. Patch suggested by Ivan Levkivskyi. files: Misc/NEWS | 4 ++++ Python/compile.c | 2 +- 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -167,6 +167,10 @@ - Issue #27369: In test_pyexpat, avoid testing an error message detail that changed in Expat 2.2.0. +- Issue #27594: Prevent assertion error when running test_ast with coverage + enabled: ensure code object has a valid first line number. + Patch suggested by Ivan Levkivskyi. + Windows ------- diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -4963,7 +4963,7 @@ /* Set firstlineno if it wasn't explicitly set. */ if (!c->u->u_firstlineno) { - if (entryblock && entryblock->b_instr) + if (entryblock && entryblock->b_instr && entryblock->b_instr->i_lineno) c->u->u_firstlineno = entryblock->b_instr->i_lineno; else c->u->u_firstlineno = 1; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 17 19:56:19 2016 From: python-checkins at python.org (alexander.belopolsky) Date: Wed, 17 Aug 2016 23:56:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2324773=3A_Include_?= =?utf-8?q?Tallinn_1999-10-31_transition_in_tests=2E?= Message-ID: <20160817235619.9446.71766.4D7113FA@psf.io> https://hg.python.org/cpython/rev/617104a6b759 changeset: 102738:617104a6b759 user: Alexander Belopolsky date: Wed Aug 17 19:56:17 2016 -0400 summary: Issue #24773: Include Tallinn 1999-10-31 transition in tests. Does not appear to be a problem anymore and I cannot figure out why it was skipped in the first place. files: Lib/test/datetimetester.py | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -4770,9 +4770,6 @@ # System support for times around the end of 32-bit time_t # and later is flaky on many systems. break - if self.zonename == 'Europe/Tallinn' and udt.date() == date(1999, 10, 31): - print("Skip %s %s transition" % (self.zonename, udt)) - continue s0 = (udt - datetime(1970, 1, 1)) // SEC ss = shift // SEC # shift seconds for x in [-40 * 3600, -20*3600, -1, 0, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 18 02:15:20 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Thu, 18 Aug 2016 06:15:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2316764=3A_Move_NEW?= =?utf-8?q?S_entry_to_correct_section_and_remove_too_strict_test=2E?= Message-ID: <20160818061520.2385.55304.7749C695@psf.io> https://hg.python.org/cpython/rev/ab0039b8a80e changeset: 102739:ab0039b8a80e user: Serhiy Storchaka date: Thu Aug 18 09:14:47 2016 +0300 summary: Issue #16764: Move NEWS entry to correct section and remove too strict test. files: Lib/test/test_zlib.py | 4 ---- Misc/NEWS | 6 +++--- 2 files changed, 3 insertions(+), 7 deletions(-) diff --git a/Lib/test/test_zlib.py b/Lib/test/test_zlib.py --- a/Lib/test/test_zlib.py +++ b/Lib/test/test_zlib.py @@ -173,10 +173,6 @@ wbits=zlib.MAX_WBITS, bufsize=zlib.DEF_BUF_SIZE), HAMLET_SCENE) - with self.assertRaises(TypeError): - zlib.decompress(data=x, - wbits=zlib.MAX_WBITS, - bufsize=zlib.DEF_BUF_SIZE) def test_speech128(self): # compress more data diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -24,9 +24,6 @@ Core and Builtins ----------------- -- Issue #16764: Support keyword arguments to zlib.decompress(). Patch by - Xiang Zhang. - - Issue #27704: Optimized creating bytes and bytearray from byte-like objects and iterables. Speed up to 3 times for short objects. Original patch by Naoki Inada. @@ -71,6 +68,9 @@ Library ------- +- Issue #16764: Support keyword arguments to zlib.decompress(). Patch by + Xiang Zhang. + - Issue #27736: Prevent segfault after interpreter re-initialization due to ref count problem introduced in code for Issue #27038 in 3.6.0a3. Patch by Xiang Zhang. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 18 12:15:03 2016 From: python-checkins at python.org (victor.stinner) Date: Thu, 18 Aug 2016 16:15:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_SystemErro?= =?utf-8?q?r_in_=22raise=22_statement?= Message-ID: <20160818161503.11681.5653.1764D0FC@psf.io> https://hg.python.org/cpython/rev/ba45fbb16499 changeset: 102740:ba45fbb16499 branch: 3.5 parent: 102733:df3a907d94cd user: Victor Stinner date: Thu Aug 18 18:13:10 2016 +0200 summary: Fix SystemError in "raise" statement Issue #27558: Fix a SystemError in the implementation of "raise" statement. In a brand new thread, raise a RuntimeError since there is no active exception to reraise. Patch written by Xiang Zhang. files: Lib/test/test_threading.py | 18 ++++++++++++++++++ Misc/NEWS | 4 ++++ Python/ceval.c | 2 +- 3 files changed, 23 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -1043,6 +1043,24 @@ self.assertEqual(out, b'') self.assertNotIn("Unhandled exception", err.decode()) + def test_bare_raise_in_brand_new_thread(self): + def bare_raise(): + raise + + class Issue27558(threading.Thread): + exc = None + + def run(self): + try: + bare_raise() + except Exception as exc: + self.exc = exc + + thread = Issue27558() + thread.start() + thread.join() + self.assertIsNotNone(thread.exc) + self.assertIsInstance(thread.exc, RuntimeError) class TimerTests(BaseTestCase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #27558: Fix a SystemError in the implementation of "raise" statement. + In a brand new thread, raise a RuntimeError since there is no active + exception to reraise. Patch written by Xiang Zhang. + - Issue #27419: Standard __import__() no longer look up "__import__" in globals or builtins for importing submodules or "from import". Fixed handling an error of non-string package name. diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4123,7 +4123,7 @@ type = tstate->exc_type; value = tstate->exc_value; tb = tstate->exc_traceback; - if (type == Py_None) { + if (type == Py_None || type == NULL) { PyErr_SetString(PyExc_RuntimeError, "No active exception to reraise"); return 0; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 18 12:15:03 2016 From: python-checkins at python.org (victor.stinner) Date: Thu, 18 Aug 2016 16:15:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChmaXggcmFpc2Up?= Message-ID: <20160818161503.23576.9838.6777FAF1@psf.io> https://hg.python.org/cpython/rev/386acc82fed4 changeset: 102741:386acc82fed4 parent: 102739:ab0039b8a80e parent: 102740:ba45fbb16499 user: Victor Stinner date: Thu Aug 18 18:14:15 2016 +0200 summary: Merge 3.5 (fix raise) files: Lib/test/test_threading.py | 18 ++++++++++++++++++ Misc/NEWS | 5 +++++ Python/ceval.c | 2 +- 3 files changed, 24 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -1043,6 +1043,24 @@ self.assertEqual(out, b'') self.assertNotIn("Unhandled exception", err.decode()) + def test_bare_raise_in_brand_new_thread(self): + def bare_raise(): + raise + + class Issue27558(threading.Thread): + exc = None + + def run(self): + try: + bare_raise() + except Exception as exc: + self.exc = exc + + thread = Issue27558() + thread.start() + thread.join() + self.assertIsNotNone(thread.exc) + self.assertIsInstance(thread.exc, RuntimeError) class TimerTests(BaseTestCase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,12 +10,17 @@ Core and Builtins ----------------- +- Issue #27558: Fix a SystemError in the implementation of "raise" statement. + In a brand new thread, raise a RuntimeError since there is no active + exception to reraise. Patch written by Xiang Zhang. + Library ------- - Issue #9998: On Linux, ctypes.util.find_library now looks in LD_LIBRARY_PATH for shared libraries. + What's New in Python 3.6.0 alpha 4 ================================== diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4154,7 +4154,7 @@ type = tstate->exc_type; value = tstate->exc_value; tb = tstate->exc_traceback; - if (type == Py_None) { + if (type == Py_None || type == NULL) { PyErr_SetString(PyExc_RuntimeError, "No active exception to reraise"); return 0; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 18 12:27:55 2016 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 18 Aug 2016 16:27:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Anti-registration_of_vario?= =?utf-8?q?us_ABC_methods=2E?= Message-ID: <20160818162752.23480.5758.0799BD99@psf.io> https://hg.python.org/cpython/rev/72b9f195569c changeset: 102742:72b9f195569c user: Guido van Rossum date: Thu Aug 18 09:22:23 2016 -0700 summary: Anti-registration of various ABC methods. - Issue #25958: Support "anti-registration" of special methods from various ABCs, like __hash__, __iter__ or __len__. All these (and several more) can be set to None in an implementation class and the behavior will be as if the method is not defined at all. (Previously, this mechanism existed only for __hash__, to make mutable classes unhashable.) Code contributed by Andrew Barnert and Ivan Levkivskyi. files: Doc/library/collections.abc.rst | 3 +- Doc/reference/datamodel.rst | 19 ++++- Lib/_collections_abc.py | 78 +++++++------------- Lib/test/test_augassign.py | 8 ++ Lib/test/test_binop.py | 50 +++++++++++++- Lib/test/test_bool.py | 11 ++ Lib/test/test_bytes.py | 30 ++++++++ Lib/test/test_collections.py | 60 ++++++++++++++- Lib/test/test_contains.py | 25 ++++++ Lib/test/test_enumerate.py | 9 ++- Lib/test/test_iter.py | 12 +++ Lib/test/test_unicode.py | 23 ++++++ Misc/NEWS | 8 ++ Objects/enumobject.c | 12 ++- Objects/typeobject.c | 14 +++ 15 files changed, 300 insertions(+), 62 deletions(-) diff --git a/Doc/library/collections.abc.rst b/Doc/library/collections.abc.rst --- a/Doc/library/collections.abc.rst +++ b/Doc/library/collections.abc.rst @@ -114,7 +114,8 @@ .. class:: Reversible - ABC for classes that provide the :meth:`__reversed__` method. + ABC for iterable classes that also provide the :meth:`__reversed__` + method. .. versionadded:: 3.6 diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1063,6 +1063,12 @@ operation raise an exception when no appropriate method is defined (typically :exc:`AttributeError` or :exc:`TypeError`). +Setting a special method to ``None`` indicates that the corresponding +operation is not available. For example, if a class sets +:meth:`__iter__` to ``None``, the class is not iterable, so calling +:func:`iter` on its instances will raise a :exc:`TypeError` (without +falling back to :meth:`__getitem__`). [#]_ + When implementing a class that emulates any built-in type, it is important that the emulation only be implemented to the degree that it makes sense for the object being modelled. For example, some sequences may work well with retrieval @@ -2113,7 +2119,7 @@ (``+``, ``-``, ``*``, ``@``, ``/``, ``//``, ``%``, :func:`divmod`, :func:`pow`, ``**``, ``<<``, ``>>``, ``&``, ``^``, ``|``) with reflected (swapped) operands. These functions are only called if the left operand does - not support the corresponding operation and the operands are of different + not support the corresponding operation [#]_ and the operands are of different types. [#]_ For instance, to evaluate the expression ``x - y``, where *y* is an instance of a class that has an :meth:`__rsub__` method, ``y.__rsub__(x)`` is called if ``x.__sub__(y)`` returns *NotImplemented*. @@ -2529,6 +2535,17 @@ controlled conditions. It generally isn't a good idea though, since it can lead to some very strange behaviour if it is handled incorrectly. +.. [#] The :meth:`__hash__`, :meth:`__iter__`, :meth:`__reversed__`, and + :meth:`__contains__` methods have special handling for this; others + will still raise a :exc:`TypeError`, but may do so by relying on + the behavior that ``None`` is not callable. + +.. [#] "Does not support" here means that the class has no such method, or + the method returns ``NotImplemented``. Do not set the method to + ``None`` if you want to force fallback to the right operand's reflected + method--that will instead have the opposite effect of explicitly + *blocking* such fallback. + .. [#] For operands of the same type, it is assumed that if the non-reflected method (such as :meth:`__add__`) fails the operation is not supported, which is why the reflected method is not called. diff --git a/Lib/_collections_abc.py b/Lib/_collections_abc.py --- a/Lib/_collections_abc.py +++ b/Lib/_collections_abc.py @@ -62,6 +62,18 @@ ### ONE-TRICK PONIES ### +def _check_methods(C, *methods): + mro = C.__mro__ + for method in methods: + for B in mro: + if method in B.__dict__: + if B.__dict__[method] is None: + return NotImplemented + break + else: + return NotImplemented + return True + class Hashable(metaclass=ABCMeta): __slots__ = () @@ -73,11 +85,7 @@ @classmethod def __subclasshook__(cls, C): if cls is Hashable: - for B in C.__mro__: - if "__hash__" in B.__dict__: - if B.__dict__["__hash__"]: - return True - break + return _check_methods(C, "__hash__") return NotImplemented @@ -92,11 +100,7 @@ @classmethod def __subclasshook__(cls, C): if cls is Awaitable: - for B in C.__mro__: - if "__await__" in B.__dict__: - if B.__dict__["__await__"]: - return True - break + return _check_methods(C, "__await__") return NotImplemented @@ -137,14 +141,7 @@ @classmethod def __subclasshook__(cls, C): if cls is Coroutine: - mro = C.__mro__ - for method in ('__await__', 'send', 'throw', 'close'): - for base in mro: - if method in base.__dict__: - break - else: - return NotImplemented - return True + return _check_methods(C, '__await__', 'send', 'throw', 'close') return NotImplemented @@ -162,8 +159,7 @@ @classmethod def __subclasshook__(cls, C): if cls is AsyncIterable: - if any("__aiter__" in B.__dict__ for B in C.__mro__): - return True + return _check_methods(C, "__aiter__") return NotImplemented @@ -182,9 +178,7 @@ @classmethod def __subclasshook__(cls, C): if cls is AsyncIterator: - if (any("__anext__" in B.__dict__ for B in C.__mro__) and - any("__aiter__" in B.__dict__ for B in C.__mro__)): - return True + return _check_methods(C, "__anext__", "__aiter__") return NotImplemented @@ -200,8 +194,7 @@ @classmethod def __subclasshook__(cls, C): if cls is Iterable: - if any("__iter__" in B.__dict__ for B in C.__mro__): - return True + return _check_methods(C, "__iter__") return NotImplemented @@ -220,9 +213,7 @@ @classmethod def __subclasshook__(cls, C): if cls is Iterator: - if (any("__next__" in B.__dict__ for B in C.__mro__) and - any("__iter__" in B.__dict__ for B in C.__mro__)): - return True + return _check_methods(C, '__iter__', '__next__') return NotImplemented Iterator.register(bytes_iterator) @@ -246,16 +237,13 @@ @abstractmethod def __reversed__(self): - return NotImplemented + while False: + yield None @classmethod def __subclasshook__(cls, C): if cls is Reversible: - for B in C.__mro__: - if "__reversed__" in B.__dict__: - if B.__dict__["__reversed__"] is not None: - return True - break + return _check_methods(C, "__reversed__", "__iter__") return NotImplemented @@ -302,17 +290,10 @@ @classmethod def __subclasshook__(cls, C): if cls is Generator: - mro = C.__mro__ - for method in ('__iter__', '__next__', 'send', 'throw', 'close'): - for base in mro: - if method in base.__dict__: - break - else: - return NotImplemented - return True + return _check_methods(C, '__iter__', '__next__', + 'send', 'throw', 'close') return NotImplemented - Generator.register(generator) @@ -327,8 +308,7 @@ @classmethod def __subclasshook__(cls, C): if cls is Sized: - if any("__len__" in B.__dict__ for B in C.__mro__): - return True + return _check_methods(C, "__len__") return NotImplemented @@ -343,8 +323,7 @@ @classmethod def __subclasshook__(cls, C): if cls is Container: - if any("__contains__" in B.__dict__ for B in C.__mro__): - return True + return _check_methods(C, "__contains__") return NotImplemented @@ -359,8 +338,7 @@ @classmethod def __subclasshook__(cls, C): if cls is Callable: - if any("__call__" in B.__dict__ for B in C.__mro__): - return True + return _check_methods(C, "__call__") return NotImplemented @@ -640,6 +618,8 @@ return NotImplemented return dict(self.items()) == dict(other.items()) + __reversed__ = None + Mapping.register(mappingproxy) diff --git a/Lib/test/test_augassign.py b/Lib/test/test_augassign.py --- a/Lib/test/test_augassign.py +++ b/Lib/test/test_augassign.py @@ -83,6 +83,10 @@ def __iadd__(self, val): return aug_test3(self.val + val) + class aug_test4(aug_test3): + """Blocks inheritance, and fallback to __add__""" + __iadd__ = None + x = aug_test(1) y = x x += 10 @@ -106,6 +110,10 @@ self.assertTrue(y is not x) self.assertEqual(x.val, 13) + x = aug_test4(4) + with self.assertRaises(TypeError): + x += 10 + def testCustomMethods2(test_self): output = [] diff --git a/Lib/test/test_binop.py b/Lib/test/test_binop.py --- a/Lib/test/test_binop.py +++ b/Lib/test/test_binop.py @@ -2,7 +2,7 @@ import unittest from test import support -from operator import eq, le +from operator import eq, le, ne from abc import ABCMeta def gcd(a, b): @@ -388,6 +388,54 @@ self.assertEqual(op_sequence(eq, B, V), ['B.__eq__', 'V.__eq__']) self.assertEqual(op_sequence(le, B, V), ['B.__le__', 'V.__ge__']) +class SupEq(object): + """Class that can test equality""" + def __eq__(self, other): + return True + +class S(SupEq): + """Subclass of SupEq that should fail""" + __eq__ = None + +class F(object): + """Independent class that should fall back""" + +class X(object): + """Independent class that should fail""" + __eq__ = None + +class SN(SupEq): + """Subclass of SupEq that can test equality, but not non-equality""" + __ne__ = None + +class XN: + """Independent class that can test equality, but not non-equality""" + def __eq__(self, other): + return True + __ne__ = None + +class FallbackBlockingTests(unittest.TestCase): + """Unit tests for None method blocking""" + + def test_fallback_rmethod_blocking(self): + e, f, s, x = SupEq(), F(), S(), X() + self.assertEqual(e, e) + self.assertEqual(e, f) + self.assertEqual(f, e) + # left operand is checked first + self.assertEqual(e, x) + self.assertRaises(TypeError, eq, x, e) + # S is a subclass, so it's always checked first + self.assertRaises(TypeError, eq, e, s) + self.assertRaises(TypeError, eq, s, e) + + def test_fallback_ne_blocking(self): + e, sn, xn = SupEq(), SN(), XN() + self.assertFalse(e != e) + self.assertRaises(TypeError, ne, e, sn) + self.assertRaises(TypeError, ne, sn, e) + self.assertFalse(e != xn) + self.assertRaises(TypeError, ne, xn, e) if __name__ == "__main__": unittest.main() diff --git a/Lib/test/test_bool.py b/Lib/test/test_bool.py --- a/Lib/test/test_bool.py +++ b/Lib/test/test_bool.py @@ -333,6 +333,17 @@ except (Exception) as e_len: self.assertEqual(str(e_bool), str(e_len)) + def test_blocked(self): + class A: + __bool__ = None + self.assertRaises(TypeError, bool, A()) + + class B: + def __len__(self): + return 10 + __bool__ = None + self.assertRaises(TypeError, bool, B()) + def test_real_and_imag(self): self.assertEqual(True.real, 1) self.assertEqual(True.imag, 0) diff --git a/Lib/test/test_bytes.py b/Lib/test/test_bytes.py --- a/Lib/test/test_bytes.py +++ b/Lib/test/test_bytes.py @@ -843,6 +843,36 @@ self.assertRaises(OverflowError, PyBytes_FromFormat, b'%c', c_int(256)) + def test_bytes_blocking(self): + class IterationBlocked(list): + __bytes__ = None + i = [0, 1, 2, 3] + self.assertEqual(bytes(i), b'\x00\x01\x02\x03') + self.assertRaises(TypeError, bytes, IterationBlocked(i)) + + # At least in CPython, because bytes.__new__ and the C API + # PyBytes_FromObject have different fallback rules, integer + # fallback is handled specially, so test separately. + class IntBlocked(int): + __bytes__ = None + self.assertEqual(bytes(3), b'\0\0\0') + self.assertRaises(TypeError, bytes, IntBlocked(3)) + + # While there is no separately-defined rule for handling bytes + # subclasses differently from other buffer-interface classes, + # an implementation may well special-case them (as CPython 2.x + # str did), so test them separately. + class BytesSubclassBlocked(bytes): + __bytes__ = None + self.assertEqual(bytes(b'ab'), b'ab') + self.assertRaises(TypeError, bytes, BytesSubclassBlocked(b'ab')) + + class BufferBlocked(bytearray): + __bytes__ = None + ba, bb = bytearray(b'ab'), BufferBlocked(b'ab') + self.assertEqual(bytes(ba), b'ab') + self.assertRaises(TypeError, bytes, bb) + class ByteArrayTest(BaseBytesTest, unittest.TestCase): type2test = bytearray diff --git a/Lib/test/test_collections.py b/Lib/test/test_collections.py --- a/Lib/test/test_collections.py +++ b/Lib/test/test_collections.py @@ -499,6 +499,9 @@ self.assertTrue(other.right_side,'Right side not called for %s.%s' % (type(instance), name)) +def _test_gen(): + yield + class TestOneTrickPonyABCs(ABCTestCase): def test_Awaitable(self): @@ -686,7 +689,7 @@ samples = [bytes(), str(), tuple(), list(), set(), frozenset(), dict(), dict().keys(), dict().items(), dict().values(), - (lambda: (yield))(), + _test_gen(), (x for x in []), ] for x in samples: @@ -700,6 +703,15 @@ self.assertFalse(issubclass(str, I)) self.validate_abstract_methods(Iterable, '__iter__') self.validate_isinstance(Iterable, '__iter__') + # Check None blocking + class It: + def __iter__(self): return iter([]) + class ItBlocked(It): + __iter__ = None + self.assertTrue(issubclass(It, Iterable)) + self.assertTrue(isinstance(It(), Iterable)) + self.assertFalse(issubclass(ItBlocked, Iterable)) + self.assertFalse(isinstance(ItBlocked(), Iterable)) def test_Reversible(self): # Check some non-reversibles @@ -707,8 +719,18 @@ for x in non_samples: self.assertNotIsInstance(x, Reversible) self.assertFalse(issubclass(type(x), Reversible), repr(type(x))) - # Check some reversibles - samples = [tuple(), list()] + # Check some non-reversible iterables + non_reversibles = [dict().keys(), dict().items(), dict().values(), + Counter(), Counter().keys(), Counter().items(), + Counter().values(), _test_gen(), + (x for x in []), iter([]), reversed([])] + for x in non_reversibles: + self.assertNotIsInstance(x, Reversible) + self.assertFalse(issubclass(type(x), Reversible), repr(type(x))) + # Check some reversible iterables + samples = [bytes(), str(), tuple(), list(), OrderedDict(), + OrderedDict().keys(), OrderedDict().items(), + OrderedDict().values()] for x in samples: self.assertIsInstance(x, Reversible) self.assertTrue(issubclass(type(x), Reversible), repr(type(x))) @@ -725,6 +747,29 @@ self.assertEqual(list(reversed(R())), []) self.assertFalse(issubclass(float, R)) self.validate_abstract_methods(Reversible, '__reversed__', '__iter__') + # Check reversible non-iterable (which is not Reversible) + class RevNoIter: + def __reversed__(self): return reversed([]) + class RevPlusIter(RevNoIter): + def __iter__(self): return iter([]) + self.assertFalse(issubclass(RevNoIter, Reversible)) + self.assertFalse(isinstance(RevNoIter(), Reversible)) + self.assertTrue(issubclass(RevPlusIter, Reversible)) + self.assertTrue(isinstance(RevPlusIter(), Reversible)) + # Check None blocking + class Rev: + def __iter__(self): return iter([]) + def __reversed__(self): return reversed([]) + class RevItBlocked(Rev): + __iter__ = None + class RevRevBlocked(Rev): + __reversed__ = None + self.assertTrue(issubclass(Rev, Reversible)) + self.assertTrue(isinstance(Rev(), Reversible)) + self.assertFalse(issubclass(RevItBlocked, Reversible)) + self.assertFalse(isinstance(RevItBlocked(), Reversible)) + self.assertFalse(issubclass(RevRevBlocked, Reversible)) + self.assertFalse(isinstance(RevRevBlocked(), Reversible)) def test_Iterator(self): non_samples = [None, 42, 3.14, 1j, b"", "", (), [], {}, set()] @@ -736,7 +781,7 @@ iter(set()), iter(frozenset()), iter(dict().keys()), iter(dict().items()), iter(dict().values()), - (lambda: (yield))(), + _test_gen(), (x for x in []), ] for x in samples: @@ -824,7 +869,7 @@ def test_Sized(self): non_samples = [None, 42, 3.14, 1j, - (lambda: (yield))(), + _test_gen(), (x for x in []), ] for x in non_samples: @@ -842,7 +887,7 @@ def test_Container(self): non_samples = [None, 42, 3.14, 1j, - (lambda: (yield))(), + _test_gen(), (x for x in []), ] for x in non_samples: @@ -861,7 +906,7 @@ def test_Callable(self): non_samples = [None, 42, 3.14, 1j, "", b"", (), [], {}, set(), - (lambda: (yield))(), + _test_gen(), (x for x in []), ] for x in non_samples: @@ -1276,6 +1321,7 @@ def __iter__(self): return iter(()) self.validate_comparison(MyMapping()) + self.assertRaises(TypeError, reversed, MyMapping()) def test_MutableMapping(self): for sample in [dict]: diff --git a/Lib/test/test_contains.py b/Lib/test/test_contains.py --- a/Lib/test/test_contains.py +++ b/Lib/test/test_contains.py @@ -84,6 +84,31 @@ self.assertTrue(container == constructor(values)) self.assertTrue(container == container) + def test_block_fallback(self): + # blocking fallback with __contains__ = None + class ByContains(object): + def __contains__(self, other): + return False + c = ByContains() + class BlockContains(ByContains): + """Is not a container + + This class is a perfectly good iterable (as tested by + list(bc)), as well as inheriting from a perfectly good + container, but __contains__ = None prevents the usual + fallback to iteration in the container protocol. That + is, normally, 0 in bc would fall back to the equivalent + of any(x==0 for x in bc), but here it's blocked from + doing so. + """ + def __iter__(self): + while False: + yield None + __contains__ = None + bc = BlockContains() + self.assertFalse(0 in c) + self.assertFalse(0 in list(bc)) + self.assertRaises(TypeError, lambda: 0 in bc) if __name__ == '__main__': unittest.main() diff --git a/Lib/test/test_enumerate.py b/Lib/test/test_enumerate.py --- a/Lib/test/test_enumerate.py +++ b/Lib/test/test_enumerate.py @@ -223,7 +223,7 @@ def test_objmethods(self): # Objects must have __len__() and __getitem__() implemented. class NoLen(object): - def __getitem__(self): return 1 + def __getitem__(self, i): return 1 nl = NoLen() self.assertRaises(TypeError, reversed, nl) @@ -232,6 +232,13 @@ ngi = NoGetItem() self.assertRaises(TypeError, reversed, ngi) + class Blocked(object): + def __getitem__(self, i): return 1 + def __len__(self): return 2 + __reversed__ = None + b = Blocked() + self.assertRaises(TypeError, reversed, b) + def test_pickle(self): for data in 'abc', range(5), tuple(enumerate('abc')), range(1,17,5): self.check_pickle(reversed(data), list(data)[::-1]) diff --git a/Lib/test/test_iter.py b/Lib/test/test_iter.py --- a/Lib/test/test_iter.py +++ b/Lib/test/test_iter.py @@ -54,6 +54,14 @@ def __getitem__(self, i): return i +class DefaultIterClass: + pass + +class NoIterClass: + def __getitem__(self, i): + return i + __iter__ = None + # Main test suite class TestCase(unittest.TestCase): @@ -995,6 +1003,10 @@ def test_free_after_iterating(self): check_free_after_iterating(self, iter, SequenceClass, (0,)) + def test_error_iter(self): + for typ in (DefaultIterClass, NoIterClass): + self.assertRaises(TypeError, iter, typ()) + def test_main(): run_unittest(TestCase) diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py --- a/Lib/test/test_unicode.py +++ b/Lib/test/test_unicode.py @@ -986,6 +986,19 @@ def __format__(self, format_spec): return int.__format__(self * 2, format_spec) + class M: + def __init__(self, x): + self.x = x + def __repr__(self): + return 'M(' + self.x + ')' + __str__ = None + + class N: + def __init__(self, x): + self.x = x + def __repr__(self): + return 'N(' + self.x + ')' + __format__ = None self.assertEqual(''.format(), '') self.assertEqual('abc'.format(), 'abc') @@ -1200,6 +1213,16 @@ self.assertEqual("0x{:0{:d}X}".format(0x0,16), "0x0000000000000000") + # Blocking fallback + m = M('data') + self.assertEqual("{!r}".format(m), 'M(data)') + self.assertRaises(TypeError, "{!s}".format, m) + self.assertRaises(TypeError, "{}".format, m) + n = N('data') + self.assertEqual("{!r}".format(n), 'N(data)') + self.assertEqual("{!s}".format(n), 'N(data)') + self.assertRaises(TypeError, "{}".format, n) + def test_format_map(self): self.assertEqual(''.format_map({}), '') self.assertEqual('a'.format_map({}), 'a') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -73,6 +73,14 @@ Library ------- +- Issue #25958: Support "anti-registration" of special methods from + various ABCs, like __hash__, __iter__ or __len__. All these (and + several more) can be set to None in an implementation class and the + behavior will be as if the method is not defined at all. + (Previously, this mechanism existed only for __hash__, to make + mutable classes unhashable.) Code contributed by Andrew Barnert and + Ivan Levkivskyi. + - Issue #16764: Support keyword arguments to zlib.decompress(). Patch by Xiang Zhang. diff --git a/Objects/enumobject.c b/Objects/enumobject.c --- a/Objects/enumobject.c +++ b/Objects/enumobject.c @@ -250,6 +250,13 @@ return NULL; reversed_meth = _PyObject_LookupSpecial(seq, &PyId___reversed__); + if (reversed_meth == Py_None) { + Py_DECREF(reversed_meth); + PyErr_Format(PyExc_TypeError, + "'%.200s' object is not reversible", + Py_TYPE(seq)->tp_name); + return NULL; + } if (reversed_meth != NULL) { PyObject *res = PyObject_CallFunctionObjArgs(reversed_meth, NULL); Py_DECREF(reversed_meth); @@ -259,8 +266,9 @@ return NULL; if (!PySequence_Check(seq)) { - PyErr_SetString(PyExc_TypeError, - "argument to reversed() must be a sequence"); + PyErr_Format(PyExc_TypeError, + "'%.200s' object is not reversible", + Py_TYPE(seq)->tp_name); return NULL; } diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -5856,6 +5856,13 @@ _Py_IDENTIFIER(__contains__); func = lookup_maybe(self, &PyId___contains__); + if (func == Py_None) { + Py_DECREF(func); + PyErr_Format(PyExc_TypeError, + "'%.200s' object is not a container", + Py_TYPE(self)->tp_name); + return -1; + } if (func != NULL) { args = PyTuple_Pack(1, value); if (args == NULL) @@ -6241,6 +6248,13 @@ _Py_IDENTIFIER(__iter__); func = lookup_method(self, &PyId___iter__); + if (func == Py_None) { + Py_DECREF(func); + PyErr_Format(PyExc_TypeError, + "'%.200s' object is not iterable", + Py_TYPE(self)->tp_name); + return NULL; + } if (func != NULL) { PyObject *args; args = res = PyTuple_New(0); -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Thu Aug 18 13:32:10 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 18 Aug 2016 18:32:10 +0100 Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-08-18 Message-ID: Results for project Python 2.7, build date 2016-08-18 02:46:02 +0000 commit: 11a9bca71528 previous commit: cddd633b959f revision date: 2016-08-17 06:35:35 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.14% -1.08% 3.62% 7.12% :-) pybench 0.20% -0.02% 5.76% 4.57% :-( regex_v8 0.64% -0.05% -2.20% 10.88% :-) nbody 0.26% -0.05% 8.81% 2.51% :-) json_dump_v2 0.30% 0.38% 2.39% 10.86% :-| normal_startup 0.68% 0.32% -0.29% 1.67% :-) ssbench 0.22% 0.09% 2.40% 1.70% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-08-18/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Thu Aug 18 13:34:27 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 18 Aug 2016 18:34:27 +0100 Subject: [Python-checkins] NEUTRAL Benchmark Results for Python Default 2016-08-18 Message-ID: <9e069306-815f-4480-91cb-4a631cc657f6@irsmsx106.ger.corp.intel.com> Results for project Python default, build date 2016-08-18 02:00:30 +0000 commit: 617104a6b759 previous commit: e615718a6455 revision date: 2016-08-17 23:56:17 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.19% 0.77% 12.13% 15.75% :-) pybench 3.95% 0.62% 6.30% 6.24% :-( regex_v8 2.66% -0.45% -3.53% 7.61% :-| nbody 0.14% -0.89% 1.71% 6.69% :-| json_dump_v2 0.44% 0.73% -1.15% 11.57% :-| normal_startup 0.41% -0.28% 0.75% 6.80% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-default-2016-08-18/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Thu Aug 18 16:24:28 2016 From: python-checkins at python.org (vinay.sajip) Date: Thu, 18 Aug 2016 20:24:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2312713=3A_Allowed?= =?utf-8?q?_abbreviation_of_subcommands_in_argparse=2E?= Message-ID: <20160818202420.11642.75454.6B87C934@psf.io> https://hg.python.org/cpython/rev/97b28115bd08 changeset: 102743:97b28115bd08 user: Vinay Sajip date: Thu Aug 18 21:23:48 2016 +0100 summary: Closes #12713: Allowed abbreviation of subcommands in argparse. files: Doc/library/argparse.rst | 12 +++ Lib/argparse.py | 23 +++++- Lib/test/test_argparse.py | 92 +++++++++++++++++++------- 3 files changed, 95 insertions(+), 32 deletions(-) diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -1668,6 +1668,18 @@ >>> parser.parse_args(['co', 'bar']) Namespace(foo='bar') + argparse supports non-ambiguous abbreviations of subparser names. + + For example, the following three calls are all supported + and do the same thing, as the abbreviations used are not ambiguous:: + + >>> parser.parse_args(['checkout', 'bar']) + Namespace(foo='bar') + >>> parser.parse_args(['check', 'bar']) + Namespace(foo='bar') + >>> parser.parse_args(['che', 'bar']) + Namespace(foo='bar') + One particularly effective way of handling sub-commands is to combine the use of the :meth:`add_subparsers` method with calls to :meth:`set_defaults` so that each subparser knows which Python function it should execute. For diff --git a/Lib/argparse.py b/Lib/argparse.py --- a/Lib/argparse.py +++ b/Lib/argparse.py @@ -1110,6 +1110,12 @@ parser_name = values[0] arg_strings = values[1:] + # get full parser_name from (optional) abbreviated one + for p in self._name_parser_map: + if p.startswith(parser_name): + parser_name = p + break + # set the parser name if requested if self.dest is not SUPPRESS: setattr(namespace, self.dest, parser_name) @@ -2307,11 +2313,18 @@ def _check_value(self, action, value): # converted value must be one of the choices (if specified) - if action.choices is not None and value not in action.choices: - args = {'value': value, - 'choices': ', '.join(map(repr, action.choices))} - msg = _('invalid choice: %(value)r (choose from %(choices)s)') - raise ArgumentError(action, msg % args) + if action.choices is not None: + ac = [ax for ax in action.choices if str(ax).startswith(str(value))] + if len(ac) == 0: + args = {'value': value, + 'choices': ', '.join(map(repr, action.choices))} + msg = _('invalid choice: %(value)r (choose from %(choices)s)') + raise ArgumentError(action, msg % args) + elif len(ac) > 1: + args = {'value': value, + 'choices': ', '.join(ac)} + msg = _('ambiguous choice: %(value)r could match %(choices)s') + raise ArgumentError(action, msg % args) # ======================= # Help-formatting methods diff --git a/Lib/test/test_argparse.py b/Lib/test/test_argparse.py --- a/Lib/test/test_argparse.py +++ b/Lib/test/test_argparse.py @@ -1842,6 +1842,22 @@ parser3.add_argument('t', type=int, help='t help') parser3.add_argument('u', nargs='...', help='u help') + # add fourth sub-parser (to test abbreviations) + parser4_kwargs = dict(description='lost description') + if subparser_help: + parser4_kwargs['help'] = 'lost help' + parser4 = subparsers.add_parser('lost', **parser4_kwargs) + parser4.add_argument('-w', type=int, help='w help') + parser4.add_argument('x', choices='abc', help='x help') + + # add fifth sub-parser, with longer name (to test abbreviations) + parser5_kwargs = dict(description='long description') + if subparser_help: + parser5_kwargs['help'] = 'long help' + parser5 = subparsers.add_parser('long', **parser5_kwargs) + parser5.add_argument('-w', type=int, help='w help') + parser5.add_argument('x', choices='abc', help='x help') + # return the main parser return parser @@ -1857,6 +1873,24 @@ args = args_str.split() self.assertArgumentParserError(self.parser.parse_args, args) + def test_parse_args_abbreviation(self): + # check some non-failure cases: + self.assertEqual( + self.parser.parse_args('0.5 long b -w 7'.split()), + NS(foo=False, bar=0.5, w=7, x='b'), + ) + self.assertEqual( + self.parser.parse_args('0.5 lon b -w 7'.split()), + NS(foo=False, bar=0.5, w=7, x='b'), + ) + self.assertEqual( + self.parser.parse_args('0.5 los b -w 7'.split()), + NS(foo=False, bar=0.5, w=7, x='b'), + ) + # check a failure case: 'lo' is ambiguous + self.assertArgumentParserError(self.parser.parse_args, + '0.5 lo b -w 7'.split()) + def test_parse_args(self): # check some non-failure cases: self.assertEqual( @@ -1909,78 +1943,80 @@ def test_help(self): self.assertEqual(self.parser.format_usage(), - 'usage: PROG [-h] [--foo] bar {1,2,3} ...\n') + 'usage: PROG [-h] [--foo] bar {1,2,3,lost,long} ...\n') self.assertEqual(self.parser.format_help(), textwrap.dedent('''\ - usage: PROG [-h] [--foo] bar {1,2,3} ... + usage: PROG [-h] [--foo] bar {1,2,3,lost,long} ... main description positional arguments: - bar bar help - {1,2,3} command help + bar bar help + {1,2,3,lost,long} command help optional arguments: - -h, --help show this help message and exit - --foo foo help + -h, --help show this help message and exit + --foo foo help ''')) def test_help_extra_prefix_chars(self): # Make sure - is still used for help if it is a non-first prefix char parser = self._get_parser(prefix_chars='+:-') self.assertEqual(parser.format_usage(), - 'usage: PROG [-h] [++foo] bar {1,2,3} ...\n') + 'usage: PROG [-h] [++foo] bar {1,2,3,lost,long} ...\n') self.assertEqual(parser.format_help(), textwrap.dedent('''\ - usage: PROG [-h] [++foo] bar {1,2,3} ... + usage: PROG [-h] [++foo] bar {1,2,3,lost,long} ... main description positional arguments: - bar bar help - {1,2,3} command help + bar bar help + {1,2,3,lost,long} command help optional arguments: - -h, --help show this help message and exit - ++foo foo help + -h, --help show this help message and exit + ++foo foo help ''')) def test_help_alternate_prefix_chars(self): parser = self._get_parser(prefix_chars='+:/') self.assertEqual(parser.format_usage(), - 'usage: PROG [+h] [++foo] bar {1,2,3} ...\n') + 'usage: PROG [+h] [++foo] bar {1,2,3,lost,long} ...\n') self.assertEqual(parser.format_help(), textwrap.dedent('''\ - usage: PROG [+h] [++foo] bar {1,2,3} ... + usage: PROG [+h] [++foo] bar {1,2,3,lost,long} ... main description positional arguments: - bar bar help - {1,2,3} command help + bar bar help + {1,2,3,lost,long} command help optional arguments: - +h, ++help show this help message and exit - ++foo foo help + +h, ++help show this help message and exit + ++foo foo help ''')) def test_parser_command_help(self): self.assertEqual(self.command_help_parser.format_usage(), - 'usage: PROG [-h] [--foo] bar {1,2,3} ...\n') + 'usage: PROG [-h] [--foo] bar {1,2,3,lost,long} ...\n') self.assertEqual(self.command_help_parser.format_help(), textwrap.dedent('''\ - usage: PROG [-h] [--foo] bar {1,2,3} ... + usage: PROG [-h] [--foo] bar {1,2,3,lost,long} ... main description positional arguments: - bar bar help - {1,2,3} command help - 1 1 help - 2 2 help - 3 3 help + bar bar help + {1,2,3,lost,long} command help + 1 1 help + 2 2 help + 3 3 help + lost lost help + long long help optional arguments: - -h, --help show this help message and exit - --foo foo help + -h, --help show this help message and exit + --foo foo help ''')) def test_subparser_title_help(self): @@ -2083,6 +2119,8 @@ 1 help 2 2 help 3 3 help + lost lost help + long long help """)) # ============ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 18 16:27:13 2016 From: python-checkins at python.org (vinay.sajip) Date: Thu, 18 Aug 2016 20:27:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Updated_NEWS_with_informat?= =?utf-8?q?ion_on_the_argparse_change=2E?= Message-ID: <20160818202706.2385.80208.379FD374@psf.io> https://hg.python.org/cpython/rev/da47269c6de8 changeset: 102744:da47269c6de8 user: Vinay Sajip date: Thu Aug 18 21:26:56 2016 +0100 summary: Updated NEWS with information on the argparse change. files: Misc/NEWS | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -20,6 +20,9 @@ - Issue #9998: On Linux, ctypes.util.find_library now looks in LD_LIBRARY_PATH for shared libraries. +- Issue #12713: Allowed abbreviation of subcommands by end-users for users of + argparse. + What's New in Python 3.6.0 alpha 4 ================================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 18 21:35:27 2016 From: python-checkins at python.org (r.david.murray) Date: Fri, 19 Aug 2016 01:35:27 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzI0NjY6IGlzbW91?= =?utf-8?q?nt_now_recognizes_mount_points_user_can=27t_access=2E?= Message-ID: <20160819013527.24504.10349.229326E6@psf.io> https://hg.python.org/cpython/rev/cbc78ca21977 changeset: 102745:cbc78ca21977 branch: 3.5 parent: 102740:ba45fbb16499 user: R David Murray date: Thu Aug 18 21:27:48 2016 -0400 summary: #2466: ismount now recognizes mount points user can't access. Patch by Robin Roth, reviewed by Serhiy Storchaka, comment wording tweaked by me. files: Lib/posixpath.py | 1 + Lib/test/test_posixpath.py | 24 ++++++++++++++++++++++-- Misc/ACKS | 1 + Misc/NEWS | 3 +++ 4 files changed, 27 insertions(+), 2 deletions(-) diff --git a/Lib/posixpath.py b/Lib/posixpath.py --- a/Lib/posixpath.py +++ b/Lib/posixpath.py @@ -193,6 +193,7 @@ parent = join(path, b'..') else: parent = join(path, '..') + parent = realpath(parent) try: s2 = os.lstat(parent) except OSError: diff --git a/Lib/test/test_posixpath.py b/Lib/test/test_posixpath.py --- a/Lib/test/test_posixpath.py +++ b/Lib/test/test_posixpath.py @@ -1,7 +1,5 @@ -import itertools import os import posixpath -import sys import unittest import warnings from posixpath import realpath, abspath, dirname, basename @@ -213,6 +211,28 @@ finally: os.lstat = save_lstat + @unittest.skipIf(posix is None, "Test requires posix module") + def test_ismount_directory_not_readable(self): + # issue #2466: Simulate ismount run on a directory that is not + # readable, which used to return False. + save_lstat = os.lstat + def fake_lstat(path): + st_ino = 0 + st_dev = 0 + if path.startswith(ABSTFN) and path != ABSTFN: + # ismount tries to read something inside the ABSTFN directory; + # simulate this being forbidden (no read permission). + raise OSError("Fake [Errno 13] Permission denied") + if path == ABSTFN: + st_dev = 1 + st_ino = 1 + return posix.stat_result((0, st_ino, st_dev, 0, 0, 0, 0, 0, 0, 0)) + try: + os.lstat = fake_lstat + self.assertIs(posixpath.ismount(ABSTFN), True) + finally: + os.lstat = save_lstat + def test_expanduser(self): self.assertEqual(posixpath.expanduser("foo"), "foo") self.assertEqual(posixpath.expanduser(b"foo"), b"foo") diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1258,6 +1258,7 @@ Just van Rossum Hugo van Rossum Saskia van Rossum +Robin Roth Clement Rouault Donald Wallace Rouse II Liam Routt diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -38,6 +38,9 @@ Library ------- +- Issue #2466: posixpath.ismount now correctly recognizes mount points which + the user does not have permission to access. + - Issue #27773: Correct some memory management errors server_hostname in _ssl.wrap_socket(). - Issue #26750: unittest.mock.create_autospec() now works properly for -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 18 21:35:29 2016 From: python-checkins at python.org (r.david.murray) Date: Fri, 19 Aug 2016 01:35:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=232466=3A_ismount_now_recognizes_mount_points_?= =?utf-8?q?user_can=27t_access=2E?= Message-ID: <20160819013528.43130.2361.1BC58F19@psf.io> https://hg.python.org/cpython/rev/db9126139969 changeset: 102746:db9126139969 parent: 102744:da47269c6de8 parent: 102745:cbc78ca21977 user: R David Murray date: Thu Aug 18 21:31:13 2016 -0400 summary: Merge: #2466: ismount now recognizes mount points user can't access. files: Lib/posixpath.py | 1 + Lib/test/test_posixpath.py | 22 ++++++++++++++++++++++ Misc/ACKS | 1 + Misc/NEWS | 3 +++ 4 files changed, 27 insertions(+), 0 deletions(-) diff --git a/Lib/posixpath.py b/Lib/posixpath.py --- a/Lib/posixpath.py +++ b/Lib/posixpath.py @@ -193,6 +193,7 @@ parent = join(path, b'..') else: parent = join(path, '..') + parent = realpath(parent) try: s2 = os.lstat(parent) except OSError: diff --git a/Lib/test/test_posixpath.py b/Lib/test/test_posixpath.py --- a/Lib/test/test_posixpath.py +++ b/Lib/test/test_posixpath.py @@ -211,6 +211,28 @@ finally: os.lstat = save_lstat + @unittest.skipIf(posix is None, "Test requires posix module") + def test_ismount_directory_not_readable(self): + # issue #2466: Simulate ismount run on a directory that is not + # readable, which used to return False. + save_lstat = os.lstat + def fake_lstat(path): + st_ino = 0 + st_dev = 0 + if path.startswith(ABSTFN) and path != ABSTFN: + # ismount tries to read something inside the ABSTFN directory; + # simulate this being forbidden (no read permission). + raise OSError("Fake [Errno 13] Permission denied") + if path == ABSTFN: + st_dev = 1 + st_ino = 1 + return posix.stat_result((0, st_ino, st_dev, 0, 0, 0, 0, 0, 0, 0)) + try: + os.lstat = fake_lstat + self.assertIs(posixpath.ismount(ABSTFN), True) + finally: + os.lstat = save_lstat + def test_expanduser(self): self.assertEqual(posixpath.expanduser("foo"), "foo") self.assertEqual(posixpath.expanduser(b"foo"), b"foo") diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1273,6 +1273,7 @@ Just van Rossum Hugo van Rossum Saskia van Rossum +Robin Roth Clement Rouault Donald Wallace Rouse II Liam Routt diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -76,6 +76,9 @@ Library ------- +- Issue #2466: posixpath.ismount now correctly recognizes mount points which + the user does not have permission to access. + - Issue #25958: Support "anti-registration" of special methods from various ABCs, like __hash__, __iter__ or __len__. All these (and several more) can be set to None in an implementation class and the -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 18 21:45:30 2016 From: python-checkins at python.org (r.david.murray) Date: Fri, 19 Aug 2016 01:45:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Rewrap_long_li?= =?utf-8?q?nes_in_Misc/NEWS=2E?= Message-ID: <20160819014530.85127.78356.2E1E55D9@psf.io> https://hg.python.org/cpython/rev/bad82d4bdf8c changeset: 102747:bad82d4bdf8c branch: 3.5 parent: 102745:cbc78ca21977 user: R David Murray date: Thu Aug 18 21:40:48 2016 -0400 summary: Rewrap long lines in Misc/NEWS. files: Misc/NEWS | 46 ++++++++++++++++++++++++------------------ 1 files changed, 26 insertions(+), 20 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -41,7 +41,8 @@ - Issue #2466: posixpath.ismount now correctly recognizes mount points which the user does not have permission to access. -- Issue #27773: Correct some memory management errors server_hostname in _ssl.wrap_socket(). +- Issue #27773: Correct some memory management errors server_hostname in + _ssl.wrap_socket(). - Issue #26750: unittest.mock.create_autospec() now works properly for subclasses of property() and other data descriptors. @@ -49,14 +50,15 @@ - In the curses module, raise an error if window.getstr() or window.instr() is passed a negative value. -- Issue #27783: Fix possible usage of uninitialized memory in operator.methodcaller. +- Issue #27783: Fix possible usage of uninitialized memory in + operator.methodcaller. - Issue #27774: Fix possible Py_DECREF on unowned object in _sre. - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. -- Issue #27758: Fix possible integer overflow in the _csv module for large record - lengths. +- Issue #27758: Fix possible integer overflow in the _csv module for large + record lengths. - Issue #27568: Prevent HTTPoxy attack (CVE-2016-1000110). Ignore the HTTP_PROXY variable when REQUEST_METHOD environment is set, which indicates @@ -116,9 +118,9 @@ when exiting, let the new chained one through. This avoids the PEP 479 bug described in issue25782. -- [Security] Issue #27278: Fix os.urandom() implementation using getrandom() on Linux. - Truncate size to INT_MAX and loop until we collected enough random bytes, - instead of casting a directly Py_ssize_t to int. +- [Security] Issue #27278: Fix os.urandom() implementation using getrandom() on + Linux. Truncate size to INT_MAX and loop until we collected enough random + bytes, instead of casting a directly Py_ssize_t to int. - Issue #26386: Fixed ttk.TreeView selection operations with item id's containing spaces. @@ -319,7 +321,8 @@ This allows sys.getsize() to work correctly with their subclasses with __slots__ defined. -- Issue #25709: Fixed problem with in-place string concatenation and utf-8 cache. +- Issue #25709: Fixed problem with in-place string concatenation and utf-8 + cache. - Issue #27147: Mention PEP 420 in the importlib docs. @@ -377,9 +380,10 @@ - Issue #21313: Fix the "platform" module to tolerate when sys.version contains truncated build information. -- [Security] Issue #26839: On Linux, :func:`os.urandom` now calls ``getrandom()`` with - ``GRND_NONBLOCK`` to fall back on reading ``/dev/urandom`` if the urandom - entropy pool is not initialized yet. Patch written by Colm Buckley. +- [Security] Issue #26839: On Linux, :func:`os.urandom` now calls + ``getrandom()`` with ``GRND_NONBLOCK`` to fall back on reading + ``/dev/urandom`` if the urandom entropy pool is not initialized yet. Patch + written by Colm Buckley. - Issue #27164: In the zlib module, allow decompressing raw Deflate streams with a predefined zdict. Based on patch by Xiang Zhang. @@ -485,9 +489,9 @@ - Issue #24838: tarfile's ustar and gnu formats now correctly calculate name and link field limits for multibyte character encodings like utf-8. -- [Security] Issue #26657: Fix directory traversal vulnerability with http.server on - Windows. This fixes a regression that was introduced in 3.3.4rc1 and - 3.4.0rc1. Based on patch by Philipp Hagemeister. +- [Security] Issue #26657: Fix directory traversal vulnerability with + http.server on Windows. This fixes a regression that was introduced in + 3.3.4rc1 and 3.4.0rc1. Based on patch by Philipp Hagemeister. - Issue #26717: Stop encoding Latin-1-ized WSGI paths with UTF-8. Patch by Anthony Sottile. @@ -534,8 +538,8 @@ - Issue #26560: Avoid potential ValueError in BaseHandler.start_response. Initial patch by Peter Inglesby. -- [Security] Issue #26313: ssl.py _load_windows_store_certs fails if windows cert store - is empty. Patch by Baji. +- [Security] Issue #26313: ssl.py _load_windows_store_certs fails if windows + cert store is empty. Patch by Baji. - Issue #26569: Fix :func:`pyclbr.readmodule` and :func:`pyclbr.readmodule_ex` to support importing packages. @@ -564,7 +568,8 @@ The bufsize parameter is no longer used. - Issue #2202: Fix UnboundLocalError in - AbstractDigestAuthHandler.get_algorithm_impls. Initial patch by Mathieu Dupuy. + AbstractDigestAuthHandler.get_algorithm_impls. Initial patch by Mathieu + Dupuy. - Issue #25718: Fixed pickling and copying the accumulate() iterator with total is None. @@ -596,7 +601,8 @@ the connected socket) when verify_request() returns false. Patch by Aviv Palivoda. -- [Security] Issue #25939: On Windows open the cert store readonly in ssl.enum_certificates. +- [Security] Issue #25939: On Windows open the cert store readonly in + ssl.enum_certificates. - Issue #25995: os.walk() no longer uses FDs proportional to the tree depth. @@ -2000,8 +2006,8 @@ - Issue #1322: platform.dist() and platform.linux_distribution() functions are now deprecated. Initial patch by Vajrasky Kok. -- Issue #22486: Added the math.gcd() function. The fractions.gcd() function now is - deprecated. Based on patch by Mark Dickinson. +- Issue #22486: Added the math.gcd() function. The fractions.gcd() function + now is deprecated. Based on patch by Mark Dickinson. - Issue #24064: Property() docstrings are now writeable. (Patch by Berker Peksag.) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 18 21:45:30 2016 From: python-checkins at python.org (r.david.murray) Date: Fri, 19 Aug 2016 01:45:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Rewrap_long_lines_in_Misc/NEWS=2E?= Message-ID: <20160819014530.3958.5520.E5EE07F0@psf.io> https://hg.python.org/cpython/rev/dd0f1fa809c5 changeset: 102748:dd0f1fa809c5 parent: 102746:db9126139969 parent: 102747:bad82d4bdf8c user: R David Murray date: Thu Aug 18 21:45:12 2016 -0400 summary: Rewrap long lines in Misc/NEWS. Also moved news item for #2466 to the correct place. files: Misc/NEWS | 68 ++++++++++++++++++++++++------------------ 1 files changed, 38 insertions(+), 30 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -17,6 +17,9 @@ Library ------- +- Issue #2466: posixpath.ismount now correctly recognizes mount points which + the user does not have permission to access. + - Issue #9998: On Linux, ctypes.util.find_library now looks in LD_LIBRARY_PATH for shared libraries. @@ -43,7 +46,8 @@ - Issue #27574: Decreased an overhead of parsing keyword arguments in functions implemented with using Argument Clinic. -- Issue #22557: Now importing already imported modules is up to 2.5 times faster. +- Issue #22557: Now importing already imported modules is up to 2.5 times + faster. - Issue #17596: Include to help with Min GW building. @@ -76,9 +80,6 @@ Library ------- -- Issue #2466: posixpath.ismount now correctly recognizes mount points which - the user does not have permission to access. - - Issue #25958: Support "anti-registration" of special methods from various ABCs, like __hash__, __iter__ or __len__. All these (and several more) can be set to None in an implementation class and the @@ -94,8 +95,8 @@ to ref count problem introduced in code for Issue #27038 in 3.6.0a3. Patch by Xiang Zhang. -- Issue #25628: The *verbose* and *rename* parameters for collections.namedtuple - are now keyword-only. +- Issue #25628: The *verbose* and *rename* parameters for + collections.namedtuple are now keyword-only. - Issue #12345: Add mathemathical constant tau to math and cmath. See also PEP 628. @@ -114,7 +115,8 @@ - Issue #6422: Add autorange method to timeit.Timer objects. -- Issue #27773: Correct some memory management errors server_hostname in _ssl.wrap_socket(). +- Issue #27773: Correct some memory management errors server_hostname in + _ssl.wrap_socket(). - Issue #26750: unittest.mock.create_autospec() now works properly for subclasses of property() and other data descriptors. Removes the never @@ -131,14 +133,15 @@ - In the curses module, raise an error if window.getstr() or window.instr() is passed a negative value. -- Issue #27783: Fix possible usage of uninitialized memory in operator.methodcaller. +- Issue #27783: Fix possible usage of uninitialized memory in + operator.methodcaller. - Issue #27774: Fix possible Py_DECREF on unowned object in _sre. - Issue #27760: Fix possible integer overflow in binascii.b2a_qp. -- Issue #27758: Fix possible integer overflow in the _csv module for large record - lengths. +- Issue #27758: Fix possible integer overflow in the _csv module for large + record lengths. - Issue #27568: Prevent HTTPoxy attack (CVE-2016-1000110). Ignore the HTTP_PROXY variable when REQUEST_METHOD environment is set, which indicates @@ -319,9 +322,9 @@ when exiting, let the new chained one through. This avoids the PEP 479 bug described in issue25782. -- [Security] Issue #27278: Fix os.urandom() implementation using getrandom() on Linux. - Truncate size to INT_MAX and loop until we collected enough random bytes, - instead of casting a directly Py_ssize_t to int. +- [Security] Issue #27278: Fix os.urandom() implementation using getrandom() on + Linux. Truncate size to INT_MAX and loop until we collected enough random + bytes, instead of casting a directly Py_ssize_t to int. - Issue #16864: sqlite3.Cursor.lastrowid now supports REPLACE statement. Initial patch by Alex LordThorsen. @@ -371,9 +374,9 @@ Previously, when IDLE was started from a console or by import, a cascade of warnings was emitted. Patch by Serhiy Storchaka. -- Issue #24137: Run IDLE, test_idle, and htest with tkinter default root disabled. - Fix code and tests that fail with this restriction. - Fix htests to not create a second and redundant root and mainloop. +- Issue #24137: Run IDLE, test_idle, and htest with tkinter default root + disabled. Fix code and tests that fail with this restriction. Fix htests to + not create a second and redundant root and mainloop. - Issue #27310: Fix IDLE.app failure to launch on OS X due to vestigial import. @@ -486,8 +489,8 @@ - [Security] Issue #26556: Update expat to 2.1.1, fixes CVE-2015-1283. -- [Security] Fix TLS stripping vulnerability in smtplib, CVE-2016-0772. Reported by Team - Oststrom +- [Security] Fix TLS stripping vulnerability in smtplib, CVE-2016-0772. + Reported by Team Oststrom. - Issue #21386: Implement missing IPv4Address.is_global property. It was documented since 07a5610bae9d. Initial patch by Roger Luethi. @@ -529,9 +532,10 @@ - Issue #21313: Fix the "platform" module to tolerate when sys.version contains truncated build information. -- [Security] Issue #26839: On Linux, :func:`os.urandom` now calls ``getrandom()`` with - ``GRND_NONBLOCK`` to fall back on reading ``/dev/urandom`` if the urandom - entropy pool is not initialized yet. Patch written by Colm Buckley. +- [Security] Issue #26839: On Linux, :func:`os.urandom` now calls + ``getrandom()`` with ``GRND_NONBLOCK`` to fall back on reading + ``/dev/urandom`` if the urandom entropy pool is not initialized yet. Patch + written by Colm Buckley. - Issue #23883: Added missing APIs to __all__ to match the documented APIs for the following modules: cgi, mailbox, mimetypes, plistlib and smtpd. @@ -893,7 +897,8 @@ This allows sys.getsize() to work correctly with their subclasses with __slots__ defined. -- Issue #25709: Fixed problem with in-place string concatenation and utf-8 cache. +- Issue #25709: Fixed problem with in-place string concatenation and utf-8 + cache. - Issue #5319: New Py_FinalizeEx() API allowing Python to set an exit status of 120 on failure to flush buffered streams. @@ -1124,9 +1129,9 @@ - Issue #24838: tarfile's ustar and gnu formats now correctly calculate name and link field limits for multibyte character encodings like utf-8. -- [Security] Issue #26657: Fix directory traversal vulnerability with http.server on - Windows. This fixes a regression that was introduced in 3.3.4rc1 and - 3.4.0rc1. Based on patch by Philipp Hagemeister. +- [Security] Issue #26657: Fix directory traversal vulnerability with + http.server on Windows. This fixes a regression that was introduced in + 3.3.4rc1 and 3.4.0rc1. Based on patch by Philipp Hagemeister. - Issue #26717: Stop encoding Latin-1-ized WSGI paths with UTF-8. Patch by Anthony Sottile. @@ -1230,8 +1235,8 @@ :class:`warnings.WarningMessage`. Add warnings._showwarnmsg() which uses tracemalloc to get the traceback where source object was allocated. -- [Security] Issue #26313: ssl.py _load_windows_store_certs fails if windows cert store - is empty. Patch by Baji. +- [Security] Issue #26313: ssl.py _load_windows_store_certs fails if windows + cert store is empty. Patch by Baji. - Issue #26569: Fix :func:`pyclbr.readmodule` and :func:`pyclbr.readmodule_ex` to support importing packages. @@ -1266,7 +1271,8 @@ isoformat() method to choose the precision of the time component. - Issue #2202: Fix UnboundLocalError in - AbstractDigestAuthHandler.get_algorithm_impls. Initial patch by Mathieu Dupuy. + AbstractDigestAuthHandler.get_algorithm_impls. Initial patch by Mathieu + Dupuy. - Issue #26167: Minimized overhead in copy.copy() and copy.deepcopy(). Optimized copying and deepcopying bytearrays, NotImplemented, slices, @@ -1313,7 +1319,8 @@ trigger the handle_error() method, and will now to stop a single-threaded server. -- [Security] Issue #25939: On Windows open the cert store readonly in ssl.enum_certificates. +- [Security] Issue #25939: On Windows open the cert store readonly in + ssl.enum_certificates. - Issue #25995: os.walk() no longer uses FDs proportional to the tree depth. @@ -1629,7 +1636,8 @@ - Issue #25011: rlcompleter now omits private and special attribute names unless the prefix starts with underscores. -- Issue #25209: rlcompleter now can add a space or a colon after completed keyword. +- Issue #25209: rlcompleter now can add a space or a colon after completed + keyword. - Issue #22241: timezone.utc name is now plain 'UTC', not 'UTC-00:00'. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 04:03:31 2016 From: python-checkins at python.org (berker.peksag) Date: Fri, 19 Aug 2016 08:03:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327157=3A_Make_onl?= =?utf-8?q?y_type=28=29_itself_accept_the_one-argument_form?= Message-ID: <20160819080331.43130.85830.F86AD6AB@psf.io> https://hg.python.org/cpython/rev/c3498187f998 changeset: 102749:c3498187f998 user: Berker Peksag date: Fri Aug 19 11:04:07 2016 +0300 summary: Issue #27157: Make only type() itself accept the one-argument form Patch by Eryk Sun and Emanuel Barry. files: Doc/library/functions.rst | 3 +++ Lib/test/test_types.py | 18 ++++++++++++++++++ Misc/NEWS | 3 +++ Objects/typeobject.c | 10 ++++++---- 4 files changed, 30 insertions(+), 4 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1463,6 +1463,9 @@ See also :ref:`bltin-type-objects`. + .. versionchanged:: 3.6 + Subclasses of :class:`type` which don't override ``type.__new__`` may no + longer use the one-argument form to get the type of an object. .. function:: vars([object]) diff --git a/Lib/test/test_types.py b/Lib/test/test_types.py --- a/Lib/test/test_types.py +++ b/Lib/test/test_types.py @@ -1000,6 +1000,24 @@ with self.assertRaises(TypeError): X = types.new_class("X", (int(), C)) + def test_one_argument_type(self): + expected_message = 'type.__new__() takes exactly 3 arguments (1 given)' + + # Only type itself can use the one-argument form (#27157) + self.assertIs(type(5), int) + + class M(type): + pass + with self.assertRaises(TypeError) as cm: + M(5) + self.assertEqual(str(cm.exception), expected_message) + + class N(type, metaclass=M): + pass + with self.assertRaises(TypeError) as cm: + N(5) + self.assertEqual(str(cm.exception), expected_message) + class SimpleNamespaceTests(unittest.TestCase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #27157: Make only type() itself accept the one-argument form. + Patch by Eryk Sun and Emanuel Barry. + - Issue #27558: Fix a SystemError in the implementation of "raise" statement. In a brand new thread, raise a RuntimeError since there is no active exception to reraise. Patch written by Xiang Zhang. diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -2287,11 +2287,13 @@ assert(kwds == NULL || PyDict_Check(kwds)); /* Special case: type(x) should return x->ob_type */ - { + /* We only want type itself to accept the one-argument form (#27157) + Note: We don't call PyType_CheckExact as that also allows subclasses */ + if (metatype == &PyType_Type) { const Py_ssize_t nargs = PyTuple_GET_SIZE(args); const Py_ssize_t nkwds = kwds == NULL ? 0 : PyDict_Size(kwds); - if (PyType_CheckExact(metatype) && nargs == 1 && nkwds == 0) { + if (nargs == 1 && nkwds == 0) { PyObject *x = PyTuple_GET_ITEM(args, 0); Py_INCREF(Py_TYPE(x)); return (PyObject *) Py_TYPE(x); @@ -2308,8 +2310,8 @@ } /* Check arguments: (name, bases, dict) */ - if (!PyArg_ParseTuple(args, "UO!O!:type", &name, &PyTuple_Type, &bases, - &PyDict_Type, &orig_dict)) + if (!PyArg_ParseTuple(args, "UO!O!:type.__new__", &name, &PyTuple_Type, + &bases, &PyDict_Type, &orig_dict)) return NULL; /* Determine the proper metatype to deal with this: */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 04:51:42 2016 From: python-checkins at python.org (berker.peksag) Date: Fri, 19 Aug 2016 08:51:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2312946=3A_Merge_from_3=2E5?= Message-ID: <20160819085141.32092.88433.25D933D8@psf.io> https://hg.python.org/cpython/rev/c2e74b88947d changeset: 102751:c2e74b88947d parent: 102749:c3498187f998 parent: 102750:bd2a4c138b76 user: Berker Peksag date: Fri Aug 19 11:52:08 2016 +0300 summary: Issue #12946: Merge from 3.5 files: Doc/c-api/module.rst | 11 +++++++---- 1 files changed, 7 insertions(+), 4 deletions(-) diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -59,10 +59,13 @@ .. index:: single: __dict__ (module attribute) Return the dictionary object that implements *module*'s namespace; this object - is the same as the :attr:`~object.__dict__` attribute of the module object. This - function never fails. It is recommended extensions use other - :c:func:`PyModule_\*` and :c:func:`PyObject_\*` functions rather than directly - manipulate a module's :attr:`~object.__dict__`. + is the same as the :attr:`~object.__dict__` attribute of the module object. + If *module* is not a module object (or a subtype of a module object), + :exc:`SystemError` is raised and *NULL* is returned. + + It is recommended extensions use other :c:func:`PyModule_\*` and + :c:func:`PyObject_\*` functions rather than directly manipulate a module's + :attr:`~object.__dict__`. .. c:function:: PyObject* PyModule_GetNameObject(PyObject *module) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 04:51:42 2016 From: python-checkins at python.org (berker.peksag) Date: Fri, 19 Aug 2016 08:51:42 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzEyOTQ2?= =?utf-8?q?=3A_Document_that_PyModule=5FGetDict_can_fail_in_some_cases?= Message-ID: <20160819085141.31811.20825.F292A8AB@psf.io> https://hg.python.org/cpython/rev/bd2a4c138b76 changeset: 102750:bd2a4c138b76 branch: 3.5 parent: 102747:bad82d4bdf8c user: Berker Peksag date: Fri Aug 19 11:51:39 2016 +0300 summary: Issue #12946: Document that PyModule_GetDict can fail in some cases files: Doc/c-api/module.rst | 11 +++++++---- 1 files changed, 7 insertions(+), 4 deletions(-) diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -59,10 +59,13 @@ .. index:: single: __dict__ (module attribute) Return the dictionary object that implements *module*'s namespace; this object - is the same as the :attr:`~object.__dict__` attribute of the module object. This - function never fails. It is recommended extensions use other - :c:func:`PyModule_\*` and :c:func:`PyObject_\*` functions rather than directly - manipulate a module's :attr:`~object.__dict__`. + is the same as the :attr:`~object.__dict__` attribute of the module object. + If *module* is not a module object (or a subtype of a module object), + :exc:`SystemError` is raised and *NULL* is returned. + + It is recommended extensions use other :c:func:`PyModule_\*` and + :c:func:`PyObject_\*` functions rather than directly manipulate a module's + :attr:`~object.__dict__`. .. c:function:: PyObject* PyModule_GetNameObject(PyObject *module) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 04:59:36 2016 From: python-checkins at python.org (berker.peksag) Date: Fri, 19 Aug 2016 08:59:36 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2312946=3A_Remove_d?= =?utf-8?q?ead_code_in_PyModule=5FGetDict?= Message-ID: <20160819085936.24226.73105.CD85DE46@psf.io> https://hg.python.org/cpython/rev/4b64a049f451 changeset: 102752:4b64a049f451 user: Berker Peksag date: Fri Aug 19 12:00:13 2016 +0300 summary: Issue #12946: Remove dead code in PyModule_GetDict PyModule_NewObject already sets md_dict to PyDict_New(): m->md_dict = PyDict_New(); files: Objects/moduleobject.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -444,8 +444,7 @@ return NULL; } d = ((PyModuleObject *)m) -> md_dict; - if (d == NULL) - ((PyModuleObject *)m) -> md_dict = d = PyDict_New(); + assert(d != NULL); return d; } -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Fri Aug 19 09:03:09 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Fri, 19 Aug 2016 14:03:09 +0100 Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-08-19 Message-ID: <1f557094-efe5-4512-bd74-fe111784641b@irsmsx101.ger.corp.intel.com> No new revisions. Here are the previous results: Results for project Python 2.7, build date 2016-08-19 02:45:54 +0000 commit: 11a9bca71528 previous commit: cddd633b959f revision date: 2016-08-17 06:35:35 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.14% -1.08% 3.62% 7.12% :-) pybench 0.20% -0.02% 5.76% 4.57% :-( regex_v8 0.64% -0.05% -2.20% 10.88% :-) nbody 0.26% -0.05% 8.81% 2.51% :-) json_dump_v2 0.30% 0.38% 2.39% 10.86% :-| normal_startup 0.68% 0.32% -0.29% 1.67% :-) ssbench 0.22% 0.09% 2.40% 1.70% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-08-19/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Fri Aug 19 09:04:11 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Fri, 19 Aug 2016 14:04:11 +0100 Subject: [Python-checkins] UGLY Benchmark Results for Python Default 2016-08-19 Message-ID: <42034ad7-c9dc-4cc2-aa0a-47ede0e00087@irsmsx101.ger.corp.intel.com> Results for project Python default, build date 2016-08-19 02:00:18 +0000 commit: dd0f1fa809c5 previous commit: 617104a6b759 revision date: 2016-08-19 01:45:12 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.16% -1.03% 11.22% 16.61% :-) pybench 0.14% -0.08% 6.22% 6.69% :-( regex_v8 2.61% 0.34% -3.18% 7.83% :-| nbody 0.12% -0.06% 1.66% 9.45% :-| json_dump_v2 0.26% 1.07% -0.06% 10.53% :-| normal_startup 0.67% 0.75% 1.59% 5.91% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-default-2016-08-19/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Fri Aug 19 09:16:46 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 13:16:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_a_clang_warning_in_gra?= =?utf-8?q?mmar=2Ec?= Message-ID: <20160819131645.6628.92817.D541E7A9@psf.io> https://hg.python.org/cpython/rev/ea00c88f7f42 changeset: 102753:ea00c88f7f42 user: Victor Stinner date: Fri Aug 19 15:11:56 2016 +0200 summary: Fix a clang warning in grammar.c Clang is smarter than GCC and emits a warning for dead code after a function declared with __attribute__((__noreturn__)) (Py_FatalError). files: Parser/grammar.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Parser/grammar.c b/Parser/grammar.c --- a/Parser/grammar.c +++ b/Parser/grammar.c @@ -122,7 +122,13 @@ } fprintf(stderr, "Label %d/'%s' not found\n", type, str); Py_FatalError("grammar.c:findlabel()"); + + /* Py_FatalError() is declared with __attribute__((__noreturn__)). + GCC emits a warning without "return 0;" (compiler bug!), but Clang is + smarter and emits a warning on the return... */ +#ifndef __clang__ return 0; /* Make gcc -Wall happy */ +#endif } /* Forward */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 10:58:56 2016 From: python-checkins at python.org (berker.peksag) Date: Fri, 19 Aug 2016 14:58:56 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3ODAx?= =?utf-8?q?=3A_Skip_test=5Fupdate=5Flines=5Fcols_when_update=5Flines=5Fcol?= =?utf-8?q?s=28=29_is_not?= Message-ID: <20160819145856.31995.58899.0840E8EC@psf.io> https://hg.python.org/cpython/rev/51368991e858 changeset: 102754:51368991e858 branch: 3.5 parent: 102750:bd2a4c138b76 user: Berker Peksag date: Fri Aug 19 17:59:01 2016 +0300 summary: Issue #27801: Skip test_update_lines_cols when update_lines_cols() is not available files: Lib/test/test_curses.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py --- a/Lib/test/test_curses.py +++ b/Lib/test/test_curses.py @@ -395,6 +395,7 @@ class MiscTests(unittest.TestCase): + @requires_curses_func('update_lines_cols') def test_update_lines_cols(self): # this doesn't actually test that LINES and COLS are updated, # because we can't automate changing them. See Issue #4254 for -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 10:58:57 2016 From: python-checkins at python.org (berker.peksag) Date: Fri, 19 Aug 2016 14:58:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327801=3A_Merge_from_3=2E5?= Message-ID: <20160819145856.43244.27429.5CD687B9@psf.io> https://hg.python.org/cpython/rev/bd9183850965 changeset: 102755:bd9183850965 parent: 102753:ea00c88f7f42 parent: 102754:51368991e858 user: Berker Peksag date: Fri Aug 19 17:59:31 2016 +0300 summary: Issue #27801: Merge from 3.5 files: Lib/test/test_curses.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py --- a/Lib/test/test_curses.py +++ b/Lib/test/test_curses.py @@ -395,6 +395,7 @@ class MiscTests(unittest.TestCase): + @requires_curses_func('update_lines_cols') def test_update_lines_cols(self): # this doesn't actually test that LINES and COLS are updated, # because we can't automate changing them. See Issue #4254 for -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 11:32:17 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 15:32:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Cleanup_callmethod=28=29?= Message-ID: <20160819153216.43438.8254.C1A3E5F0@psf.io> https://hg.python.org/cpython/rev/adceb14cab96 changeset: 102760:adceb14cab96 user: Victor Stinner date: Fri Aug 19 16:56:49 2016 +0200 summary: Cleanup callmethod() Make callmethod() less weird: don't decrement func reference counter, the caller is now responsible to do that. Issue #27128. files: Objects/abstract.c | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-) diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2343,9 +2343,10 @@ { PyObject *args, *result; + assert(func != NULL); + if (!PyCallable_Check(func)) { type_error("attribute of type '%.200s' is not callable", func); - Py_XDECREF(func); return NULL; } @@ -2363,7 +2364,6 @@ } result = call_function_tail(func, args); - Py_XDECREF(func); Py_DECREF(args); return result; } @@ -2385,6 +2385,7 @@ va_start(va, format); retval = callmethod(func, format, va, 0); va_end(va); + Py_DECREF(func); return retval; } @@ -2406,6 +2407,7 @@ va_start(va, format); retval = callmethod(func, format, va, 0); va_end(va); + Py_DECREF(func); return retval; } @@ -2426,6 +2428,7 @@ va_start(va, format); retval = callmethod(func, format, va, 1); va_end(va); + Py_DECREF(func); return retval; } @@ -2447,6 +2450,7 @@ va_start(va, format); retval = callmethod(func, format, va, 1); va_end(va); + Py_DECREF(func); return retval; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 11:32:17 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 15:32:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_call=5Ffunction=5Ftail=28?= =?utf-8?q?=29_uses_fast_call?= Message-ID: <20160819153216.6440.46236.907E35AA@psf.io> https://hg.python.org/cpython/rev/7cd479573de9 changeset: 102758:7cd479573de9 user: Victor Stinner date: Fri Aug 19 16:44:19 2016 +0200 summary: call_function_tail() uses fast call Issue #27128: Modify call_function_tail() to use _PyObject_FastCall() when args is not a tuple to avoid the creation of a temporary tuple. call_function_tail() is used by: * PyObject_CallFunction() * PyObject_CallMethod() * _PyObject_CallMethodId() files: Objects/abstract.c | 19 ++++++------------- 1 files changed, 6 insertions(+), 13 deletions(-) diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2272,27 +2272,20 @@ static PyObject* call_function_tail(PyObject *callable, PyObject *args) { - PyObject *retval; + PyObject *result; if (args == NULL) return NULL; if (!PyTuple_Check(args)) { - PyObject *a; - - a = PyTuple_New(1); - if (a == NULL) { - Py_DECREF(args); - return NULL; - } - PyTuple_SET_ITEM(a, 0, args); - args = a; + result = _PyObject_FastCall(callable, &args, 1, NULL); } - retval = PyObject_Call(callable, args, NULL); + else { + result = PyObject_Call(callable, args, NULL); + } Py_DECREF(args); - - return retval; + return result; } PyObject * -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 11:32:17 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 15:32:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_=5FPyObject=5FFastCall?= =?utf-8?b?KCk=?= Message-ID: <20160819153215.24246.70405.B5844DFA@psf.io> https://hg.python.org/cpython/rev/a1a29d20f52d changeset: 102756:a1a29d20f52d user: Victor Stinner date: Fri Aug 19 16:11:43 2016 +0200 summary: Add _PyObject_FastCall() Issue #27128: Add _PyObject_FastCall(), a new calling convention avoiding a temporary tuple to pass positional parameters in most cases, but create a temporary tuple if needed (ex: for the tp_call slot). The API is prepared to support keyword parameters, but the full implementation will come later (_PyFunction_FastCall() doesn't support keyword parameters yet). Add also: * _PyStack_AsTuple() helper function: convert a "stack" of parameters to a tuple. * _PyCFunction_FastCall(): fast call implementation for C functions * _PyFunction_FastCall(): fast call implementation for Python functions files: Include/abstract.h | 18 ++- Include/funcobject.h | 7 + Include/methodobject.h | 6 + Objects/abstract.c | 76 +++++++++++++ Objects/methodobject.c | 93 ++++++++++++++++ Python/ceval.c | 165 ++++++++++++++++++++-------- 6 files changed, 315 insertions(+), 50 deletions(-) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -267,10 +267,26 @@ PyObject *args, PyObject *kw); #ifndef Py_LIMITED_API + PyAPI_FUNC(PyObject*) _PyStack_AsTuple(PyObject **stack, + Py_ssize_t nargs); + + /* Call the callable object func with the "fast call" calling convention: + args is a C array for positional parameters (nargs is the number of + positional paramater), kwargs is a dictionary for keyword parameters. + + If nargs is equal to zero, args can be NULL. kwargs can be NULL. + nargs must be greater or equal to zero. + + Return the result on success. Raise an exception on return NULL on + error. */ + PyAPI_FUNC(PyObject *) _PyObject_FastCall(PyObject *func, + PyObject **args, int nargs, + PyObject *kwargs); + PyAPI_FUNC(PyObject *) _Py_CheckFunctionResult(PyObject *func, PyObject *result, const char *where); -#endif +#endif /* Py_LIMITED_API */ /* Call a callable Python object, callable_object, with diff --git a/Include/funcobject.h b/Include/funcobject.h --- a/Include/funcobject.h +++ b/Include/funcobject.h @@ -58,6 +58,13 @@ PyAPI_FUNC(PyObject *) PyFunction_GetAnnotations(PyObject *); PyAPI_FUNC(int) PyFunction_SetAnnotations(PyObject *, PyObject *); +#ifndef Py_LIMITED_API +PyAPI_FUNC(PyObject *) _PyFunction_FastCall( + PyObject *func, + PyObject **args, int nargs, + PyObject *kwargs); +#endif + /* Macros for direct access to these values. Type checks are *not* done, so use with care. */ #define PyFunction_GET_CODE(func) \ diff --git a/Include/methodobject.h b/Include/methodobject.h --- a/Include/methodobject.h +++ b/Include/methodobject.h @@ -37,6 +37,12 @@ #endif PyAPI_FUNC(PyObject *) PyCFunction_Call(PyObject *, PyObject *, PyObject *); +#ifndef Py_LIMITED_API +PyAPI_FUNC(PyObject *) _PyCFunction_FastCall(PyObject *func, + PyObject **args, int nargs, + PyObject *kwargs); +#endif + struct PyMethodDef { const char *ml_name; /* The name of the built-in function/method */ PyCFunction ml_meth; /* The C function that implements it */ diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2193,6 +2193,82 @@ return _Py_CheckFunctionResult(func, result, NULL); } +PyObject* +_PyStack_AsTuple(PyObject **stack, Py_ssize_t nargs) +{ + PyObject *args; + Py_ssize_t i; + + args = PyTuple_New(nargs); + if (args == NULL) { + return NULL; + } + + for (i=0; i < nargs; i++) { + PyObject *item = stack[i]; + Py_INCREF(item); + PyTuple_SET_ITEM(args, i, item); + } + + return args; +} + +PyObject * +_PyObject_FastCall(PyObject *func, PyObject **args, int nargs, PyObject *kwargs) +{ + ternaryfunc call; + PyObject *result = NULL; + + /* _PyObject_FastCall() must not be called with an exception set, + because it may clear it (directly or indirectly) and so the + caller loses its exception */ + assert(!PyErr_Occurred()); + + assert(func != NULL); + assert(nargs >= 0); + assert(nargs == 0 || args != NULL); + /* issue #27128: support for keywords will come later: + _PyFunction_FastCall() doesn't support keyword arguments yet */ + assert(kwargs == NULL); + + if (Py_EnterRecursiveCall(" while calling a Python object")) { + return NULL; + } + + if (PyFunction_Check(func)) { + result = _PyFunction_FastCall(func, args, nargs, kwargs); + } + else if (PyCFunction_Check(func)) { + result = _PyCFunction_FastCall(func, args, nargs, kwargs); + } + else { + PyObject *tuple; + + /* Slow-path: build a temporary tuple */ + call = func->ob_type->tp_call; + if (call == NULL) { + PyErr_Format(PyExc_TypeError, "'%.200s' object is not callable", + func->ob_type->tp_name); + goto exit; + } + + tuple = _PyStack_AsTuple(args, nargs); + if (tuple == NULL) { + goto exit; + } + + result = (*call)(func, tuple, kwargs); + Py_DECREF(tuple); + } + + result = _Py_CheckFunctionResult(func, result, NULL); + +exit: + Py_LeaveRecursiveCall(); + + return result; +} + static PyObject* call_function_tail(PyObject *callable, PyObject *args) { diff --git a/Objects/methodobject.c b/Objects/methodobject.c --- a/Objects/methodobject.c +++ b/Objects/methodobject.c @@ -145,6 +145,99 @@ return _Py_CheckFunctionResult(func, res, NULL); } +PyObject * +_PyCFunction_FastCall(PyObject *func_obj, PyObject **args, int nargs, + PyObject *kwargs) +{ + PyCFunctionObject* func = (PyCFunctionObject*)func_obj; + PyCFunction meth = PyCFunction_GET_FUNCTION(func); + PyObject *self = PyCFunction_GET_SELF(func); + PyObject *result; + int flags; + + /* _PyCFunction_FastCall() must not be called with an exception set, + because it may clear it (directly or indirectly) and so the + caller loses its exception */ + assert(!PyErr_Occurred()); + + flags = PyCFunction_GET_FLAGS(func) & ~(METH_CLASS | METH_STATIC | METH_COEXIST); + + switch (flags) + { + case METH_NOARGS: + if (kwargs != NULL && PyDict_Size(kwargs) != 0) { + PyErr_Format(PyExc_TypeError, "%.200s() takes no keyword arguments", + func->m_ml->ml_name); + return NULL; + } + + if (nargs != 0) { + PyErr_Format(PyExc_TypeError, + "%.200s() takes no arguments (%zd given)", + func->m_ml->ml_name, nargs); + return NULL; + } + + result = (*meth) (self, NULL); + break; + + case METH_O: + if (kwargs != NULL && PyDict_Size(kwargs) != 0) { + PyErr_Format(PyExc_TypeError, "%.200s() takes no keyword arguments", + func->m_ml->ml_name); + return NULL; + } + + if (nargs != 1) { + PyErr_Format(PyExc_TypeError, + "%.200s() takes exactly one argument (%zd given)", + func->m_ml->ml_name, nargs); + return NULL; + } + + result = (*meth) (self, args[0]); + break; + + case METH_VARARGS: + case METH_VARARGS | METH_KEYWORDS: + { + /* Slow-path: create a temporary tuple */ + PyObject *tuple; + + if (!(flags & METH_KEYWORDS) && kwargs != NULL && PyDict_Size(kwargs) != 0) { + PyErr_Format(PyExc_TypeError, + "%.200s() takes no keyword arguments", + func->m_ml->ml_name); + return NULL; + } + + tuple = _PyStack_AsTuple(args, nargs); + if (tuple == NULL) { + return NULL; + } + + if (flags & METH_KEYWORDS) { + result = (*(PyCFunctionWithKeywords)meth) (self, tuple, kwargs); + } + else { + result = (*meth) (self, tuple); + } + Py_DECREF(tuple); + break; + } + + default: + PyErr_SetString(PyExc_SystemError, + "Bad call flags in PyCFunction_Call. " + "METH_OLDARGS is no longer supported!"); + return NULL; + } + + result = _Py_CheckFunctionResult(func_obj, result, NULL); + + return result; +} + /* Methods (the standard built-in methods, that is) */ static void diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -113,7 +113,7 @@ #else static PyObject * call_function(PyObject ***, int); #endif -static PyObject * fast_function(PyObject *, PyObject ***, int, int, int); +static PyObject * fast_function(PyObject *, PyObject **, int, int, int); static PyObject * do_call(PyObject *, PyObject ***, int, int); static PyObject * ext_do_call(PyObject *, PyObject ***, int, int, int); static PyObject * update_keyword_args(PyObject *, int, PyObject ***, @@ -3779,6 +3779,7 @@ Py_DECREF(kwonly_sig); } + /* This is gonna seem *real weird*, but if you put some other code between PyEval_EvalFrame() and PyEval_EvalCodeEx() you will need to adjust the test in the if statements in Misc/gdbinit (pystack and pystackv). */ @@ -4068,8 +4069,10 @@ PyObject **defs, int defcount, PyObject *kwdefs, PyObject *closure) { return _PyEval_EvalCodeWithName(_co, globals, locals, - args, argcount, kws, kwcount, - defs, defcount, kwdefs, closure, + args, argcount, + kws, kwcount, + defs, defcount, + kwdefs, closure, NULL, NULL); } @@ -4757,10 +4760,12 @@ } else Py_INCREF(func); READ_TIMESTAMP(*pintr0); - if (PyFunction_Check(func)) - x = fast_function(func, pp_stack, n, na, nk); - else + if (PyFunction_Check(func)) { + x = fast_function(func, (*pp_stack) - n, n, na, nk); + } + else { x = do_call(func, pp_stack, na, nk); + } READ_TIMESTAMP(*pintr1); Py_DECREF(func); @@ -4790,62 +4795,124 @@ done before evaluating the frame. */ +static PyObject* +_PyFunction_FastCallNoKw(PyObject **args, Py_ssize_t na, + PyCodeObject *co, PyObject *globals) +{ + PyFrameObject *f; + PyThreadState *tstate = PyThreadState_GET(); + PyObject **fastlocals; + Py_ssize_t i; + PyObject *result; + + PCALL(PCALL_FASTER_FUNCTION); + assert(globals != NULL); + /* XXX Perhaps we should create a specialized + PyFrame_New() that doesn't take locals, but does + take builtins without sanity checking them. + */ + assert(tstate != NULL); + f = PyFrame_New(tstate, co, globals, NULL); + if (f == NULL) { + return NULL; + } + + fastlocals = f->f_localsplus; + + for (i = 0; i < na; i++) { + Py_INCREF(*args); + fastlocals[i] = *args++; + } + result = PyEval_EvalFrameEx(f,0); + + ++tstate->recursion_depth; + Py_DECREF(f); + --tstate->recursion_depth; + + return result; +} + static PyObject * -fast_function(PyObject *func, PyObject ***pp_stack, int n, int na, int nk) +fast_function(PyObject *func, PyObject **stack, int n, int na, int nk) { PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func); PyObject *globals = PyFunction_GET_GLOBALS(func); PyObject *argdefs = PyFunction_GET_DEFAULTS(func); - PyObject *kwdefs = PyFunction_GET_KW_DEFAULTS(func); - PyObject *name = ((PyFunctionObject *)func) -> func_name; - PyObject *qualname = ((PyFunctionObject *)func) -> func_qualname; - PyObject **d = NULL; - int nd = 0; + PyObject *kwdefs, *closure, *name, *qualname; + PyObject **d; + int nd; PCALL(PCALL_FUNCTION); PCALL(PCALL_FAST_FUNCTION); - if (argdefs == NULL && co->co_argcount == n && - co->co_kwonlyargcount == 0 && nk==0 && - co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) { - PyFrameObject *f; - PyObject *retval = NULL; - PyThreadState *tstate = PyThreadState_GET(); - PyObject **fastlocals, **stack; - int i; - - PCALL(PCALL_FASTER_FUNCTION); - assert(globals != NULL); - /* XXX Perhaps we should create a specialized - PyFrame_New() that doesn't take locals, but does - take builtins without sanity checking them. - */ - assert(tstate != NULL); - f = PyFrame_New(tstate, co, globals, NULL); - if (f == NULL) - return NULL; - - fastlocals = f->f_localsplus; - stack = (*pp_stack) - n; - - for (i = 0; i < n; i++) { - Py_INCREF(*stack); - fastlocals[i] = *stack++; - } - retval = PyEval_EvalFrameEx(f,0); - ++tstate->recursion_depth; - Py_DECREF(f); - --tstate->recursion_depth; - return retval; + + if (argdefs == NULL && co->co_argcount == na && + co->co_kwonlyargcount == 0 && nk == 0 && + co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) + { + return _PyFunction_FastCallNoKw(stack, na, co, globals); } + + kwdefs = PyFunction_GET_KW_DEFAULTS(func); + closure = PyFunction_GET_CLOSURE(func); + name = ((PyFunctionObject *)func) -> func_name; + qualname = ((PyFunctionObject *)func) -> func_qualname; + if (argdefs != NULL) { d = &PyTuple_GET_ITEM(argdefs, 0); nd = Py_SIZE(argdefs); } - return _PyEval_EvalCodeWithName((PyObject*)co, globals, - (PyObject *)NULL, (*pp_stack)-n, na, - (*pp_stack)-2*nk, nk, d, nd, kwdefs, - PyFunction_GET_CLOSURE(func), - name, qualname); + else { + d = NULL; + nd = 0; + } + return _PyEval_EvalCodeWithName((PyObject*)co, globals, (PyObject *)NULL, + stack, na, + stack + na, nk, + d, nd, kwdefs, + closure, name, qualname); +} + +PyObject * +_PyFunction_FastCall(PyObject *func, PyObject **args, int nargs, PyObject *kwargs) +{ + PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func); + PyObject *globals = PyFunction_GET_GLOBALS(func); + PyObject *argdefs = PyFunction_GET_DEFAULTS(func); + PyObject *kwdefs, *closure, *name, *qualname; + PyObject **d; + int nd; + + PCALL(PCALL_FUNCTION); + PCALL(PCALL_FAST_FUNCTION); + + /* issue #27128: support for keywords will come later */ + assert(kwargs == NULL); + + if (argdefs == NULL && co->co_argcount == nargs && + co->co_kwonlyargcount == 0 && + co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) + { + return _PyFunction_FastCallNoKw(args, nargs, co, globals); + } + + kwdefs = PyFunction_GET_KW_DEFAULTS(func); + closure = PyFunction_GET_CLOSURE(func); + name = ((PyFunctionObject *)func) -> func_name; + qualname = ((PyFunctionObject *)func) -> func_qualname; + + if (argdefs != NULL) { + d = &PyTuple_GET_ITEM(argdefs, 0); + nd = Py_SIZE(argdefs); + } + else { + d = NULL; + nd = 0; + } + return _PyEval_EvalCodeWithName((PyObject*)co, globals, (PyObject *)NULL, + args, nargs, + NULL, 0, + d, nd, kwdefs, + closure, name, qualname); } static PyObject * -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 11:32:17 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 15:32:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Cleanup_call=5Ffunction=5F?= =?utf-8?b?dGFpbCgp?= Message-ID: <20160819153216.19989.26759.6757E73D@psf.io> https://hg.python.org/cpython/rev/34af2edface9 changeset: 102759:34af2edface9 user: Victor Stinner date: Fri Aug 19 16:50:49 2016 +0200 summary: Cleanup call_function_tail() Make call_function_tail() less weird: don't decrement args reference counter, the caller is now responsible to do that. The caller now also checks if args is NULL. Issue #27128. files: Objects/abstract.c | 55 +++++++++++++++++++-------------- 1 files changed, 32 insertions(+), 23 deletions(-) diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2274,8 +2274,7 @@ { PyObject *result; - if (args == NULL) - return NULL; + assert(args != NULL); if (!PyTuple_Check(args)) { result = _PyObject_FastCall(callable, &args, 1, NULL); @@ -2284,7 +2283,6 @@ result = PyObject_Call(callable, args, NULL); } - Py_DECREF(args); return result; } @@ -2292,7 +2290,7 @@ PyObject_CallFunction(PyObject *callable, const char *format, ...) { va_list va; - PyObject *args; + PyObject *args, *result; if (callable == NULL) return null_error(); @@ -2302,19 +2300,23 @@ args = Py_VaBuildValue(format, va); va_end(va); } - else + else { args = PyTuple_New(0); - if (args == NULL) + } + if (args == NULL) { return NULL; - - return call_function_tail(callable, args); + } + + result = call_function_tail(callable, args); + Py_DECREF(args); + return result; } PyObject * _PyObject_CallFunction_SizeT(PyObject *callable, const char *format, ...) { va_list va; - PyObject *args; + PyObject *args, *result; if (callable == NULL) return null_error(); @@ -2324,21 +2326,27 @@ args = _Py_VaBuildValue_SizeT(format, va); va_end(va); } - else + else { args = PyTuple_New(0); - - return call_function_tail(callable, args); + } + if (args == NULL) { + return NULL; + } + + result = call_function_tail(callable, args); + Py_DECREF(args); + return result; } static PyObject* callmethod(PyObject* func, const char *format, va_list va, int is_size_t) { - PyObject *retval = NULL; - PyObject *args; + PyObject *args, *result; if (!PyCallable_Check(func)) { type_error("attribute of type '%.200s' is not callable", func); - goto exit; + Py_XDECREF(func); + return NULL; } if (format && *format) { @@ -2347,16 +2355,17 @@ else args = Py_VaBuildValue(format, va); } - else + else { args = PyTuple_New(0); - - retval = call_function_tail(func, args); - - exit: - /* args gets consumed in call_function_tail */ + } + if (args == NULL) { + return NULL; + } + + result = call_function_tail(func, args); Py_XDECREF(func); - - return retval; + Py_DECREF(args); + return result; } PyObject * -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 11:32:17 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 15:32:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_PyEval=5FCallObjectWithKey?= =?utf-8?q?words=28=29_uses_fast_call?= Message-ID: <20160819153216.24226.6586.744E40E8@psf.io> https://hg.python.org/cpython/rev/89e4ad001f3d changeset: 102757:89e4ad001f3d user: Victor Stinner date: Fri Aug 19 16:42:42 2016 +0200 summary: PyEval_CallObjectWithKeywords() uses fast call Issue #27128: Modify PyEval_CallObjectWithKeywords() to use _PyObject_FastCall() when args==NULL and kw==NULL. It avoids the creation of a temporary empty tuple for positional arguments. files: Python/ceval.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4592,6 +4592,10 @@ #endif if (arg == NULL) { + if (kw == NULL) { + return _PyObject_FastCall(func, NULL, 0, 0); + } + arg = PyTuple_New(0); if (arg == NULL) return NULL; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 11:32:21 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 15:32:21 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbjogUEVQIDc6IGFkZCB7Li4ufSBh?= =?utf-8?q?round_null=5Ferror=28=29_in_abstract=2Ec?= Message-ID: <20160819153221.55291.37733.D4717282@psf.io> https://hg.python.org/cpython/rev/10f1a4910adb changeset: 102761:10f1a4910adb user: Victor Stinner date: Fri Aug 19 16:59:55 2016 +0200 summary: PEP 7: add {...} around null_error() in abstract.c Issue #27128. files: Objects/abstract.c | 93 +++++++++++++++++++++++---------- 1 files changed, 65 insertions(+), 28 deletions(-) diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -32,8 +32,10 @@ { PyObject *v; - if (o == NULL) + if (o == NULL) { return null_error(); + } + v = (PyObject *)o->ob_type; Py_INCREF(v); return v; @@ -137,8 +139,9 @@ { PyMappingMethods *m; - if (o == NULL || key == NULL) + if (o == NULL || key == NULL) { return null_error(); + } m = o->ob_type->tp_as_mapping; if (m && m->mp_subscript) { @@ -1125,8 +1128,10 @@ { PyNumberMethods *m; - if (o == NULL) + if (o == NULL) { return null_error(); + } + m = o->ob_type->tp_as_number; if (m && m->nb_negative) return (*m->nb_negative)(o); @@ -1139,8 +1144,10 @@ { PyNumberMethods *m; - if (o == NULL) + if (o == NULL) { return null_error(); + } + m = o->ob_type->tp_as_number; if (m && m->nb_positive) return (*m->nb_positive)(o); @@ -1153,8 +1160,10 @@ { PyNumberMethods *m; - if (o == NULL) + if (o == NULL) { return null_error(); + } + m = o->ob_type->tp_as_number; if (m && m->nb_invert) return (*m->nb_invert)(o); @@ -1167,8 +1176,10 @@ { PyNumberMethods *m; - if (o == NULL) + if (o == NULL) { return null_error(); + } + m = o->ob_type->tp_as_number; if (m && m->nb_absolute) return m->nb_absolute(o); @@ -1184,8 +1195,10 @@ PyNumber_Index(PyObject *item) { PyObject *result = NULL; - if (item == NULL) + if (item == NULL) { return null_error(); + } + if (PyLong_Check(item)) { Py_INCREF(item); return item; @@ -1273,8 +1286,10 @@ Py_buffer view; _Py_IDENTIFIER(__trunc__); - if (o == NULL) + if (o == NULL) { return null_error(); + } + if (PyLong_CheckExact(o)) { Py_INCREF(o); return o; @@ -1349,8 +1364,10 @@ { PyNumberMethods *m; - if (o == NULL) + if (o == NULL) { return null_error(); + } + if (PyFloat_CheckExact(o)) { Py_INCREF(o); return o; @@ -1451,8 +1468,9 @@ { PySequenceMethods *m; - if (s == NULL || o == NULL) + if (s == NULL || o == NULL) { return null_error(); + } m = s->ob_type->tp_as_sequence; if (m && m->sq_concat) @@ -1475,8 +1493,9 @@ { PySequenceMethods *m; - if (o == NULL) + if (o == NULL) { return null_error(); + } m = o->ob_type->tp_as_sequence; if (m && m->sq_repeat) @@ -1504,8 +1523,9 @@ { PySequenceMethods *m; - if (s == NULL || o == NULL) + if (s == NULL || o == NULL) { return null_error(); + } m = s->ob_type->tp_as_sequence; if (m && m->sq_inplace_concat) @@ -1528,8 +1548,9 @@ { PySequenceMethods *m; - if (o == NULL) + if (o == NULL) { return null_error(); + } m = o->ob_type->tp_as_sequence; if (m && m->sq_inplace_repeat) @@ -1557,8 +1578,9 @@ { PySequenceMethods *m; - if (s == NULL) + if (s == NULL) { return null_error(); + } m = s->ob_type->tp_as_sequence; if (m && m->sq_item) { @@ -1583,7 +1605,9 @@ { PyMappingMethods *mp; - if (!s) return null_error(); + if (!s) { + return null_error(); + } mp = s->ob_type->tp_as_mapping; if (mp && mp->mp_subscript) { @@ -1710,8 +1734,9 @@ PyObject *result = NULL; Py_ssize_t j; - if (v == NULL) + if (v == NULL) { return null_error(); + } /* Special-case the common tuple and list cases, for efficiency. */ if (PyTuple_CheckExact(v)) { @@ -1791,8 +1816,9 @@ PyObject *result; /* result list */ PyObject *rv; /* return value from PyList_Extend */ - if (v == NULL) + if (v == NULL) { return null_error(); + } result = PyList_New(0); if (result == NULL) @@ -1812,8 +1838,9 @@ { PyObject *it; - if (v == NULL) + if (v == NULL) { return null_error(); + } if (PyList_CheckExact(v) || PyTuple_CheckExact(v)) { Py_INCREF(v); @@ -1996,8 +2023,9 @@ { PyObject *okey, *r; - if (key == NULL) + if (key == NULL) { return null_error(); + } okey = PyUnicode_FromString(key); if (okey == NULL) @@ -2292,8 +2320,9 @@ va_list va; PyObject *args, *result; - if (callable == NULL) + if (callable == NULL) { return null_error(); + } if (format && *format) { va_start(va, format); @@ -2318,8 +2347,9 @@ va_list va; PyObject *args, *result; - if (callable == NULL) + if (callable == NULL) { return null_error(); + } if (format && *format) { va_start(va, format); @@ -2375,8 +2405,9 @@ PyObject *func = NULL; PyObject *retval = NULL; - if (o == NULL || name == NULL) + if (o == NULL || name == NULL) { return null_error(); + } func = PyObject_GetAttrString(o, name); if (func == NULL) @@ -2397,8 +2428,9 @@ PyObject *func = NULL; PyObject *retval = NULL; - if (o == NULL || name == NULL) + if (o == NULL || name == NULL) { return null_error(); + } func = _PyObject_GetAttrId(o, name); if (func == NULL) @@ -2419,8 +2451,9 @@ PyObject *func = NULL; PyObject *retval; - if (o == NULL || name == NULL) + if (o == NULL || name == NULL) { return null_error(); + } func = PyObject_GetAttrString(o, name); if (func == NULL) @@ -2440,8 +2473,9 @@ PyObject *func = NULL; PyObject *retval; - if (o == NULL || name == NULL) + if (o == NULL || name == NULL) { return null_error(); + } func = _PyObject_GetAttrId(o, name); if (func == NULL) { @@ -2482,8 +2516,9 @@ PyObject *args, *tmp; va_list vargs; - if (callable == NULL || name == NULL) + if (callable == NULL || name == NULL) { return null_error(); + } callable = PyObject_GetAttr(callable, name); if (callable == NULL) @@ -2511,8 +2546,9 @@ PyObject *args, *tmp; va_list vargs; - if (callable == NULL || name == NULL) + if (callable == NULL || name == NULL) { return null_error(); + } callable = _PyObject_GetAttrId(callable, name); if (callable == NULL) @@ -2539,8 +2575,9 @@ PyObject *args, *tmp; va_list vargs; - if (callable == NULL) + if (callable == NULL) { return null_error(); + } /* count the args */ va_start(vargs, callable); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 11:32:22 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 15:32:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_PyObject=5FCall=28=29_?= =?utf-8?q?parameter_names?= Message-ID: <20160819153222.85066.69904.12B4558F@psf.io> https://hg.python.org/cpython/rev/f1ad6f64a11e changeset: 102763:f1ad6f64a11e user: Victor Stinner date: Fri Aug 19 17:12:23 2016 +0200 summary: Fix PyObject_Call() parameter names Issue #27128: arg=>args, kw=>kwargs. Same change for PyEval_CallObjectWithKeywords(). files: Include/abstract.h | 2 +- Include/ceval.h | 2 +- Objects/abstract.c | 6 ++++-- Python/ceval.c | 27 ++++++++++++++------------- 4 files changed, 20 insertions(+), 17 deletions(-) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -264,7 +264,7 @@ */ PyAPI_FUNC(PyObject *) PyObject_Call(PyObject *callable_object, - PyObject *args, PyObject *kw); + PyObject *args, PyObject *kwargs); #ifndef Py_LIMITED_API PyAPI_FUNC(PyObject*) _PyStack_AsTuple(PyObject **stack, diff --git a/Include/ceval.h b/Include/ceval.h --- a/Include/ceval.h +++ b/Include/ceval.h @@ -8,7 +8,7 @@ /* Interface to random parts in ceval.c */ PyAPI_FUNC(PyObject *) PyEval_CallObjectWithKeywords( - PyObject *, PyObject *, PyObject *); + PyObject *func, PyObject *args, PyObject *kwargs); /* Inline this */ #define PyEval_CallObject(func,arg) \ diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2194,7 +2194,7 @@ } PyObject * -PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw) +PyObject_Call(PyObject *func, PyObject *args, PyObject *kwargs) { ternaryfunc call; PyObject *result; @@ -2203,6 +2203,8 @@ because it may clear it (directly or indirectly) and so the caller loses its exception */ assert(!PyErr_Occurred()); + assert(PyTuple_Check(args)); + assert(kwargs == NULL || PyDict_Check(kwargs)); call = func->ob_type->tp_call; if (call == NULL) { @@ -2214,7 +2216,7 @@ if (Py_EnterRecursiveCall(" while calling a Python object")) return NULL; - result = (*call)(func, arg, kw); + result = (*call)(func, args, kwargs); Py_LeaveRecursiveCall(); diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4580,7 +4580,7 @@ The arg must be a tuple or NULL. The kw must be a dict or NULL. */ PyObject * -PyEval_CallObjectWithKeywords(PyObject *func, PyObject *arg, PyObject *kw) +PyEval_CallObjectWithKeywords(PyObject *func, PyObject *args, PyObject *kwargs) { PyObject *result; @@ -4591,32 +4591,33 @@ assert(!PyErr_Occurred()); #endif - if (arg == NULL) { - if (kw == NULL) { + if (args == NULL) { + if (kwargs == NULL) { return _PyObject_FastCall(func, NULL, 0, 0); } - arg = PyTuple_New(0); - if (arg == NULL) + args = PyTuple_New(0); + if (args == NULL) return NULL; } - else if (!PyTuple_Check(arg)) { + else if (!PyTuple_Check(args)) { PyErr_SetString(PyExc_TypeError, "argument list must be a tuple"); return NULL; } - else - Py_INCREF(arg); - - if (kw != NULL && !PyDict_Check(kw)) { + else { + Py_INCREF(args); + } + + if (kwargs != NULL && !PyDict_Check(kwargs)) { PyErr_SetString(PyExc_TypeError, "keyword list must be a dictionary"); - Py_DECREF(arg); + Py_DECREF(args); return NULL; } - result = PyObject_Call(func, arg, kw); - Py_DECREF(arg); + result = PyObject_Call(func, args, kwargs); + Py_DECREF(args); return result; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 11:32:22 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 15:32:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Avoid_call=5Ffunction=5Fta?= =?utf-8?q?il=28=29_for_empty_format_str?= Message-ID: <20160819153221.31696.1424.F392E3E7@psf.io> https://hg.python.org/cpython/rev/5cf9524f2923 changeset: 102762:5cf9524f2923 user: Victor Stinner date: Fri Aug 19 17:04:54 2016 +0200 summary: Avoid call_function_tail() for empty format str Issue #27128, PyObject_CallFunction(), _PyObject_FastCall() and callmethod(): if the format string of parameters is empty, avoid the creation of an empty tuple: call _PyObject_FastCall() without parameters. files: Objects/abstract.c | 39 ++++++++++++++++----------------- 1 files changed, 19 insertions(+), 20 deletions(-) diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2324,14 +2324,13 @@ return null_error(); } - if (format && *format) { - va_start(va, format); - args = Py_VaBuildValue(format, va); - va_end(va); + if (!format || !*format) { + return _PyObject_FastCall(callable, NULL, 0, NULL); } - else { - args = PyTuple_New(0); - } + + va_start(va, format); + args = Py_VaBuildValue(format, va); + va_end(va); if (args == NULL) { return NULL; } @@ -2351,14 +2350,13 @@ return null_error(); } - if (format && *format) { - va_start(va, format); - args = _Py_VaBuildValue_SizeT(format, va); - va_end(va); + if (!format || !*format) { + return _PyObject_FastCall(callable, NULL, 0, NULL); } - else { - args = PyTuple_New(0); - } + + va_start(va, format); + args = _Py_VaBuildValue_SizeT(format, va); + va_end(va); if (args == NULL) { return NULL; } @@ -2380,14 +2378,15 @@ return NULL; } - if (format && *format) { - if (is_size_t) - args = _Py_VaBuildValue_SizeT(format, va); - else - args = Py_VaBuildValue(format, va); + if (!format || !*format) { + return _PyObject_FastCall(func, NULL, 0, NULL); + } + + if (is_size_t) { + args = _Py_VaBuildValue_SizeT(format, va); } else { - args = PyTuple_New(0); + args = Py_VaBuildValue(format, va); } if (args == NULL) { return NULL; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:00:02 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:00:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_contains_and_rich_compare_?= =?utf-8?q?slots_use_fast_call?= Message-ID: <20160819160000.6768.47914.129FFCFD@psf.io> https://hg.python.org/cpython/rev/2da6dc1c30d8 changeset: 102764:2da6dc1c30d8 user: Victor Stinner date: Fri Aug 19 17:48:51 2016 +0200 summary: contains and rich compare slots use fast call Issue #27128. Modify slot_sq_contains() and slot_tp_richcompare() to use fast call to avoid a temporary tuple to pass a single positional parameter. files: Objects/typeobject.c | 20 ++++---------------- 1 files changed, 4 insertions(+), 16 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -5853,7 +5853,7 @@ static int slot_sq_contains(PyObject *self, PyObject *value) { - PyObject *func, *res, *args; + PyObject *func, *res; int result = -1; _Py_IDENTIFIER(__contains__); @@ -5866,13 +5866,7 @@ return -1; } if (func != NULL) { - args = PyTuple_Pack(1, value); - if (args == NULL) - res = NULL; - else { - res = PyObject_Call(func, args, NULL); - Py_DECREF(args); - } + res = _PyObject_FastCall(func, &value, 1, NULL); Py_DECREF(func); if (res != NULL) { result = PyObject_IsTrue(res); @@ -6225,20 +6219,14 @@ static PyObject * slot_tp_richcompare(PyObject *self, PyObject *other, int op) { - PyObject *func, *args, *res; + PyObject *func, *res; func = lookup_method(self, &name_op[op]); if (func == NULL) { PyErr_Clear(); Py_RETURN_NOTIMPLEMENTED; } - args = PyTuple_Pack(1, other); - if (args == NULL) - res = NULL; - else { - res = PyObject_Call(func, args, NULL); - Py_DECREF(args); - } + res = _PyObject_FastCall(func, &other, 1, NULL); Py_DECREF(func); return res; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:00:05 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:00:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_a_refleak_?= =?utf-8?q?in_call=5Fmethod=28=29?= Message-ID: <20160819160001.43154.89616.D0E0EA07@psf.io> https://hg.python.org/cpython/rev/2d4d40da2aba changeset: 102765:2d4d40da2aba branch: 3.5 parent: 102754:51368991e858 user: Victor Stinner date: Fri Aug 19 17:51:49 2016 +0200 summary: Fix a refleak in call_method() Issue #27128. Fix a reference leak if creating the tuple to pass positional parameters fails. files: Objects/typeobject.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -1432,8 +1432,10 @@ va_end(va); - if (args == NULL) + if (args == NULL) { + Py_DECREF(func); return NULL; + } assert(PyTuple_Check(args)); retval = PyObject_Call(func, args, NULL); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:00:09 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:00:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_a_refleak_?= =?utf-8?b?aW4gY2FsbF9tYXliZSgp?= Message-ID: <20160819160007.32040.42008.9CC6EF32@psf.io> https://hg.python.org/cpython/rev/df4efc23ab18 changeset: 102769:df4efc23ab18 branch: 3.5 parent: 102765:2d4d40da2aba user: Victor Stinner date: Fri Aug 19 17:58:12 2016 +0200 summary: Fix a refleak in call_maybe() Issue #27128. Fix a reference leak if creating the tuple to pass positional parameters fails. files: Objects/typeobject.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -1470,8 +1470,10 @@ va_end(va); - if (args == NULL) + if (args == NULL) { + Py_DECREF(func); return NULL; + } assert(PyTuple_Check(args)); retval = PyObject_Call(func, args, NULL); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:00:09 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:00:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_regrtest=3A_replace_=22Res?= =?utf-8?q?ult=3A=22_with_=22Tests_result=3A=22?= Message-ID: <20160819160007.32079.30862.5E6D5E5C@psf.io> https://hg.python.org/cpython/rev/2e6ca74ebc05 changeset: 102768:2e6ca74ebc05 parent: 102766:beb261f996b7 user: Victor Stinner date: Fri Aug 19 17:54:25 2016 +0200 summary: regrtest: replace "Result:" with "Tests result:" files: Lib/test/libregrtest/main.py | 2 +- Lib/test/test_regrtest.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -437,7 +437,7 @@ result = "INTERRUPTED" else: result = "SUCCESS" - print("Result: %s" % result) + print("Tests result: %s" % result) if self.ns.runleaks: os.system("leaks %d" % os.getpid()) diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -407,7 +407,7 @@ result = 'INTERRUPTED' else: result = 'SUCCESS' - self.check_line(output, 'Result: %s' % result) + self.check_line(output, 'Tests result: %s' % result) def parse_random_seed(self, output): match = self.regex_search(r'Using random seed ([0-9]+)', output) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:00:09 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:00:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChmaXggcmVmbGVhayBpbiBjYWxsX21heWJlKCkp?= Message-ID: <20160819160007.55037.51939.25273454@psf.io> https://hg.python.org/cpython/rev/14fa255ea5ff changeset: 102770:14fa255ea5ff parent: 102768:2e6ca74ebc05 parent: 102769:df4efc23ab18 user: Victor Stinner date: Fri Aug 19 17:58:30 2016 +0200 summary: Merge 3.5 (fix refleak in call_maybe()) files: Objects/typeobject.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -1480,8 +1480,10 @@ va_end(va); - if (args == NULL) + if (args == NULL) { + Py_DECREF(func); return NULL; + } assert(PyTuple_Check(args)); retval = PyObject_Call(func, args, NULL); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:00:09 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:00:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_a_refleak_?= =?utf-8?q?in_call=5Fmethod=28=29?= Message-ID: <20160819160006.85127.25940.F4821D75@psf.io> https://hg.python.org/cpython/rev/5b1ed48aedef changeset: 102767:5b1ed48aedef branch: 2.7 parent: 102712:11a9bca71528 user: Victor Stinner date: Fri Aug 19 17:52:53 2016 +0200 summary: Fix a refleak in call_method() Issue #27128. Fix a reference leak if creating the tuple to pass positional parameters fails. files: Objects/typeobject.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -1274,8 +1274,10 @@ va_end(va); - if (args == NULL) + if (args == NULL) { + Py_DECREF(func); return NULL; + } assert(PyTuple_Check(args)); retval = PyObject_Call(func, args, NULL); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:00:09 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:00:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_a_refleak_?= =?utf-8?b?aW4gY2FsbF9tYXliZSgp?= Message-ID: <20160819160007.43291.53757.3C4AB47B@psf.io> https://hg.python.org/cpython/rev/7669fb39a9ce changeset: 102771:7669fb39a9ce branch: 2.7 parent: 102767:5b1ed48aedef user: Victor Stinner date: Fri Aug 19 17:58:54 2016 +0200 summary: Fix a refleak in call_maybe() Issue #27128. Fix a reference leak if creating the tuple to pass positional parameters fails. files: Objects/typeobject.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -1314,8 +1314,10 @@ va_end(va); - if (args == NULL) + if (args == NULL) { + Py_DECREF(func); return NULL; + } assert(PyTuple_Check(args)); retval = PyObject_Call(func, args, NULL); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:00:09 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:00:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_=28fix_refleak_in_call=5Fmethod=29?= Message-ID: <20160819160006.55037.61314.0AACC204@psf.io> https://hg.python.org/cpython/rev/beb261f996b7 changeset: 102766:beb261f996b7 parent: 102764:2da6dc1c30d8 parent: 102765:2d4d40da2aba user: Victor Stinner date: Fri Aug 19 17:52:08 2016 +0200 summary: Merge 3.5 (fix refleak in call_method) files: Objects/typeobject.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -1442,8 +1442,10 @@ va_end(va); - if (args == NULL) + if (args == NULL) { + Py_DECREF(func); return NULL; + } assert(PyTuple_Check(args)); retval = PyObject_Call(func, args, NULL); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:53:53 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:53:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_call=5Fmethod=28=29_and_ca?= =?utf-8?q?ll=5Fmaybe=28=29_now_use_fast_call?= Message-ID: <20160819165353.31933.8743.456458BC@psf.io> https://hg.python.org/cpython/rev/8e085070ab28 changeset: 102773:8e085070ab28 user: Victor Stinner date: Fri Aug 19 18:05:37 2016 +0200 summary: call_method() and call_maybe() now use fast call Issue #27128. The call_method() and call_maybe() functions of typeobject.c now use fast call for empty format string to avoid the creation of a temporary empty tuple. files: Objects/typeobject.c | 54 +++++++++++++++++-------------- 1 files changed, 30 insertions(+), 24 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -1424,7 +1424,7 @@ call_method(PyObject *o, _Py_Identifier *nameid, const char *format, ...) { va_list va; - PyObject *args, *func = 0, *retval; + PyObject *func = NULL, *retval; func = lookup_maybe(o, nameid); if (func == NULL) { @@ -1434,22 +1434,25 @@ } if (format && *format) { + PyObject *args; + va_start(va, format); args = Py_VaBuildValue(format, va); va_end(va); + + if (args == NULL) { + Py_DECREF(func); + return NULL; + } + assert(PyTuple_Check(args)); + + retval = PyObject_Call(func, args, NULL); + Py_DECREF(args); } else { - args = PyTuple_New(0); - } - if (args == NULL) { - Py_DECREF(func); - return NULL; - } - - assert(PyTuple_Check(args)); - retval = PyObject_Call(func, args, NULL); - - Py_DECREF(args); + retval = _PyObject_FastCall(func, NULL, 0, NULL); + } + Py_DECREF(func); return retval; @@ -1461,7 +1464,7 @@ call_maybe(PyObject *o, _Py_Identifier *nameid, const char *format, ...) { va_list va; - PyObject *args, *func = 0, *retval; + PyObject *func = NULL, *retval; func = lookup_maybe(o, nameid); if (func == NULL) { @@ -1471,22 +1474,25 @@ } if (format && *format) { + PyObject *args; + va_start(va, format); args = Py_VaBuildValue(format, va); va_end(va); + + if (args == NULL) { + Py_DECREF(func); + return NULL; + } + assert(PyTuple_Check(args)); + + retval = PyObject_Call(func, args, NULL); + Py_DECREF(args); } else { - args = PyTuple_New(0); - } - if (args == NULL) { - Py_DECREF(func); - return NULL; - } - - assert(PyTuple_Check(args)); - retval = PyObject_Call(func, args, NULL); - - Py_DECREF(args); + retval = _PyObject_FastCall(func, NULL, 0, NULL); + } + Py_DECREF(func); return retval; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:53:53 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:53:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Cleanup_call=5Fmethod=28?= =?utf-8?b?KSBhbmQgY2FsbF9tYXliZSgp?= Message-ID: <20160819165353.31678.72163.6760D8D9@psf.io> https://hg.python.org/cpython/rev/73b00fb1dc9d changeset: 102772:73b00fb1dc9d parent: 102770:14fa255ea5ff user: Victor Stinner date: Fri Aug 19 18:01:41 2016 +0200 summary: Cleanup call_method() and call_maybe() Issue #27128. Move va_start/va_end around Py_VaBuildValue(). files: Objects/typeobject.c | 26 ++++++++++++-------------- 1 files changed, 12 insertions(+), 14 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -1425,23 +1425,22 @@ { va_list va; PyObject *args, *func = 0, *retval; - va_start(va, format); func = lookup_maybe(o, nameid); if (func == NULL) { - va_end(va); if (!PyErr_Occurred()) PyErr_SetObject(PyExc_AttributeError, nameid->object); return NULL; } - if (format && *format) + if (format && *format) { + va_start(va, format); args = Py_VaBuildValue(format, va); - else + va_end(va); + } + else { args = PyTuple_New(0); - - va_end(va); - + } if (args == NULL) { Py_DECREF(func); return NULL; @@ -1463,23 +1462,22 @@ { va_list va; PyObject *args, *func = 0, *retval; - va_start(va, format); func = lookup_maybe(o, nameid); if (func == NULL) { - va_end(va); if (!PyErr_Occurred()) Py_RETURN_NOTIMPLEMENTED; return NULL; } - if (format && *format) + if (format && *format) { + va_start(va, format); args = Py_VaBuildValue(format, va); - else + va_end(va); + } + else { args = PyTuple_New(0); - - va_end(va); - + } if (args == NULL) { Py_DECREF(func); return NULL; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:53:54 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:53:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327128=3A_slot=5Fs?= =?utf-8?q?q=5Fitem=28=29_uses_fast_call?= Message-ID: <20160819165354.128048.71349.EF8BC53F@psf.io> https://hg.python.org/cpython/rev/6eb586b85fa1 changeset: 102775:6eb586b85fa1 user: Victor Stinner date: Fri Aug 19 18:19:42 2016 +0200 summary: Issue #27128: slot_sq_item() uses fast call slot_sq_item() now calls _PyObject_FastCall() to avoid the creation of a temporary tuple of 1 item to pass the 'item' argument to the slot function. files: Objects/typeobject.c | 13 +++---------- 1 files changed, 3 insertions(+), 10 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -5808,7 +5808,7 @@ static PyObject * slot_sq_item(PyObject *self, Py_ssize_t i) { - PyObject *func, *ival = NULL, *args, *retval = NULL; + PyObject *func, *ival = NULL, *retval = NULL; descrgetfunc f; func = _PyType_LookupId(Py_TYPE(self), &PyId___getitem__); @@ -5834,20 +5834,13 @@ goto error; } - args = PyTuple_New(1); - if (args == NULL) { - goto error; - } - - PyTuple_SET_ITEM(args, 0, ival); - retval = PyObject_Call(func, args, NULL); + retval = _PyObject_FastCall(func, &ival, 1, NULL); Py_DECREF(func); - Py_DECREF(args); + Py_DECREF(ival); return retval; error: Py_DECREF(func); - Py_XDECREF(ival); return NULL; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:53:54 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:53:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327128=3A_Cleanup_?= =?utf-8?b?c2xvdF9zcV9pdGVtKCk=?= Message-ID: <20160819165353.43130.69552.F7DDAE08@psf.io> https://hg.python.org/cpython/rev/2d2bc1906b5b changeset: 102774:2d2bc1906b5b user: Victor Stinner date: Fri Aug 19 18:17:37 2016 +0200 summary: Issue #27128: Cleanup slot_sq_item() * Invert condition of test to avoid levels of indentation * Remove useless Py_XDECREF(args) in the error block * Replace Py_XDECREF(func) with Py_DECREF(func) in the error block: func cannot be NULL when reaching the error block files: Objects/typeobject.c | 60 ++++++++++++++++++------------- 1 files changed, 34 insertions(+), 26 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -5808,38 +5808,46 @@ static PyObject * slot_sq_item(PyObject *self, Py_ssize_t i) { - PyObject *func, *args = NULL, *ival = NULL, *retval = NULL; + PyObject *func, *ival = NULL, *args, *retval = NULL; descrgetfunc f; func = _PyType_LookupId(Py_TYPE(self), &PyId___getitem__); - if (func != NULL) { - if ((f = Py_TYPE(func)->tp_descr_get) == NULL) - Py_INCREF(func); - else { - func = f(func, self, (PyObject *)(Py_TYPE(self))); - if (func == NULL) { - return NULL; - } - } - ival = PyLong_FromSsize_t(i); - if (ival != NULL) { - args = PyTuple_New(1); - if (args != NULL) { - PyTuple_SET_ITEM(args, 0, ival); - retval = PyObject_Call(func, args, NULL); - Py_DECREF(args); - Py_DECREF(func); - return retval; - } - } - } - else { + if (func == NULL) { PyObject *getitem_str = _PyUnicode_FromId(&PyId___getitem__); PyErr_SetObject(PyExc_AttributeError, getitem_str); - } - Py_XDECREF(args); + return NULL; + } + + f = Py_TYPE(func)->tp_descr_get; + if (f == NULL) { + Py_INCREF(func); + } + else { + func = f(func, self, (PyObject *)(Py_TYPE(self))); + if (func == NULL) { + return NULL; + } + } + + ival = PyLong_FromSsize_t(i); + if (ival == NULL) { + goto error; + } + + args = PyTuple_New(1); + if (args == NULL) { + goto error; + } + + PyTuple_SET_ITEM(args, 0, ival); + retval = PyObject_Call(func, args, NULL); + Py_DECREF(func); + Py_DECREF(args); + return retval; + +error: + Py_DECREF(func); Py_XDECREF(ival); - Py_XDECREF(func); return NULL; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:53:54 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:53:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327128=3A_Cleanup_?= =?utf-8?b?c2xvdF9uYl9ib29sKCk=?= Message-ID: <20160819165354.43244.75680.90C0F984@psf.io> https://hg.python.org/cpython/rev/605a42a50496 changeset: 102776:605a42a50496 user: Victor Stinner date: Fri Aug 19 18:26:05 2016 +0200 summary: Issue #27128: Cleanup slot_nb_bool() Use an error label to reduce the level of indentation. files: Objects/typeobject.c | 68 +++++++++++++++++++------------ 1 files changed, 42 insertions(+), 26 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -5946,44 +5946,60 @@ static int slot_nb_bool(PyObject *self) { - PyObject *func, *args; - int result = -1; + PyObject *func, *args, *value; + int result; int using_len = 0; _Py_IDENTIFIER(__bool__); func = lookup_maybe(self, &PyId___bool__); if (func == NULL) { - if (PyErr_Occurred()) + if (PyErr_Occurred()) { return -1; + } + func = lookup_maybe(self, &PyId___len__); - if (func == NULL) - return PyErr_Occurred() ? -1 : 1; + if (func == NULL) { + if (PyErr_Occurred()) { + return -1; + } + return 1; + } using_len = 1; } + args = PyTuple_New(0); - if (args != NULL) { - PyObject *temp = PyObject_Call(func, args, NULL); - Py_DECREF(args); - if (temp != NULL) { - if (using_len) { - /* enforced by slot_nb_len */ - result = PyObject_IsTrue(temp); - } - else if (PyBool_Check(temp)) { - result = PyObject_IsTrue(temp); - } - else { - PyErr_Format(PyExc_TypeError, - "__bool__ should return " - "bool, returned %s", - Py_TYPE(temp)->tp_name); - result = -1; - } - Py_DECREF(temp); - } - } + if (args == NULL) { + goto error; + } + + value = PyObject_Call(func, args, NULL); + Py_DECREF(args); + if (value == NULL) { + goto error; + } + + if (using_len) { + /* bool type enforced by slot_nb_len */ + result = PyObject_IsTrue(value); + } + else if (PyBool_Check(value)) { + result = PyObject_IsTrue(value); + } + else { + PyErr_Format(PyExc_TypeError, + "__bool__ should return " + "bool, returned %s", + Py_TYPE(value)->tp_name); + result = -1; + } + + Py_DECREF(value); Py_DECREF(func); return result; + +error: + Py_DECREF(func); + return -1; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:53:54 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:53:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbjogc2xvdF9uYl9ib29sKCkgbm93?= =?utf-8?q?_uses_fast_call?= Message-ID: <20160819165354.127856.28088.414D5F4F@psf.io> https://hg.python.org/cpython/rev/6a21b6599692 changeset: 102777:6a21b6599692 user: Victor Stinner date: Fri Aug 19 18:28:25 2016 +0200 summary: slot_nb_bool() now uses fast call Issue #27128: slot_nb_bool() now calls _PyObject_FastCall() to avoid a temporary empty tuple to call the slot function. files: Objects/typeobject.c | 10 ++-------- 1 files changed, 2 insertions(+), 8 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -5946,7 +5946,7 @@ static int slot_nb_bool(PyObject *self) { - PyObject *func, *args, *value; + PyObject *func, *value; int result; int using_len = 0; _Py_IDENTIFIER(__bool__); @@ -5967,13 +5967,7 @@ using_len = 1; } - args = PyTuple_New(0); - if (args == NULL) { - goto error; - } - - value = PyObject_Call(func, args, NULL); - Py_DECREF(args); + value = _PyObject_FastCall(func, NULL, 0, NULL); if (value == NULL) { goto error; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:53:54 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:53:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbjogc2xvdF90cF9pdGVyKCkgbm93?= =?utf-8?q?_uses_fast_call?= Message-ID: <20160819165354.20267.90645.C3E140CE@psf.io> https://hg.python.org/cpython/rev/45d2b5c12b19 changeset: 102778:45d2b5c12b19 user: Victor Stinner date: Fri Aug 19 18:41:02 2016 +0200 summary: slot_tp_iter() now uses fast call Issue #27128: slot_tp_iter() now calls _PyObject_FastCall() to avoid a temporary empty tuple. files: Objects/typeobject.c | 9 +++------ 1 files changed, 3 insertions(+), 6 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -6264,16 +6264,13 @@ Py_TYPE(self)->tp_name); return NULL; } + if (func != NULL) { - PyObject *args; - args = res = PyTuple_New(0); - if (args != NULL) { - res = PyObject_Call(func, args, NULL); - Py_DECREF(args); - } + res = _PyObject_FastCall(func, NULL, 0, NULL); Py_DECREF(func); return res; } + PyErr_Clear(); func = lookup_method(self, &PyId___getitem__); if (func == NULL) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:53:54 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:53:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_keyobject=5Frichcompare=28?= =?utf-8?q?=29_now_uses_fast_call?= Message-ID: <20160819165354.24254.60548.958B1B77@psf.io> https://hg.python.org/cpython/rev/71c22e592a9b changeset: 102780:71c22e592a9b user: Victor Stinner date: Fri Aug 19 18:52:35 2016 +0200 summary: keyobject_richcompare() now uses fast call Issue #27128: keyobject_richcompare() now calls _PyObject_FastCall() using a small stack allocated on the C stack to avoid a temporary tuple. files: Modules/_functoolsmodule.c | 18 +++++++----------- 1 files changed, 7 insertions(+), 11 deletions(-) diff --git a/Modules/_functoolsmodule.c b/Modules/_functoolsmodule.c --- a/Modules/_functoolsmodule.c +++ b/Modules/_functoolsmodule.c @@ -461,12 +461,12 @@ keyobject_richcompare(PyObject *ko, PyObject *other, int op) { PyObject *res; - PyObject *args; PyObject *x; PyObject *y; PyObject *compare; PyObject *answer; static PyObject *zero; + PyObject* stack[2]; if (zero == NULL) { zero = PyLong_FromLong(0); @@ -490,17 +490,13 @@ /* Call the user's comparison function and translate the 3-way * result into true or false (or error). */ - args = PyTuple_New(2); - if (args == NULL) + stack[0] = x; + stack[1] = y; + res = _PyObject_FastCall(compare, stack, 2, NULL); + if (res == NULL) { return NULL; - Py_INCREF(x); - Py_INCREF(y); - PyTuple_SET_ITEM(args, 0, x); - PyTuple_SET_ITEM(args, 1, y); - res = PyObject_Call(compare, args, NULL); - Py_DECREF(args); - if (res == NULL) - return NULL; + } + answer = PyObject_RichCompare(res, zero, op); Py_DECREF(res); return answer; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 12:53:54 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 16:53:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_calliter=5Fiternext=28=29_?= =?utf-8?q?now_uses_fast_call?= Message-ID: <20160819165354.20158.18077.1392D8EB@psf.io> https://hg.python.org/cpython/rev/124d5d0ef81f changeset: 102779:124d5d0ef81f user: Victor Stinner date: Fri Aug 19 18:47:10 2016 +0200 summary: calliter_iternext() now uses fast call Issue #27128: calliter_iternext() now calls _PyObject_FastCall() to avoid a temporary empty tuple. Cleanup also the code to reduce the indentation level. files: Objects/iterobject.c | 40 ++++++++++++++++--------------- 1 files changed, 21 insertions(+), 19 deletions(-) diff --git a/Objects/iterobject.c b/Objects/iterobject.c --- a/Objects/iterobject.c +++ b/Objects/iterobject.c @@ -208,30 +208,32 @@ static PyObject * calliter_iternext(calliterobject *it) { - if (it->it_callable != NULL) { - PyObject *args = PyTuple_New(0); - PyObject *result; - if (args == NULL) - return NULL; - result = PyObject_Call(it->it_callable, args, NULL); - Py_DECREF(args); - if (result != NULL) { - int ok; - ok = PyObject_RichCompareBool(it->it_sentinel, result, Py_EQ); - if (ok == 0) - return result; /* Common case, fast path */ - Py_DECREF(result); - if (ok > 0) { - Py_CLEAR(it->it_callable); - Py_CLEAR(it->it_sentinel); - } + PyObject *result; + + if (it->it_callable == NULL) { + return NULL; + } + + result = _PyObject_FastCall(it->it_callable, NULL, 0, NULL); + if (result != NULL) { + int ok; + + ok = PyObject_RichCompareBool(it->it_sentinel, result, Py_EQ); + if (ok == 0) { + return result; /* Common case, fast path */ } - else if (PyErr_ExceptionMatches(PyExc_StopIteration)) { - PyErr_Clear(); + + Py_DECREF(result); + if (ok > 0) { Py_CLEAR(it->it_callable); Py_CLEAR(it->it_sentinel); } } + else if (PyErr_ExceptionMatches(PyExc_StopIteration)) { + PyErr_Clear(); + Py_CLEAR(it->it_callable); + Py_CLEAR(it->it_sentinel); + } return NULL; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 13:01:22 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 17:01:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327128=3A_=5Fpickl?= =?utf-8?q?e_uses_fast_call?= Message-ID: <20160819170121.92054.14082.C0C0078C@psf.io> https://hg.python.org/cpython/rev/3ab32f7add6e changeset: 102781:3ab32f7add6e user: Victor Stinner date: Fri Aug 19 18:59:15 2016 +0200 summary: Issue #27128: _pickle uses fast call Use _PyObject_FastCall() to avoid the creation of temporary tuple. files: Modules/_pickle.c | 19 ++++--------------- 1 files changed, 4 insertions(+), 15 deletions(-) diff --git a/Modules/_pickle.c b/Modules/_pickle.c --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -345,7 +345,6 @@ _Pickle_FastCall(PyObject *func, PyObject *obj) { PyObject *result; - PyObject *arg_tuple = PyTuple_New(1); /* Note: this function used to reuse the argument tuple. This used to give a slight performance boost with older pickle implementations where many @@ -358,13 +357,8 @@ significantly reduced the number of function calls we do. Thus, the benefits became marginal at best. */ - if (arg_tuple == NULL) { - Py_DECREF(obj); - return NULL; - } - PyTuple_SET_ITEM(arg_tuple, 0, obj); - result = PyObject_Call(func, arg_tuple, NULL); - Py_CLEAR(arg_tuple); + result = _PyObject_FastCall(func, &obj, 1, NULL); + Py_DECREF(obj); return result; } @@ -1157,9 +1151,7 @@ return -1; if (n == READ_WHOLE_LINE) { - PyObject *empty_tuple = PyTuple_New(0); - data = PyObject_Call(self->readline, empty_tuple, NULL); - Py_DECREF(empty_tuple); + data = _PyObject_FastCall(self->readline, NULL, 0, NULL); } else { PyObject *len; @@ -3956,10 +3948,7 @@ /* Check for a __reduce__ method. */ reduce_func = _PyObject_GetAttrId(obj, &PyId___reduce__); if (reduce_func != NULL) { - PyObject *empty_tuple = PyTuple_New(0); - reduce_value = PyObject_Call(reduce_func, empty_tuple, - NULL); - Py_DECREF(empty_tuple); + reduce_value = _PyObject_FastCall(reduce_func, NULL, 0, NULL); } else { PyErr_Format(st->PicklingError, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 19:50:39 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 23:50:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_import=5Fname=28=29_now_us?= =?utf-8?q?es_fast_call?= Message-ID: <20160819235039.90288.31879.981393F8@psf.io> https://hg.python.org/cpython/rev/0da1ce362d15 changeset: 102783:0da1ce362d15 user: Victor Stinner date: Sat Aug 20 00:44:42 2016 +0200 summary: import_name() now uses fast call Issue #27128: import_name() now calls _PyObject_FastCall() to avoid the creation of a temporary tuple. files: Python/ceval.c | 22 +++++++++------------- 1 files changed, 9 insertions(+), 13 deletions(-) diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -5247,7 +5247,8 @@ import_name(PyFrameObject *f, PyObject *name, PyObject *fromlist, PyObject *level) { _Py_IDENTIFIER(__import__); - PyObject *import_func, *args, *res; + PyObject *import_func, *res; + PyObject* stack[5]; import_func = _PyDict_GetItemId(f->f_builtins, &PyId___import__); if (import_func == NULL) { @@ -5271,18 +5272,13 @@ } Py_INCREF(import_func); - args = PyTuple_Pack(5, - name, - f->f_globals, - f->f_locals == NULL ? Py_None : f->f_locals, - fromlist, - level); - if (args == NULL) { - Py_DECREF(import_func); - return NULL; - } - res = PyEval_CallObject(import_func, args); - Py_DECREF(args); + + stack[0] = name; + stack[1] = f->f_globals; + stack[2] = f->f_locals == NULL ? Py_None : f->f_locals; + stack[3] = fromlist; + stack[4] = level; + res = _PyObject_FastCall(import_func, stack, 5, NULL); Py_DECREF(import_func); return res; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 19:50:38 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 23:50:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_PyFile=5FWriteObject=28=29?= =?utf-8?q?_now_uses_fast_call?= Message-ID: <20160819235038.7307.8346.72327AF9@psf.io> https://hg.python.org/cpython/rev/c2af917bde71 changeset: 102782:c2af917bde71 user: Victor Stinner date: Sat Aug 20 00:44:04 2016 +0200 summary: PyFile_WriteObject() now uses fast call Issue #27128: PyFile_WriteObject() now calls _PyObject_FastCall() to avoid the creation of a temporary tuple. files: Objects/fileobject.c | 11 ++--------- 1 files changed, 2 insertions(+), 9 deletions(-) diff --git a/Objects/fileobject.c b/Objects/fileobject.c --- a/Objects/fileobject.c +++ b/Objects/fileobject.c @@ -127,7 +127,7 @@ int PyFile_WriteObject(PyObject *v, PyObject *f, int flags) { - PyObject *writer, *value, *args, *result; + PyObject *writer, *value, *result; _Py_IDENTIFIER(write); if (f == NULL) { @@ -146,14 +146,7 @@ Py_DECREF(writer); return -1; } - args = PyTuple_Pack(1, value); - if (args == NULL) { - Py_DECREF(value); - Py_DECREF(writer); - return -1; - } - result = PyEval_CallObject(writer, args); - Py_DECREF(args); + result = _PyObject_FastCall(writer, &value, 1, NULL); Py_DECREF(value); Py_DECREF(writer); if (result == NULL) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 19:50:39 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 23:50:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_call=5Ftrampoline=28=29_no?= =?utf-8?q?w_uses_fast_call?= Message-ID: <20160819235039.90288.52236.ECBCFBAD@psf.io> https://hg.python.org/cpython/rev/154f78d387f9 changeset: 102785:154f78d387f9 user: Victor Stinner date: Sat Aug 20 01:22:57 2016 +0200 summary: call_trampoline() now uses fast call Issue #27128. files: Python/sysmodule.c | 29 ++++++++++------------------- 1 files changed, 10 insertions(+), 19 deletions(-) diff --git a/Python/sysmodule.c b/Python/sysmodule.c --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -368,34 +368,25 @@ call_trampoline(PyObject* callback, PyFrameObject *frame, int what, PyObject *arg) { - PyObject *args; - PyObject *whatstr; PyObject *result; + PyObject *stack[3]; - args = PyTuple_New(3); - if (args == NULL) + if (PyFrame_FastToLocalsWithError(frame) < 0) { return NULL; - if (PyFrame_FastToLocalsWithError(frame) < 0) - return NULL; + } - Py_INCREF(frame); - whatstr = whatstrings[what]; - Py_INCREF(whatstr); - if (arg == NULL) - arg = Py_None; - Py_INCREF(arg); - PyTuple_SET_ITEM(args, 0, (PyObject *)frame); - PyTuple_SET_ITEM(args, 1, whatstr); - PyTuple_SET_ITEM(args, 2, arg); + stack[0] = (PyObject *)frame; + stack[1] = whatstrings[what]; + stack[2] = (arg != NULL) ? arg : Py_None; /* call the Python-level function */ - result = PyEval_CallObject(callback, args); + result = _PyObject_FastCall(callback, stack, 3, NULL); + PyFrame_LocalsToFast(frame, 1); - if (result == NULL) + if (result == NULL) { PyTraceBack_Here(frame); + } - /* cleanup */ - Py_DECREF(args); return result; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 19:50:39 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 23:50:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_sys=5Fpyfile=5Fwrite=5Funi?= =?utf-8?q?code=28=29_now_uses_fast_call?= Message-ID: <20160819235039.90468.65046.BBBD7163@psf.io> https://hg.python.org/cpython/rev/351b987d6d1c changeset: 102786:351b987d6d1c user: Victor Stinner date: Sat Aug 20 01:24:22 2016 +0200 summary: sys_pyfile_write_unicode() now uses fast call Issue #27128. files: Python/sysmodule.c | 9 ++------- 1 files changed, 2 insertions(+), 7 deletions(-) diff --git a/Python/sysmodule.c b/Python/sysmodule.c --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -2112,7 +2112,7 @@ static int sys_pyfile_write_unicode(PyObject *unicode, PyObject *file) { - PyObject *writer = NULL, *args = NULL, *result = NULL; + PyObject *writer = NULL, *result = NULL; int err; if (file == NULL) @@ -2122,11 +2122,7 @@ if (writer == NULL) goto error; - args = PyTuple_Pack(1, unicode); - if (args == NULL) - goto error; - - result = PyEval_CallObject(writer, args); + result = _PyObject_FastCall(writer, &unicode, 1, NULL); if (result == NULL) { goto error; } else { @@ -2138,7 +2134,6 @@ err = -1; finally: Py_XDECREF(writer); - Py_XDECREF(args); Py_XDECREF(result); return err; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 19:50:39 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 23:50:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_PyErr=5FPrintEx=28=29_now_?= =?utf-8?q?uses_fast_call?= Message-ID: <20160819235039.63280.64209.F3C57FBF@psf.io> https://hg.python.org/cpython/rev/e5b24f595235 changeset: 102784:e5b24f595235 user: Victor Stinner date: Sat Aug 20 00:57:43 2016 +0200 summary: PyErr_PrintEx() now uses fast call Issue #27128. files: Python/pythonrun.c | 10 +++++++--- 1 files changed, 7 insertions(+), 3 deletions(-) diff --git a/Python/pythonrun.c b/Python/pythonrun.c --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -630,8 +630,13 @@ } hook = _PySys_GetObjectId(&PyId_excepthook); if (hook) { - PyObject *args = PyTuple_Pack(3, exception, v, tb); - PyObject *result = PyEval_CallObject(hook, args); + PyObject* stack[3]; + PyObject *result; + + stack[0] = exception; + stack[1] = v; + stack[2] = tb; + result = _PyObject_FastCall(hook, stack, 3, NULL); if (result == NULL) { PyObject *exception2, *v2, *tb2; if (PyErr_ExceptionMatches(PyExc_SystemExit)) { @@ -660,7 +665,6 @@ Py_XDECREF(tb2); } Py_XDECREF(result); - Py_XDECREF(args); } else { PySys_WriteStderr("sys.excepthook is missing\n"); PyErr_Display(exception, v, tb); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 19:50:44 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 23:50:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=5Felementtree=3A_deepcopy?= =?utf-8?q?=28=29_now_uses_fast_call?= Message-ID: <20160819235044.31843.62189.B03B664F@psf.io> https://hg.python.org/cpython/rev/abb93035ebb7 changeset: 102787:abb93035ebb7 user: Victor Stinner date: Sat Aug 20 01:34:44 2016 +0200 summary: _elementtree: deepcopy() now uses fast call Issue #27128. files: Modules/_elementtree.c | 12 ++++-------- 1 files changed, 4 insertions(+), 8 deletions(-) diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -819,9 +819,8 @@ deepcopy(PyObject *object, PyObject *memo) { /* do a deep copy of the given object */ - PyObject *args; - PyObject *result; elementtreestate *st; + PyObject *stack[2]; /* Fast paths */ if (object == Py_None || PyUnicode_CheckExact(object)) { @@ -857,12 +856,9 @@ return NULL; } - args = PyTuple_Pack(2, object, memo); - if (!args) - return NULL; - result = PyObject_CallObject(st->deepcopy_obj, args); - Py_DECREF(args); - return result; + stack[0] = object; + stack[1] = memo; + return _PyObject_FastCall(st->deepcopy_obj, stack, 2, NULL); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 19:50:45 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 19 Aug 2016 23:50:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_pattern=5Fsubx=28=29_now_u?= =?utf-8?q?ses_fast_call?= Message-ID: <20160819235045.63438.34424.8E0A6F7F@psf.io> https://hg.python.org/cpython/rev/2954d2aa4c90 changeset: 102788:2954d2aa4c90 user: Victor Stinner date: Sat Aug 20 01:38:00 2016 +0200 summary: pattern_subx() now uses fast call Issue #27128. files: Modules/_sre.c | 9 +-------- 1 files changed, 1 insertions(+), 8 deletions(-) diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -1056,7 +1056,6 @@ PyObject* joiner; PyObject* item; PyObject* filter; - PyObject* args; PyObject* match; void* ptr; Py_ssize_t status; @@ -1158,13 +1157,7 @@ match = pattern_new_match(self, &state, 1); if (!match) goto error; - args = PyTuple_Pack(1, match); - if (!args) { - Py_DECREF(match); - goto error; - } - item = PyObject_CallObject(filter, args); - Py_DECREF(args); + item = _PyObject_FastCall(filter, &match, 1, NULL); Py_DECREF(match); if (!item) goto error; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 20:38:45 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 20 Aug 2016 00:38:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327366=3A_Fix_init?= =?utf-8?b?X3N1YmNsYXNzKCk=?= Message-ID: <20160820003845.90418.76577.41710D71@psf.io> https://hg.python.org/cpython/rev/545bfa4c20eb changeset: 102789:545bfa4c20eb user: Victor Stinner date: Sat Aug 20 02:37:41 2016 +0200 summary: Issue #27366: Fix init_subclass() Handle PyTuple_New(0) failure. files: Objects/typeobject.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -7018,6 +7018,11 @@ return -1; tuple = PyTuple_New(0); + if (tuple == NULL) { + Py_DECREF(func); + return 0; + } + tmp = PyObject_Call(func, tuple, kwds); Py_DECREF(tuple); Py_DECREF(func); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 21:06:50 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 20 Aug 2016 01:06:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_reference_leak_in_tb?= =?utf-8?q?=5Fprintinternal=28=29?= Message-ID: <20160820010650.21975.26585.CFA9BE3A@psf.io> https://hg.python.org/cpython/rev/a7f3678f9509 changeset: 102790:a7f3678f9509 user: Victor Stinner date: Sat Aug 20 03:05:13 2016 +0200 summary: Fix reference leak in tb_printinternal() Issue #26823. files: Python/traceback.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Python/traceback.c b/Python/traceback.c --- a/Python/traceback.c +++ b/Python/traceback.c @@ -435,6 +435,7 @@ line = PyUnicode_FromFormat( " [Previous line repeated %d more times]\n", cnt-3); err = PyFile_WriteObject(line, f, Py_PRINT_RAW); + Py_DECREF(line); } last_file = tb->tb_frame->f_code->co_filename; last_line = tb->tb_lineno; @@ -456,6 +457,7 @@ line = PyUnicode_FromFormat( " [Previous line repeated %d more times]\n", cnt-3); err = PyFile_WriteObject(line, f, Py_PRINT_RAW); + Py_DECREF(line); } return err; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 22:13:30 2016 From: python-checkins at python.org (ned.deily) Date: Sat, 20 Aug 2016 02:13:30 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NzEz?= =?utf-8?q?=3A_Surpress_spurious_build_warnings_when_updating_importlib=27?= =?utf-8?q?s?= Message-ID: <20160820021330.13921.32602.7432FD66@psf.io> https://hg.python.org/cpython/rev/97b996d81c62 changeset: 102791:97b996d81c62 branch: 3.5 parent: 102769:df4efc23ab18 user: Ned Deily date: Fri Aug 19 22:12:06 2016 -0400 summary: Issue #27713: Surpress spurious build warnings when updating importlib's bootstrap files: Could not find platform dependent libraries [:] Patch by Xiang Zhang files: Misc/NEWS | 3 +++ Programs/_freeze_importlib.c | 1 + 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -177,6 +177,9 @@ Build ----- +- Issue #27713: Surpress spurious build warnings when updating importlib's + bootstrap files. Patch by Xiang Zhang + - Issue #25825: Correct the references to Modules/python.exp, which is required on AIX. The references were accidentally changed in 3.5.0a1. diff --git a/Programs/_freeze_importlib.c b/Programs/_freeze_importlib.c --- a/Programs/_freeze_importlib.c +++ b/Programs/_freeze_importlib.c @@ -77,6 +77,7 @@ Py_NoUserSiteDirectory++; Py_NoSiteFlag++; Py_IgnoreEnvironmentFlag++; + Py_FrozenFlag++; Py_SetProgramName(L"./_freeze_importlib"); /* Don't install importlib, since it could execute outdated bytecode. */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Aug 19 22:13:30 2016 From: python-checkins at python.org (ned.deily) Date: Sat, 20 Aug 2016 02:13:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327713=3A_merge_from_3=2E5?= Message-ID: <20160820021330.11031.61315.19EC4FBC@psf.io> https://hg.python.org/cpython/rev/751ca458d669 changeset: 102792:751ca458d669 parent: 102790:a7f3678f9509 parent: 102791:97b996d81c62 user: Ned Deily date: Fri Aug 19 22:13:14 2016 -0400 summary: Issue #27713: merge from 3.5 files: Misc/NEWS | 3 +++ Programs/_freeze_importlib.c | 1 + 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -211,6 +211,9 @@ Build ----- +- Issue #27713: Surpress spurious build warnings when updating importlib's + bootstrap files. Patch by Xiang Zhang + - Issue #25825: Correct the references to Modules/python.exp, which is required on AIX. The references were accidentally changed in 3.5.0a1. diff --git a/Programs/_freeze_importlib.c b/Programs/_freeze_importlib.c --- a/Programs/_freeze_importlib.c +++ b/Programs/_freeze_importlib.c @@ -77,6 +77,7 @@ Py_NoUserSiteDirectory++; Py_NoSiteFlag++; Py_IgnoreEnvironmentFlag++; + Py_FrozenFlag++; Py_SetProgramName(L"./_freeze_importlib"); /* Don't install importlib, since it could execute outdated bytecode. */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 03:01:23 2016 From: python-checkins at python.org (ethan.furman) Date: Sat, 20 Aug 2016 07:01:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue26988=3A_remove_AutoE?= =?utf-8?q?num?= Message-ID: <20160820070116.28897.15191.F137BA28@psf.io> https://hg.python.org/cpython/rev/2e243f78720e changeset: 102793:2e243f78720e user: Ethan Furman date: Sat Aug 20 00:00:52 2016 -0700 summary: Issue26988: remove AutoEnum files: Doc/library/enum.rst | 281 +++++-------------------- Lib/enum.py | 133 +----------- Lib/test/test_enum.py | 324 +----------------------------- Misc/NEWS | 2 - 4 files changed, 77 insertions(+), 663 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -37,13 +37,6 @@ Base class for creating enumerated constants that are also subclasses of :class:`int`. -.. class:: AutoEnum - - Base class for creating automatically numbered members (may - be combined with IntEnum if desired). - - .. versionadded:: 3.6 - .. function:: unique Enum class decorator that ensures only one name is bound to any one value. @@ -54,14 +47,14 @@ Enumerations are created using the :keyword:`class` syntax, which makes them easy to read and write. An alternative creation method is described in -`Functional API`_. To define a simple enumeration, subclass :class:`AutoEnum` -as follows:: +`Functional API`_. To define an enumeration, subclass :class:`Enum` as +follows:: - >>> from enum import AutoEnum - >>> class Color(AutoEnum): - ... red - ... green - ... blue + >>> from enum import Enum + >>> class Color(Enum): + ... red = 1 + ... green = 2 + ... blue = 3 ... .. note:: Nomenclature @@ -79,33 +72,6 @@ are not normal Python classes. See `How are Enums different?`_ for more details. -To create your own automatic :class:`Enum` classes, you need to add a -:meth:`_generate_next_value_` method; it will be used to create missing values -for any members after its definition. - -.. versionadded:: 3.6 - -If you need full control of the member values, use :class:`Enum` as the base -class and specify the values manually:: - - >>> from enum import Enum - >>> class Color(Enum): - ... red = 19 - ... green = 7.9182 - ... blue = 'periwinkle' - ... - -We'll use the following Enum for the examples below:: - - >>> class Color(Enum): - ... red = 1 - ... green = 2 - ... blue = 3 - ... - -Enum Details ------------- - Enumeration members have human readable string representations:: >>> print(Color.red) @@ -269,11 +235,7 @@ The ``__members__`` attribute can be used for detailed programmatic access to the enumeration members. For example, finding all the aliases:: - >>> [ - ... name - ... for name, member in Shape.__members__.items() - ... if member.name != name - ... ] + >>> [name for name, member in Shape.__members__.items() if member.name != name] ['alias_for_square'] @@ -295,7 +257,7 @@ >>> Color.red < Color.blue Traceback (most recent call last): File "", line 1, in - TypeError: '<' not supported between instances of 'Color' and 'Color' + TypeError: unorderable types: Color() < Color() Equality comparisons are defined though:: @@ -318,10 +280,10 @@ ---------------------------------------------- The examples above use integers for enumeration values. Using integers is -short and handy (and provided by default by :class:`AutoEnum` and the -`Functional API`_), but not strictly enforced. In the vast majority of -use-cases, one doesn't care what the actual value of an enumeration is. -But if the value *is* important, enumerations can have arbitrary values. +short and handy (and provided by default by the `Functional API`_), but not +strictly enforced. In the vast majority of use-cases, one doesn't care what +the actual value of an enumeration is. But if the value *is* important, +enumerations can have arbitrary values. Enumerations are Python classes, and can have methods and special methods as usual. If we have this enumeration:: @@ -431,21 +393,17 @@ >>> list(Animal) [, , , ] -The semantics of this API resemble :class:`~collections.namedtuple`. +The semantics of this API resemble :class:`~collections.namedtuple`. The first +argument of the call to :class:`Enum` is the name of the enumeration. -- the first argument of the call to :class:`Enum` is the name of the - enumeration; - -- the second argument is the *source* of enumeration member names. It can be a - whitespace-separated string of names, a sequence of names, a sequence of - 2-tuples with key/value pairs, or a mapping (e.g. dictionary) of names to - values; - -- the last two options enable assigning arbitrary values to enumerations; the - others auto-assign increasing integers starting with 1 (use the ``start`` - parameter to specify a different starting value). A new class derived from - :class:`Enum` is returned. In other words, the above assignment to - :class:`Animal` is equivalent to:: +The second argument is the *source* of enumeration member names. It can be a +whitespace-separated string of names, a sequence of names, a sequence of +2-tuples with key/value pairs, or a mapping (e.g. dictionary) of names to +values. The last two options enable assigning arbitrary values to +enumerations; the others auto-assign increasing integers starting with 1 (use +the ``start`` parameter to specify a different starting value). A +new class derived from :class:`Enum` is returned. In other words, the above +assignment to :class:`Animal` is equivalent to:: >>> class Animal(Enum): ... ant = 1 @@ -461,7 +419,7 @@ Pickling enums created with the functional API can be tricky as frame stack implementation details are used to try and figure out which module the enumeration is being created in (e.g. it will fail if you use a utility -function in a separate module, and also may not work on IronPython or Jython). +function in separate module, and also may not work on IronPython or Jython). The solution is to specify the module name explicitly as follows:: >>> Animal = Enum('Animal', 'ant bee cat dog', module=__name__) @@ -481,15 +439,7 @@ The complete signature is:: - Enum( - value='NewEnumName', - names=<...>, - *, - module='...', - qualname='...', - type=, - start=1, - ) + Enum(value='NewEnumName', names=<...>, *, module='...', qualname='...', type=, start=1) :value: What the new Enum class will record as its name. @@ -525,41 +475,10 @@ Derived Enumerations -------------------- -AutoEnum -^^^^^^^^ - -This version of :class:`Enum` automatically assigns numbers as the values -for the enumeration members, while still allowing values to be specified -when needed:: - - >>> from enum import AutoEnum - >>> class Color(AutoEnum): - ... red - ... green = 5 - ... blue - ... - >>> list(Color) - [, , ] - -.. note:: Name Lookup - - By default the names :func:`property`, :func:`classmethod`, and - :func:`staticmethod` are shielded from becoming members. To enable - them, or to specify a different set of shielded names, specify the - ignore parameter:: - - >>> class AddressType(AutoEnum, ignore='classmethod staticmethod'): - ... pobox - ... mailbox - ... property - ... - -.. versionadded:: 3.6 - IntEnum ^^^^^^^ -Another variation of :class:`Enum` which is also a subclass of +A variation of :class:`Enum` is provided which is also a subclass of :class:`int`. Members of an :class:`IntEnum` can be compared to integers; by extension, integer enumerations of different types can also be compared to each other:: @@ -602,13 +521,14 @@ >>> [i for i in range(Shape.square)] [0, 1] -For the vast majority of code, :class:`Enum` and :class:`AutoEnum` are strongly -recommended, since :class:`IntEnum` breaks some semantic promises of an -enumeration (by being comparable to integers, and thus by transitivity to other -unrelated ``IntEnum`` enumerations). It should be used only in special cases -where there's no other choice; for example, when integer constants are replaced -with enumerations and backwards compatibility is required with code that still -expects integers. +For the vast majority of code, :class:`Enum` is strongly recommended, +since :class:`IntEnum` breaks some semantic promises of an enumeration (by +being comparable to integers, and thus by transitivity to other +unrelated enumerations). It should be used only in special cases where +there's no other choice; for example, when integer constants are +replaced with enumerations and backwards compatibility is required with code +that still expects integers. + Others ^^^^^^ @@ -620,9 +540,7 @@ pass This demonstrates how similar derived enumerations can be defined; for example -an :class:`AutoIntEnum` that mixes in :class:`int` with :class:`AutoEnum` -to get members that are :class:`int` (but keep in mind the warnings for -:class:`IntEnum`). +a :class:`StrEnum` that mixes in :class:`str` instead of :class:`int`. Some rules: @@ -649,35 +567,31 @@ Interesting examples -------------------- -While :class:`Enum`, :class:`AutoEnum`, and :class:`IntEnum` are expected -to cover the majority of use-cases, they cannot cover them all. Here are -recipes for some different types of enumerations that can be used directly, -or as examples for creating one's own. +While :class:`Enum` and :class:`IntEnum` are expected to cover the majority of +use-cases, they cannot cover them all. Here are recipes for some different +types of enumerations that can be used directly, or as examples for creating +one's own. -AutoDocEnum -^^^^^^^^^^^ +AutoNumber +^^^^^^^^^^ -Automatically numbers the members, and uses the given value as the -:attr:`__doc__` string:: +Avoids having to specify the value for each enumeration member:: - >>> class AutoDocEnum(Enum): - ... def __new__(cls, doc): + >>> class AutoNumber(Enum): + ... def __new__(cls): ... value = len(cls.__members__) + 1 ... obj = object.__new__(cls) ... obj._value_ = value - ... obj.__doc__ = doc ... return obj ... - >>> class Color(AutoDocEnum): - ... red = 'stop' - ... green = 'go' - ... blue = 'what?' + >>> class Color(AutoNumber): + ... red = () + ... green = () + ... blue = () ... >>> Color.green.value == 2 True - >>> Color.green.__doc__ - 'go' .. note:: @@ -685,23 +599,6 @@ members; it is then replaced by Enum's :meth:`__new__` which is used after class creation for lookup of existing members. -AutoNameEnum -^^^^^^^^^^^^ - -Automatically sets the member's value to its name:: - - >>> class AutoNameEnum(Enum): - ... def _generate_next_value_(name, start, count, last_value): - ... return name - ... - >>> class Color(AutoNameEnum): - ... red - ... green - ... blue - ... - >>> Color.green.value == 'green' - True - OrderedEnum ^^^^^^^^^^^ @@ -834,63 +731,10 @@ Finer Points ^^^^^^^^^^^^ -Enum class signature -~~~~~~~~~~~~~~~~~~~~ - -:: - - class SomeName( - AnEnum, - start=None, - ignore='staticmethod classmethod property', - ): - -*start* can be used by a :meth:`_generate_next_value_` method to specify a -starting value. - -*ignore* specifies which names, if any, will not attempt to auto-generate -a new value (they will also be removed from the class body). - - -Supported ``__dunder__`` names -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The :attr:`__members__` attribute is only available on the class. - -:meth:`__new__`, if specified, must create and return the enum members; it is -also a very good idea to set the member's :attr:`_value_` appropriately. Once -all the members are created it is no longer used. - - -Supported ``_sunder_`` names -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -- ``_order_`` -- used in Python 2/3 code to ensure member order is consistent [class attribute] - -- ``_name_`` -- name of the member (but use ``name`` for normal access) -- ``_value_`` -- value of the member; can be set / modified in ``__new__`` (see ``_name_``) -- ``_missing_`` -- a lookup function used when a value is not found (only after class creation) -- ``_generate_next_value_`` -- a function to generate missing values (only during class creation) - - -:meth:`_generate_next_value_` signature -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - ``def _generate_next_value_(name, start, count, last_value):`` - -- ``name`` is the name of the member -- ``start`` is the initital start value (if any) or None -- ``count`` is the number of existing members in the enumeration -- ``last_value`` is the value of the last enum member (if any) or None - - -Enum member type -~~~~~~~~~~~~~~~~ - -``Enum`` members are instances of an ``Enum`` class, and even -though they are accessible as ``EnumClass.member``, they should not be accessed +:class:`Enum` members are instances of an :class:`Enum` class, and even +though they are accessible as `EnumClass.member`, they should not be accessed directly from the member as that lookup may fail or, worse, return something -besides the ``Enum`` member you are looking for:: +besides the :class:`Enum` member you looking for:: >>> class FieldTypes(Enum): ... name = 0 @@ -904,24 +748,18 @@ .. versionchanged:: 3.5 - -Boolean value of ``Enum`` classes and members -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Enum classes that are mixed with non-Enum types (such as +Boolean evaluation: Enum classes that are mixed with non-Enum types (such as :class:`int`, :class:`str`, etc.) are evaluated according to the mixed-in -type's rules; otherwise, all members evaluate as :data:`True`. To make your own +type's rules; otherwise, all members evaluate as ``True``. To make your own Enum's boolean evaluation depend on the member's value add the following to your class:: def __bool__(self): return bool(self.value) +The :attr:`__members__` attribute is only available on the class. -Enum classes with methods -~~~~~~~~~~~~~~~~~~~~~~~~~ - -If you give your ``Enum`` subclass extra methods, like the `Planet`_ +If you give your :class:`Enum` subclass extra methods, like the `Planet`_ class above, those methods will show up in a :func:`dir` of the member, but not of the class:: @@ -929,3 +767,12 @@ ['EARTH', 'JUPITER', 'MARS', 'MERCURY', 'NEPTUNE', 'SATURN', 'URANUS', 'VENUS', '__class__', '__doc__', '__members__', '__module__'] >>> dir(Planet.EARTH) ['__class__', '__doc__', '__module__', 'name', 'surface_gravity', 'value'] + +The :meth:`__new__` method will only be used for the creation of the +:class:`Enum` members -- after that it is replaced. Any custom :meth:`__new__` +method must create the object and set the :attr:`_value_` attribute +appropriately. + +If you wish to change how :class:`Enum` members are looked up you should either +write a helper function or a :func:`classmethod` for the :class:`Enum` +subclass. diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -8,9 +8,7 @@ from collections import OrderedDict -__all__ = [ - 'EnumMeta', 'Enum', 'IntEnum', 'AutoEnum', 'unique', - ] +__all__ = ['EnumMeta', 'Enum', 'IntEnum', 'unique'] def _is_descriptor(obj): @@ -54,30 +52,7 @@ """ def __init__(self): super().__init__() - # list of enum members self._member_names = [] - # starting value - self._start = None - # last assigned value - self._last_value = None - # when the magic turns off - self._locked = True - # list of temporary names - self._ignore = [] - - def __getitem__(self, key): - if ( - self._generate_next_value_ is None - or self._locked - or key in self - or key in self._ignore - or _is_sunder(key) - or _is_dunder(key) - ): - return super(_EnumDict, self).__getitem__(key) - next_value = self._generate_next_value_(key, self._start, len(self._member_names), self._last_value) - self[key] = next_value - return next_value def __setitem__(self, key, value): """Changes anything not dundered or not a descriptor. @@ -89,55 +64,19 @@ """ if _is_sunder(key): - if key not in ('_settings_', '_order_', '_ignore_', '_start_', '_generate_next_value_'): - raise ValueError('_names_ are reserved for future Enum use') - elif key == '_generate_next_value_': - if isinstance(value, staticmethod): - value = value.__get__(None, self) - self._generate_next_value_ = value - self._locked = False - elif key == '_ignore_': - if isinstance(value, str): - value = value.split() - else: - value = list(value) - self._ignore = value - already = set(value) & set(self._member_names) - if already: - raise ValueError( - '_ignore_ cannot specify already set names: %r' - % (already, )) - elif key == '_start_': - self._start = value - self._locked = False + raise ValueError('_names_ are reserved for future Enum use') elif _is_dunder(key): - if key == '__order__': - key = '_order_' - if _is_descriptor(value): - self._locked = True + pass elif key in self._member_names: # descriptor overwriting an enum? raise TypeError('Attempted to reuse key: %r' % key) - elif key in self._ignore: - pass elif not _is_descriptor(value): if key in self: # enum overwriting a descriptor? - raise TypeError('%r already defined as: %r' % (key, self[key])) + raise TypeError('Key already defined as: %r' % self[key]) self._member_names.append(key) - if self._generate_next_value_ is not None: - self._last_value = value - else: - # not a new member, turn off the autoassign magic - self._locked = True super().__setitem__(key, value) - # for magic "auto values" an Enum class should specify a `_generate_next_value_` - # method; that method will be used to generate missing values, and is - # implicitly a staticmethod; - # the signature should be `def _generate_next_value_(name, last_value)` - # last_value will be the last value created and/or assigned, or None - _generate_next_value_ = None # Dummy value for Enum as EnumMeta explicitly checks for it, but of course @@ -145,31 +84,14 @@ # This is also why there are checks in EnumMeta like `if Enum is not None` Enum = None -_ignore_sentinel = object() + class EnumMeta(type): """Metaclass for Enum""" @classmethod - def __prepare__(metacls, cls, bases, start=None, ignore=_ignore_sentinel): - # create the namespace dict - enum_dict = _EnumDict() - # inherit previous flags and _generate_next_value_ function - member_type, first_enum = metacls._get_mixins_(bases) - if first_enum is not None: - enum_dict['_generate_next_value_'] = getattr(first_enum, '_generate_next_value_', None) - if start is None: - start = getattr(first_enum, '_start_', None) - if ignore is _ignore_sentinel: - enum_dict['_ignore_'] = 'property classmethod staticmethod'.split() - elif ignore: - enum_dict['_ignore_'] = ignore - if start is not None: - enum_dict['_start_'] = start - return enum_dict + def __prepare__(metacls, cls, bases): + return _EnumDict() - def __init__(cls, *args , **kwds): - super(EnumMeta, cls).__init__(*args) - - def __new__(metacls, cls, bases, classdict, **kwds): + def __new__(metacls, cls, bases, classdict): # an Enum class is final once enumeration items have been defined; it # cannot be mixed with other types (int, float, etc.) if it has an # inherited __new__ unless a new __new__ is defined (or the resulting @@ -180,24 +102,12 @@ # save enum items into separate mapping so they don't get baked into # the new class - enum_members = {k: classdict[k] for k in classdict._member_names} + members = {k: classdict[k] for k in classdict._member_names} for name in classdict._member_names: del classdict[name] - # adjust the sunders - _order_ = classdict.pop('_order_', None) - classdict.pop('_ignore_', None) - - # py3 support for definition order (helps keep py2/py3 code in sync) - if _order_ is not None: - if isinstance(_order_, str): - _order_ = _order_.replace(',', ' ').split() - unique_members = [n for n in clsdict._member_names if n in _order_] - if _order_ != unique_members: - raise TypeError('member order does not match _order_') - # check for illegal enum names (any others?) - invalid_names = set(enum_members) & {'mro', } + invalid_names = set(members) & {'mro', } if invalid_names: raise ValueError('Invalid enum member name: {0}'.format( ','.join(invalid_names))) @@ -241,7 +151,7 @@ # a custom __new__ is doing something funky with the values -- such as # auto-numbering ;) for member_name in classdict._member_names: - value = enum_members[member_name] + value = members[member_name] if not isinstance(value, tuple): args = (value, ) else: @@ -255,10 +165,7 @@ else: enum_member = __new__(enum_class, *args) if not hasattr(enum_member, '_value_'): - if member_type is object: - enum_member._value_ = value - else: - enum_member._value_ = member_type(*args) + enum_member._value_ = member_type(*args) value = enum_member._value_ enum_member._name_ = member_name enum_member.__objclass__ = enum_class @@ -665,22 +572,6 @@ def _reduce_ex_by_name(self, proto): return self.name -class AutoEnum(Enum): - """Enum where values are automatically assigned.""" - def _generate_next_value_(name, start, count, last_value): - """ - Generate the next value when not given. - - name: the name of the member - start: the initital start value or None - count: the number of existing members - last_value: the last value assigned or None - """ - # add one to the last assigned value - if not count: - return start if start is not None else 1 - return last_value + 1 - def unique(enumeration): """Class decorator for enumerations ensuring unique member values.""" duplicates = [] diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -3,7 +3,7 @@ import pydoc import unittest from collections import OrderedDict -from enum import EnumMeta, Enum, IntEnum, AutoEnum, unique +from enum import Enum, IntEnum, EnumMeta, unique from io import StringIO from pickle import dumps, loads, PicklingError, HIGHEST_PROTOCOL from test import support @@ -1570,328 +1570,6 @@ self.assertEqual(LabelledList.unprocessed, 1) self.assertEqual(LabelledList(1), LabelledList.unprocessed) - def test_ignore_as_str(self): - from datetime import timedelta - class Period(Enum, ignore='Period i'): - """ - different lengths of time - """ - def __new__(cls, value, period): - obj = object.__new__(cls) - obj._value_ = value - obj.period = period - return obj - Period = vars() - for i in range(367): - Period['Day%d' % i] = timedelta(days=i), 'day' - for i in range(53): - Period['Week%d' % i] = timedelta(days=i*7), 'week' - for i in range(13): - Period['Month%d' % i] = i, 'month' - OneDay = Day1 - OneWeek = Week1 - self.assertEqual(Period.Day7.value, timedelta(days=7)) - self.assertEqual(Period.Day7.period, 'day') - - def test_ignore_as_list(self): - from datetime import timedelta - class Period(Enum, ignore=['Period', 'i']): - """ - different lengths of time - """ - def __new__(cls, value, period): - obj = object.__new__(cls) - obj._value_ = value - obj.period = period - return obj - Period = vars() - for i in range(367): - Period['Day%d' % i] = timedelta(days=i), 'day' - for i in range(53): - Period['Week%d' % i] = timedelta(days=i*7), 'week' - for i in range(13): - Period['Month%d' % i] = i, 'month' - OneDay = Day1 - OneWeek = Week1 - self.assertEqual(Period.Day7.value, timedelta(days=7)) - self.assertEqual(Period.Day7.period, 'day') - - def test_new_with_no_value_and_int_base_class(self): - class NoValue(int, Enum): - def __new__(cls, value): - obj = int.__new__(cls, value) - obj.index = len(cls.__members__) - return obj - this = 1 - that = 2 - self.assertEqual(list(NoValue), [NoValue.this, NoValue.that]) - self.assertEqual(NoValue.this, 1) - self.assertEqual(NoValue.this.value, 1) - self.assertEqual(NoValue.this.index, 0) - self.assertEqual(NoValue.that, 2) - self.assertEqual(NoValue.that.value, 2) - self.assertEqual(NoValue.that.index, 1) - - def test_new_with_no_value(self): - class NoValue(Enum): - def __new__(cls, value): - obj = object.__new__(cls) - obj.index = len(cls.__members__) - return obj - this = 1 - that = 2 - self.assertEqual(list(NoValue), [NoValue.this, NoValue.that]) - self.assertEqual(NoValue.this.value, 1) - self.assertEqual(NoValue.this.index, 0) - self.assertEqual(NoValue.that.value, 2) - self.assertEqual(NoValue.that.index, 1) - - -class TestAutoNumber(unittest.TestCase): - - def test_autonumbering(self): - class Color(AutoEnum): - red - green - blue - self.assertEqual(list(Color), [Color.red, Color.green, Color.blue]) - self.assertEqual(Color.red.value, 1) - self.assertEqual(Color.green.value, 2) - self.assertEqual(Color.blue.value, 3) - - def test_autointnumbering(self): - class Color(int, AutoEnum): - red - green - blue - self.assertTrue(isinstance(Color.red, int)) - self.assertEqual(Color.green, 2) - self.assertTrue(Color.blue > Color.red) - - def test_autonumbering_with_start(self): - class Color(AutoEnum, start=7): - red - green - blue - self.assertEqual(list(Color), [Color.red, Color.green, Color.blue]) - self.assertEqual(Color.red.value, 7) - self.assertEqual(Color.green.value, 8) - self.assertEqual(Color.blue.value, 9) - - def test_autonumbering_with_start_and_skip(self): - class Color(AutoEnum, start=7): - red - green - blue = 11 - brown - self.assertEqual(list(Color), [Color.red, Color.green, Color.blue, Color.brown]) - self.assertEqual(Color.red.value, 7) - self.assertEqual(Color.green.value, 8) - self.assertEqual(Color.blue.value, 11) - self.assertEqual(Color.brown.value, 12) - - - def test_badly_overridden_ignore(self): - with self.assertRaisesRegex(TypeError, "'int' object is not callable"): - class Color(AutoEnum): - _ignore_ = () - red - green - blue - @property - def whatever(self): - pass - with self.assertRaisesRegex(TypeError, "'int' object is not callable"): - class Color(AutoEnum, ignore=None): - red - green - blue - @property - def whatever(self): - pass - with self.assertRaisesRegex(TypeError, "'int' object is not callable"): - class Color(AutoEnum, ignore='classmethod staticmethod'): - red - green - blue - @property - def whatever(self): - pass - - def test_property(self): - class Color(AutoEnum): - red - green - blue - @property - def cap_name(self): - return self.name.title() - self.assertEqual(Color.blue.cap_name, 'Blue') - - def test_magic_turns_off(self): - with self.assertRaisesRegex(NameError, "brown"): - class Color(AutoEnum): - red - green - blue - @property - def cap_name(self): - return self.name.title() - brown - - with self.assertRaisesRegex(NameError, "rose"): - class Color(AutoEnum): - red - green - blue - def hello(self): - print('Hello! My serial is %s.' % self.value) - rose - - with self.assertRaisesRegex(NameError, "cyan"): - class Color(AutoEnum): - red - green - blue - def __init__(self, *args): - pass - cyan - - -class TestGenerateMethod(unittest.TestCase): - - def test_autonaming(self): - class Color(Enum): - def _generate_next_value_(name, start, count, last_value): - return name - Red - Green - Blue - self.assertEqual(list(Color), [Color.Red, Color.Green, Color.Blue]) - self.assertEqual(Color.Red.value, 'Red') - self.assertEqual(Color.Green.value, 'Green') - self.assertEqual(Color.Blue.value, 'Blue') - - def test_autonamestr(self): - class Color(str, Enum): - def _generate_next_value_(name, start, count, last_value): - return name - Red - Green - Blue - self.assertTrue(isinstance(Color.Red, str)) - self.assertEqual(Color.Green, 'Green') - self.assertTrue(Color.Blue < Color.Red) - - def test_generate_as_staticmethod(self): - class Color(str, Enum): - @staticmethod - def _generate_next_value_(name, start, count, last_value): - return name.lower() - Red - Green - Blue - self.assertTrue(isinstance(Color.Red, str)) - self.assertEqual(Color.Green, 'green') - self.assertTrue(Color.Blue < Color.Red) - - - def test_overridden_ignore(self): - with self.assertRaisesRegex(TypeError, "'str' object is not callable"): - class Color(Enum): - def _generate_next_value_(name, start, count, last_value): - return name - _ignore_ = () - red - green - blue - @property - def whatever(self): - pass - with self.assertRaisesRegex(TypeError, "'str' object is not callable"): - class Color(Enum, ignore=None): - def _generate_next_value_(name, start, count, last_value): - return name - red - green - blue - @property - def whatever(self): - pass - - def test_property(self): - class Color(Enum): - def _generate_next_value_(name, start, count, last_value): - return name - red - green - blue - @property - def upper_name(self): - return self.name.upper() - self.assertEqual(Color.blue.upper_name, 'BLUE') - - def test_magic_turns_off(self): - with self.assertRaisesRegex(NameError, "brown"): - class Color(Enum): - def _generate_next_value_(name, start, count, last_value): - return name - red - green - blue - @property - def cap_name(self): - return self.name.title() - brown - - with self.assertRaisesRegex(NameError, "rose"): - class Color(Enum): - def _generate_next_value_(name, start, count, last_value): - return name - red - green - blue - def hello(self): - print('Hello! My value %s.' % self.value) - rose - - with self.assertRaisesRegex(NameError, "cyan"): - class Color(Enum): - def _generate_next_value_(name, start, count, last_value): - return name - red - green - blue - def __init__(self, *args): - pass - cyan - - def test_powers_of_two(self): - class Bits(Enum): - def _generate_next_value_(name, start, count, last_value): - return 2 ** count - one - two - four - eight - self.assertEqual(Bits.one.value, 1) - self.assertEqual(Bits.two.value, 2) - self.assertEqual(Bits.four.value, 4) - self.assertEqual(Bits.eight.value, 8) - - def test_powers_of_two_as_int(self): - class Bits(int, Enum): - def _generate_next_value_(name, start, count, last_value): - return 2 ** count - one - two - four - eight - self.assertEqual(Bits.one, 1) - self.assertEqual(Bits.two, 2) - self.assertEqual(Bits.four, 4) - self.assertEqual(Bits.eight, 8) - class TestUnique(unittest.TestCase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -179,8 +179,6 @@ - Issue #27512: Fix a segfault when os.fspath() called a an __fspath__() method that raised an exception. Patch by Xiang Zhang. -- Issue #26988: Add AutoEnum. - Tests ----- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 03:45:14 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 20 Aug 2016 07:45:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327787=3A_Merge_regrtest_fixup_from_3=2E5?= Message-ID: <20160820074514.13973.91339.43524F19@psf.io> https://hg.python.org/cpython/rev/e5777c5d108c changeset: 102797:e5777c5d108c parent: 102793:2e243f78720e parent: 102794:7ddbc2263943 user: Martin Panter date: Sat Aug 20 07:38:21 2016 +0000 summary: Issue #27787: Merge regrtest fixup from 3.5 files: Lib/test/libregrtest/save_env.py | 1 + Misc/NEWS | 6 ++++++ 2 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Lib/test/libregrtest/save_env.py b/Lib/test/libregrtest/save_env.py --- a/Lib/test/libregrtest/save_env.py +++ b/Lib/test/libregrtest/save_env.py @@ -268,6 +268,7 @@ def __exit__(self, exc_type, exc_val, exc_tb): saved_values = self.saved_values del self.saved_values + support.gc_collect() # Some resources use weak references for name, get, restore in self.resource_info(): current = get() original = saved_values.pop(name) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,12 @@ - Issue #12713: Allowed abbreviation of subcommands by end-users for users of argparse. +Tests +----- + +- Issue #27787: Call gc.collect() before checking each test for "dangling + threads", since the dangling threads are weak references. + What's New in Python 3.6.0 alpha 4 ================================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 03:45:15 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 20 Aug 2016 07:45:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327614=3A_Merge_test=5Fdocxmlrpc_from_3=2E5?= Message-ID: <20160820074514.30665.36468.D843F120@psf.io> https://hg.python.org/cpython/rev/17d688dedfca changeset: 102798:17d688dedfca parent: 102797:e5777c5d108c parent: 102795:397f05044172 user: Martin Panter date: Sat Aug 20 07:39:15 2016 +0000 summary: Issue #27614: Merge test_docxmlrpc from 3.5 files: Lib/test/test_docxmlrpc.py | 41 ++++++------------------- 1 files changed, 11 insertions(+), 30 deletions(-) diff --git a/Lib/test/test_docxmlrpc.py b/Lib/test/test_docxmlrpc.py --- a/Lib/test/test_docxmlrpc.py +++ b/Lib/test/test_docxmlrpc.py @@ -3,12 +3,8 @@ import sys from test import support threading = support.import_module('threading') -import time -import socket import unittest -PORT = None - def make_request_and_skipIf(condition, reason): # If we skip the test, we have to make a request because # the server created in setUp blocks expecting one to come in. @@ -23,13 +19,10 @@ return decorator -def server(evt, numrequests): +def make_server(): serv = DocXMLRPCServer(("localhost", 0), logRequests=False) try: - global PORT - PORT = serv.socket.getsockname()[1] - # Add some documentation serv.set_server_title("DocXMLRPCServer Test Documentation") serv.set_server_name("DocXMLRPCServer Test Docs") @@ -66,43 +59,31 @@ serv.register_function(lambda x, y: x-y) serv.register_function(annotation) serv.register_instance(ClassWithAnnotation()) - - while numrequests > 0: - serv.handle_request() - numrequests -= 1 - except socket.timeout: - pass - finally: + return serv + except: serv.server_close() - PORT = None - evt.set() + raise class DocXMLRPCHTTPGETServer(unittest.TestCase): def setUp(self): - self._threads = support.threading_setup() # Enable server feedback DocXMLRPCServer._send_traceback_header = True - self.evt = threading.Event() - threading.Thread(target=server, args=(self.evt, 1)).start() + self.serv = make_server() + self.thread = threading.Thread(target=self.serv.serve_forever) + self.thread.start() - # wait for port to be assigned - deadline = time.monotonic() + 10.0 - while PORT is None: - time.sleep(0.010) - if time.monotonic() > deadline: - break - + PORT = self.serv.server_address[1] self.client = http.client.HTTPConnection("localhost:%d" % PORT) def tearDown(self): self.client.close() - self.evt.wait() - # Disable server feedback DocXMLRPCServer._send_traceback_header = False - support.threading_cleanup(*self._threads) + self.serv.shutdown() + self.thread.join() + self.serv.server_close() def test_valid_get_response(self): self.client.request("GET", "/") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 03:45:14 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 20 Aug 2016 07:45:14 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3Nzg3?= =?utf-8?q?=3A_Clean_up_weak_references_before_checking_for_dangling_threa?= =?utf-8?q?ds?= Message-ID: <20160820074514.11050.21660.B1117BEB@psf.io> https://hg.python.org/cpython/rev/7ddbc2263943 changeset: 102794:7ddbc2263943 branch: 3.5 parent: 102791:97b996d81c62 user: Martin Panter date: Sat Aug 20 06:50:14 2016 +0000 summary: Issue #27787: Clean up weak references before checking for dangling threads files: Lib/test/regrtest.py | 1 + Misc/NEWS | 3 +++ 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py --- a/Lib/test/regrtest.py +++ b/Lib/test/regrtest.py @@ -1258,6 +1258,7 @@ def __exit__(self, exc_type, exc_val, exc_tb): saved_values = self.saved_values del self.saved_values + support.gc_collect() # Some resources use weak references for name, get, restore in self.resource_info(): current = get() original = saved_values.pop(name) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -155,6 +155,9 @@ Tests ----- +- Issue #27787: Call gc.collect() before checking each test for "dangling + threads", since the dangling threads are weak references. + - Issue #27369: In test_pyexpat, avoid testing an error message detail that changed in Expat 2.2.0. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 03:45:14 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 20 Aug 2016 07:45:14 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3NjE0?= =?utf-8?q?=3A_Avoid_race_in_test=5Fdocxmlrpc_server_setup?= Message-ID: <20160820074514.20787.75157.AF44E5E2@psf.io> https://hg.python.org/cpython/rev/7136304ecf4c changeset: 102796:7136304ecf4c branch: 2.7 parent: 102771:7669fb39a9ce user: Martin Panter date: Thu Aug 18 06:12:22 2016 +0000 summary: Issue #27614: Avoid race in test_docxmlrpc server setup files: Lib/test/test_docxmlrpc.py | 40 +++++++------------------ 1 files changed, 11 insertions(+), 29 deletions(-) diff --git a/Lib/test/test_docxmlrpc.py b/Lib/test/test_docxmlrpc.py --- a/Lib/test/test_docxmlrpc.py +++ b/Lib/test/test_docxmlrpc.py @@ -3,12 +3,8 @@ import sys from test import test_support threading = test_support.import_module('threading') -import time -import socket import unittest -PORT = None - def make_request_and_skipIf(condition, reason): # If we skip the test, we have to make a request because # the server created in setUp blocks expecting one to come in. @@ -23,13 +19,10 @@ return decorator -def server(evt, numrequests): +def make_server(): serv = DocXMLRPCServer(("localhost", 0), logRequests=False) try: - global PORT - PORT = serv.socket.getsockname()[1] - # Add some documentation serv.set_server_title("DocXMLRPCServer Test Documentation") serv.set_server_name("DocXMLRPCServer Test Docs") @@ -56,42 +49,31 @@ serv.register_function(add) serv.register_function(lambda x, y: x-y) - - while numrequests > 0: - serv.handle_request() - numrequests -= 1 - except socket.timeout: - pass - finally: + return serv + except: serv.server_close() - PORT = None - evt.set() + raise class DocXMLRPCHTTPGETServer(unittest.TestCase): def setUp(self): - self._threads = test_support.threading_setup() # Enable server feedback DocXMLRPCServer._send_traceback_header = True - self.evt = threading.Event() - threading.Thread(target=server, args=(self.evt, 1)).start() + self.serv = make_server() + self.thread = threading.Thread(target=self.serv.serve_forever) + self.thread.start() - # wait for port to be assigned - n = 1000 - while n > 0 and PORT is None: - time.sleep(0.001) - n -= 1 - + PORT = self.serv.server_address[1] self.client = httplib.HTTPConnection("localhost:%d" % PORT) def tearDown(self): self.client.close() - self.evt.wait() - # Disable server feedback DocXMLRPCServer._send_traceback_header = False - test_support.threading_cleanup(*self._threads) + self.serv.shutdown() + self.thread.join() + self.serv.server_close() def test_valid_get_response(self): self.client.request("GET", "/") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 03:45:14 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 20 Aug 2016 07:45:14 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NjE0?= =?utf-8?q?=3A_Avoid_race_in_test=5Fdocxmlrpc_server_setup?= Message-ID: <20160820074514.2392.36439.1E8A2BE0@psf.io> https://hg.python.org/cpython/rev/397f05044172 changeset: 102795:397f05044172 branch: 3.5 user: Martin Panter date: Sat Aug 20 06:50:58 2016 +0000 summary: Issue #27614: Avoid race in test_docxmlrpc server setup files: Lib/test/test_docxmlrpc.py | 41 ++++++------------------- 1 files changed, 11 insertions(+), 30 deletions(-) diff --git a/Lib/test/test_docxmlrpc.py b/Lib/test/test_docxmlrpc.py --- a/Lib/test/test_docxmlrpc.py +++ b/Lib/test/test_docxmlrpc.py @@ -3,12 +3,8 @@ import sys from test import support threading = support.import_module('threading') -import time -import socket import unittest -PORT = None - def make_request_and_skipIf(condition, reason): # If we skip the test, we have to make a request because # the server created in setUp blocks expecting one to come in. @@ -23,13 +19,10 @@ return decorator -def server(evt, numrequests): +def make_server(): serv = DocXMLRPCServer(("localhost", 0), logRequests=False) try: - global PORT - PORT = serv.socket.getsockname()[1] - # Add some documentation serv.set_server_title("DocXMLRPCServer Test Documentation") serv.set_server_name("DocXMLRPCServer Test Docs") @@ -66,43 +59,31 @@ serv.register_function(lambda x, y: x-y) serv.register_function(annotation) serv.register_instance(ClassWithAnnotation()) - - while numrequests > 0: - serv.handle_request() - numrequests -= 1 - except socket.timeout: - pass - finally: + return serv + except: serv.server_close() - PORT = None - evt.set() + raise class DocXMLRPCHTTPGETServer(unittest.TestCase): def setUp(self): - self._threads = support.threading_setup() # Enable server feedback DocXMLRPCServer._send_traceback_header = True - self.evt = threading.Event() - threading.Thread(target=server, args=(self.evt, 1)).start() + self.serv = make_server() + self.thread = threading.Thread(target=self.serv.serve_forever) + self.thread.start() - # wait for port to be assigned - deadline = time.monotonic() + 10.0 - while PORT is None: - time.sleep(0.010) - if time.monotonic() > deadline: - break - + PORT = self.serv.server_address[1] self.client = http.client.HTTPConnection("localhost:%d" % PORT) def tearDown(self): self.client.close() - self.evt.wait() - # Disable server feedback DocXMLRPCServer._send_traceback_header = False - support.threading_cleanup(*self._threads) + self.serv.shutdown() + self.thread.join() + self.serv.server_close() def test_valid_get_response(self): self.client.request("GET", "/") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 04:11:39 2016 From: python-checkins at python.org (berker.peksag) Date: Sat, 20 Aug 2016 08:11:39 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzEyOTgy?= =?utf-8?q?=3A_Thanks_to_PEP_488=2C_Python_no_longer_creates_=2Epyo_files?= Message-ID: <20160820081139.2276.86333.B85B5A15@psf.io> https://hg.python.org/cpython/rev/8e9dc3e4ea91 changeset: 102799:8e9dc3e4ea91 branch: 3.5 parent: 102795:397f05044172 user: Berker Peksag date: Sat Aug 20 11:11:52 2016 +0300 summary: Issue #12982: Thanks to PEP 488, Python no longer creates .pyo files files: Doc/using/cmdline.rst | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -239,9 +239,7 @@ .. cmdoption:: -O - Turn on basic optimizations. This changes the filename extension for - compiled (:term:`bytecode`) files from ``.pyc`` to ``.pyo``. See also - :envvar:`PYTHONOPTIMIZE`. + Turn on basic optimizations. See also :envvar:`PYTHONOPTIMIZE`. .. cmdoption:: -OO -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 04:11:39 2016 From: python-checkins at python.org (berker.peksag) Date: Sat, 20 Aug 2016 08:11:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2312982=3A_Merge_from_3=2E5?= Message-ID: <20160820081139.2522.47374.81E760D5@psf.io> https://hg.python.org/cpython/rev/1455851e7332 changeset: 102800:1455851e7332 parent: 102798:17d688dedfca parent: 102799:8e9dc3e4ea91 user: Berker Peksag date: Sat Aug 20 11:12:20 2016 +0300 summary: Issue #12982: Merge from 3.5 files: Doc/using/cmdline.rst | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -239,9 +239,7 @@ .. cmdoption:: -O - Turn on basic optimizations. This changes the filename extension for - compiled (:term:`bytecode`) files from ``.pyc`` to ``.pyo``. See also - :envvar:`PYTHONOPTIMIZE`. + Turn on basic optimizations. See also :envvar:`PYTHONOPTIMIZE`. .. cmdoption:: -OO -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 04:28:12 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 20 Aug 2016 08:28:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Minor_spelling?= =?utf-8?q?_fixes?= Message-ID: <20160820082812.30551.7858.947218D0@psf.io> https://hg.python.org/cpython/rev/a5b8af72e80c changeset: 102802:a5b8af72e80c branch: 3.5 parent: 102799:8e9dc3e4ea91 user: Martin Panter date: Sat Aug 20 08:03:06 2016 +0000 summary: Minor spelling fixes files: Doc/library/asyncio-protocol.rst | 2 +- Doc/library/email.contentmanager.rst | 2 +- Doc/library/xml.dom.pulldom.rst | 2 +- Doc/whatsnew/2.7.rst | 2 +- Include/pymacconfig.h | 4 ++-- Misc/NEWS | 2 +- 6 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -11,7 +11,7 @@ Transports are classes provided by :mod:`asyncio` in order to abstract various kinds of communication channels. You generally won't instantiate -a transport yourself; instead, you will call a :class:`AbstractEventLoop` method +a transport yourself; instead, you will call an :class:`AbstractEventLoop` method which will create the transport and try to initiate the underlying communication channel, calling you back when it succeeds. diff --git a/Doc/library/email.contentmanager.rst b/Doc/library/email.contentmanager.rst --- a/Doc/library/email.contentmanager.rst +++ b/Doc/library/email.contentmanager.rst @@ -347,7 +347,7 @@ .. method:: get_content(msg, errors='replace') - Return the payload of the part as either a string (for ``text`` parts), a + Return the payload of the part as either a string (for ``text`` parts), an :class:`~email.message.EmailMessage` object (for ``message/rfc822`` parts), or a ``bytes`` object (for all other non-multipart types). Raise a :exc:`KeyError` if called on a ``multipart``. If the part is a diff --git a/Doc/library/xml.dom.pulldom.rst b/Doc/library/xml.dom.pulldom.rst --- a/Doc/library/xml.dom.pulldom.rst +++ b/Doc/library/xml.dom.pulldom.rst @@ -74,7 +74,7 @@ .. function:: parse(stream_or_string, parser=None, bufsize=None) Return a :class:`DOMEventStream` from the given input. *stream_or_string* may be - either a file name, or a file-like object. *parser*, if given, must be a + either a file name, or a file-like object. *parser*, if given, must be an :class:`~xml.sax.xmlreader.XMLReader` object. This function will change the document handler of the parser and activate namespace support; other parser configuration (like diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -532,7 +532,7 @@ calling ``log.getChild('network.listen')`` is equivalent to ``getLogger('app.network.listen')``. -* The :class:`~logging.LoggerAdapter` class gained a +* The :class:`~logging.LoggerAdapter` class gained an :meth:`~logging.LoggerAdapter.isEnabledFor` method that takes a *level* and returns whether the underlying logger would process a message of that level of importance. diff --git a/Include/pymacconfig.h b/Include/pymacconfig.h --- a/Include/pymacconfig.h +++ b/Include/pymacconfig.h @@ -63,7 +63,7 @@ # if defined(__LP64__) /* MacOSX 10.4 (the first release to support 64-bit code * at all) only supports 64-bit in the UNIX layer. - * Therefore surpress the toolbox-glue in 64-bit mode. + * Therefore suppress the toolbox-glue in 64-bit mode. */ /* In 64-bit mode setpgrp always has no arguments, in 32-bit @@ -91,7 +91,7 @@ * * Specifically: OSX 10.4 has limited supported for '%zd', while * 10.5 has full support for '%zd'. A binary built on 10.5 won't - * work properly on 10.4 unless we surpress the definition + * work properly on 10.4 unless we suppress the definition * of PY_FORMAT_SIZE_T */ #undef PY_FORMAT_SIZE_T diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -180,7 +180,7 @@ Build ----- -- Issue #27713: Surpress spurious build warnings when updating importlib's +- Issue #27713: Suppress spurious build warnings when updating importlib's bootstrap files. Patch by Xiang Zhang - Issue #25825: Correct the references to Modules/python.exp, which is -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 04:28:12 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 20 Aug 2016 08:28:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_some_spell?= =?utf-8?q?ing_errors?= Message-ID: <20160820082812.30412.76907.E17B4F7A@psf.io> https://hg.python.org/cpython/rev/03dc2f3fa0c6 changeset: 102801:03dc2f3fa0c6 branch: 2.7 parent: 102796:7136304ecf4c user: Martin Panter date: Sat Aug 20 08:00:53 2016 +0000 summary: Fix some spelling errors files: Doc/whatsnew/2.7.rst | 2 +- Include/pymacconfig.h | 4 ++-- Misc/developers.txt | 2 +- setup.py | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -532,7 +532,7 @@ calling ``log.getChild('network.listen')`` is equivalent to ``getLogger('app.network.listen')``. -* The :class:`~logging.LoggerAdapter` class gained a +* The :class:`~logging.LoggerAdapter` class gained an :meth:`~logging.LoggerAdapter.isEnabledFor` method that takes a *level* and returns whether the underlying logger would process a message of that level of importance. diff --git a/Include/pymacconfig.h b/Include/pymacconfig.h --- a/Include/pymacconfig.h +++ b/Include/pymacconfig.h @@ -63,7 +63,7 @@ # if defined(__LP64__) /* MacOSX 10.4 (the first release to support 64-bit code * at all) only supports 64-bit in the UNIX layer. - * Therefore surpress the toolbox-glue in 64-bit mode. + * Therefore suppress the toolbox-glue in 64-bit mode. */ /* In 64-bit mode setpgrp always has no arguments, in 32-bit @@ -91,7 +91,7 @@ * * Specifically: OSX 10.4 has limited supported for '%zd', while * 10.5 has full support for '%zd'. A binary built on 10.5 won't - * work properly on 10.4 unless we surpress the definition + * work properly on 10.4 unless we suppress the definition * of PY_FORMAT_SIZE_T */ #undef PY_FORMAT_SIZE_T diff --git a/Misc/developers.txt b/Misc/developers.txt --- a/Misc/developers.txt +++ b/Misc/developers.txt @@ -159,7 +159,7 @@ for general contributions to Python. - Mark Dickinson was given SVN access on 6 January 2008 by Facundo - Batista for his work on mathemathics and number related issues. + Batista for his work on mathematics and number related issues. - Amaury Forgeot d'Arc was given SVN access on 9 November 2007 by MvL, for general contributions to Python. diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -1652,7 +1652,7 @@ if int(os.uname()[2].split('.')[0]) >= 8: # We're on Mac OS X 10.4 or later, the compiler should # support '-Wno-deprecated-declarations'. This will - # surpress deprecation warnings for the Carbon extensions, + # suppress deprecation warnings for the Carbon extensions, # these extensions wrap the Carbon APIs and even those # parts that are deprecated. carbon_extra_compile_args = ['-Wno-deprecated-declarations'] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 04:28:13 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 20 Aug 2016 08:28:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_more_typos?= Message-ID: <20160820082813.130269.50907.B6A15354@psf.io> https://hg.python.org/cpython/rev/90bb32c9921c changeset: 102804:90bb32c9921c user: Martin Panter date: Sat Aug 20 08:27:06 2016 +0000 summary: Fix more typos files: Misc/NEWS | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -107,7 +107,7 @@ - Issue #25628: The *verbose* and *rename* parameters for collections.namedtuple are now keyword-only. -- Issue #12345: Add mathemathical constant tau to math and cmath. See also +- Issue #12345: Add mathematical constant tau to math and cmath. See also PEP 628. - Issue #26823: traceback.StackSummary.format now abbreviates large sections of @@ -182,7 +182,7 @@ - Issue #27522: Avoid an unintentional reference cycle in email.feedparser. -- Issue #27512: Fix a segfault when os.fspath() called a an __fspath__() method +- Issue #27512: Fix a segfault when os.fspath() called an __fspath__() method that raised an exception. Patch by Xiang Zhang. Tests -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 04:28:12 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 20 Aug 2016 08:28:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_spelling_fixes_from_3=2E5?= Message-ID: <20160820082812.30551.1399.F8EB8BBD@psf.io> https://hg.python.org/cpython/rev/578b97af256b changeset: 102803:578b97af256b parent: 102800:1455851e7332 parent: 102802:a5b8af72e80c user: Martin Panter date: Sat Aug 20 08:26:16 2016 +0000 summary: Merge spelling fixes from 3.5 files: Doc/library/asyncio-protocol.rst | 2 +- Doc/library/email.contentmanager.rst | 2 +- Doc/library/xml.dom.pulldom.rst | 2 +- Doc/whatsnew/2.7.rst | 2 +- Include/pymacconfig.h | 4 ++-- Misc/NEWS | 2 +- 6 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -11,7 +11,7 @@ Transports are classes provided by :mod:`asyncio` in order to abstract various kinds of communication channels. You generally won't instantiate -a transport yourself; instead, you will call a :class:`AbstractEventLoop` method +a transport yourself; instead, you will call an :class:`AbstractEventLoop` method which will create the transport and try to initiate the underlying communication channel, calling you back when it succeeds. diff --git a/Doc/library/email.contentmanager.rst b/Doc/library/email.contentmanager.rst --- a/Doc/library/email.contentmanager.rst +++ b/Doc/library/email.contentmanager.rst @@ -347,7 +347,7 @@ .. method:: get_content(msg, errors='replace') - Return the payload of the part as either a string (for ``text`` parts), a + Return the payload of the part as either a string (for ``text`` parts), an :class:`~email.message.EmailMessage` object (for ``message/rfc822`` parts), or a ``bytes`` object (for all other non-multipart types). Raise a :exc:`KeyError` if called on a ``multipart``. If the part is a diff --git a/Doc/library/xml.dom.pulldom.rst b/Doc/library/xml.dom.pulldom.rst --- a/Doc/library/xml.dom.pulldom.rst +++ b/Doc/library/xml.dom.pulldom.rst @@ -74,7 +74,7 @@ .. function:: parse(stream_or_string, parser=None, bufsize=None) Return a :class:`DOMEventStream` from the given input. *stream_or_string* may be - either a file name, or a file-like object. *parser*, if given, must be a + either a file name, or a file-like object. *parser*, if given, must be an :class:`~xml.sax.xmlreader.XMLReader` object. This function will change the document handler of the parser and activate namespace support; other parser configuration (like diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -532,7 +532,7 @@ calling ``log.getChild('network.listen')`` is equivalent to ``getLogger('app.network.listen')``. -* The :class:`~logging.LoggerAdapter` class gained a +* The :class:`~logging.LoggerAdapter` class gained an :meth:`~logging.LoggerAdapter.isEnabledFor` method that takes a *level* and returns whether the underlying logger would process a message of that level of importance. diff --git a/Include/pymacconfig.h b/Include/pymacconfig.h --- a/Include/pymacconfig.h +++ b/Include/pymacconfig.h @@ -63,7 +63,7 @@ # if defined(__LP64__) /* MacOSX 10.4 (the first release to support 64-bit code * at all) only supports 64-bit in the UNIX layer. - * Therefore surpress the toolbox-glue in 64-bit mode. + * Therefore suppress the toolbox-glue in 64-bit mode. */ /* In 64-bit mode setpgrp always has no arguments, in 32-bit @@ -91,7 +91,7 @@ * * Specifically: OSX 10.4 has limited supported for '%zd', while * 10.5 has full support for '%zd'. A binary built on 10.5 won't - * work properly on 10.4 unless we surpress the definition + * work properly on 10.4 unless we suppress the definition * of PY_FORMAT_SIZE_T */ #undef PY_FORMAT_SIZE_T diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -215,7 +215,7 @@ Build ----- -- Issue #27713: Surpress spurious build warnings when updating importlib's +- Issue #27713: Suppress spurious build warnings when updating importlib's bootstrap files. Patch by Xiang Zhang - Issue #25825: Correct the references to Modules/python.exp, which is -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 10:19:52 2016 From: python-checkins at python.org (ethan.furman) Date: Sat, 20 Aug 2016 14:19:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_issue26981=3A_add_=5Forder?= =?utf-8?q?=5F_compatibility_shim_to_enum=2EEnum?= Message-ID: <20160820141952.100688.63482.BB187061@psf.io> https://hg.python.org/cpython/rev/10830e1b2966 changeset: 102805:10830e1b2966 user: Ethan Furman date: Sat Aug 20 07:19:31 2016 -0700 summary: issue26981: add _order_ compatibility shim to enum.Enum files: Doc/library/enum.rst | 21 ++++++++++- Lib/enum.py | 17 +++++++- Lib/test/test_enum.py | 62 +++++++++++++++++++++++++++++++ Misc/NEWS | 3 + 4 files changed, 100 insertions(+), 3 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -257,7 +257,7 @@ >>> Color.red < Color.blue Traceback (most recent call last): File "", line 1, in - TypeError: unorderable types: Color() < Color() + TypeError: '<' not supported between instances of 'Color' and 'Color' Equality comparisons are defined though:: @@ -776,3 +776,22 @@ If you wish to change how :class:`Enum` members are looked up you should either write a helper function or a :func:`classmethod` for the :class:`Enum` subclass. + +To help keep Python 2 / Python 3 code in sync a user-specified :attr:`_order_`, +if provided, will be checked to ensure the actual order of the enumeration +matches:: + + >>> class Color(Enum): + ... _order_ = 'red green blue' + ... red = 1 + ... blue = 3 + ... green = 2 + ... + Traceback (most recent call last): + ... + TypeError: member order does not match _order_ + +.. note:: + + In Python 2 code the :attr:`_order_` attribute is necessary as definition + order is lost during class creation. diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -64,9 +64,11 @@ """ if _is_sunder(key): - raise ValueError('_names_ are reserved for future Enum use') + if key not in ('_order_', ): + raise ValueError('_names_ are reserved for future Enum use') elif _is_dunder(key): - pass + if key == '__order__': + key = '_order_' elif key in self._member_names: # descriptor overwriting an enum? raise TypeError('Attempted to reuse key: %r' % key) @@ -106,6 +108,9 @@ for name in classdict._member_names: del classdict[name] + # adjust the sunders + _order_ = classdict.pop('_order_', None) + # check for illegal enum names (any others?) invalid_names = set(members) & {'mro', } if invalid_names: @@ -210,6 +215,14 @@ if save_new: enum_class.__new_member__ = __new__ enum_class.__new__ = Enum.__new__ + + # py3 support for definition order (helps keep py2/py3 code in sync) + if _order_ is not None: + if isinstance(_order_, str): + _order_ = _order_.replace(',', ' ').split() + if _order_ != enum_class._member_names_: + raise TypeError('member order does not match _order_') + return enum_class def __bool__(self): diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -1571,6 +1571,68 @@ self.assertEqual(LabelledList(1), LabelledList.unprocessed) +class TestOrder(unittest.TestCase): + + def test_same_members(self): + class Color(Enum): + _order_ = 'red green blue' + red = 1 + green = 2 + blue = 3 + + def test_same_members_with_aliases(self): + class Color(Enum): + _order_ = 'red green blue' + red = 1 + green = 2 + blue = 3 + verde = green + + def test_same_members_wrong_order(self): + with self.assertRaisesRegex(TypeError, 'member order does not match _order_'): + class Color(Enum): + _order_ = 'red green blue' + red = 1 + blue = 3 + green = 2 + + def test_order_has_extra_members(self): + with self.assertRaisesRegex(TypeError, 'member order does not match _order_'): + class Color(Enum): + _order_ = 'red green blue purple' + red = 1 + green = 2 + blue = 3 + + def test_order_has_extra_members_with_aliases(self): + with self.assertRaisesRegex(TypeError, 'member order does not match _order_'): + class Color(Enum): + _order_ = 'red green blue purple' + red = 1 + green = 2 + blue = 3 + verde = green + + def test_enum_has_extra_members(self): + with self.assertRaisesRegex(TypeError, 'member order does not match _order_'): + class Color(Enum): + _order_ = 'red green blue' + red = 1 + green = 2 + blue = 3 + purple = 4 + + def test_enum_has_extra_members_with_aliases(self): + with self.assertRaisesRegex(TypeError, 'member order does not match _order_'): + class Color(Enum): + _order_ = 'red green blue' + red = 1 + green = 2 + blue = 3 + purple = 4 + verde = green + + class TestUnique(unittest.TestCase): def test_unique_clean(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -137,6 +137,9 @@ - Issue #26800: Undocumented support of general bytes-like objects as paths in os functions is now deprecated. +- Issue #26981: Add _order_ compatibility ship to enum.Enum for + Python 2/3 code bases. + - Issue #27661: Added tzinfo keyword argument to datetime.combine. - In the curses module, raise an error if window.getstr() or window.instr() is -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 11:56:55 2016 From: python-checkins at python.org (ethan.furman) Date: Sat, 20 Aug 2016 15:56:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_issue26981=3A_fix_typo?= Message-ID: <20160820155655.130057.63106.CC58BCCB@psf.io> https://hg.python.org/cpython/rev/bdfd4c8384de changeset: 102806:bdfd4c8384de user: Ethan Furman date: Sat Aug 20 08:56:40 2016 -0700 summary: issue26981: fix typo files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -137,7 +137,7 @@ - Issue #26800: Undocumented support of general bytes-like objects as paths in os functions is now deprecated. -- Issue #26981: Add _order_ compatibility ship to enum.Enum for +- Issue #26981: Add _order_ compatibility shim to enum.Enum for Python 2/3 code bases. - Issue #27661: Added tzinfo keyword argument to datetime.combine. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 14:22:46 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 20 Aug 2016 18:22:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327692=3A_Removed_?= =?utf-8?q?unnecessary_NULL_checks_in_exceptions=2Ec=2E?= Message-ID: <20160820182246.83204.79348.2BC4F118@psf.io> https://hg.python.org/cpython/rev/6e6aa2054824 changeset: 102807:6e6aa2054824 user: Serhiy Storchaka date: Sat Aug 20 21:22:03 2016 +0300 summary: Issue #27692: Removed unnecessary NULL checks in exceptions.c. Patch by Xiang Zhang. files: Objects/exceptions.c | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) diff --git a/Objects/exceptions.c b/Objects/exceptions.c --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -230,7 +230,7 @@ return -1; } - Py_XINCREF(tb); + Py_INCREF(tb); Py_XSETREF(self->traceback, tb); return 0; } @@ -985,7 +985,7 @@ return 0; error: - Py_XDECREF(args); + Py_DECREF(args); return -1; } @@ -1065,8 +1065,7 @@ } if (self->myerrno && self->strerror) return PyUnicode_FromFormat("[Errno %S] %S", - self->myerrno ? self->myerrno: Py_None, - self->strerror ? self->strerror: Py_None); + self->myerrno, self->strerror); return BaseException_str((PyBaseExceptionObject *)self); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 17:31:13 2016 From: python-checkins at python.org (jason.coombs) Date: Sat, 20 Aug 2016 21:31:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327819=3A_Simply_d?= =?utf-8?q?efault_to_gztar_for_sdist_formats_by_default_on_all?= Message-ID: <20160820213113.100638.56157.84C0A5C8@psf.io> https://hg.python.org/cpython/rev/8d09ec0ee934 changeset: 102808:8d09ec0ee934 user: Jason R. Coombs date: Sat Aug 20 17:31:07 2016 -0400 summary: Issue #27819: Simply default to gztar for sdist formats by default on all platforms. files: Lib/distutils/command/sdist.py | 12 +----------- Misc/NEWS | 3 +++ 2 files changed, 4 insertions(+), 11 deletions(-) diff --git a/Lib/distutils/command/sdist.py b/Lib/distutils/command/sdist.py --- a/Lib/distutils/command/sdist.py +++ b/Lib/distutils/command/sdist.py @@ -91,9 +91,6 @@ negative_opt = {'no-defaults': 'use-defaults', 'no-prune': 'prune' } - default_format = {'posix': 'gztar', - 'nt': 'zip' } - sub_commands = [('check', checking_metadata)] def initialize_options(self): @@ -110,7 +107,7 @@ self.manifest_only = 0 self.force_manifest = 0 - self.formats = None + self.formats = ['gztar'] self.keep_temp = 0 self.dist_dir = None @@ -126,13 +123,6 @@ self.template = "MANIFEST.in" self.ensure_string_list('formats') - if self.formats is None: - try: - self.formats = [self.default_format[os.name]] - except KeyError: - raise DistutilsPlatformError( - "don't know how to create source distributions " - "on platform %s" % os.name) bad_format = archive_util.check_archive_formats(self.formats) if bad_format: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -20,6 +20,9 @@ Library ------- +- Issue #27819: In distutils sdists, simply produce the "gztar" (gzipped tar + format) distributions on all platforms unless "formats" is supplied. + - Issue #2466: posixpath.ismount now correctly recognizes mount points which the user does not have permission to access. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 21:09:10 2016 From: python-checkins at python.org (terry.reedy) Date: Sun, 21 Aug 2016 01:09:10 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3NzE0?= =?utf-8?q?=3A_Remove_unneeded_non-idempotent_call_that_fails_on_retest=2E?= Message-ID: <20160821010910.13921.818.6A3260BC@psf.io> https://hg.python.org/cpython/rev/1c4cb01723a0 changeset: 102809:1c4cb01723a0 branch: 2.7 parent: 102801:03dc2f3fa0c6 user: Terry Jan Reedy date: Sat Aug 20 21:07:32 2016 -0400 summary: Issue #27714: Remove unneeded non-idempotent call that fails on retest. files: Lib/idlelib/idle_test/test_autocomplete.py | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/idle_test/test_autocomplete.py b/Lib/idlelib/idle_test/test_autocomplete.py --- a/Lib/idlelib/idle_test/test_autocomplete.py +++ b/Lib/idlelib/idle_test/test_autocomplete.py @@ -4,7 +4,6 @@ import idlelib.AutoComplete as ac import idlelib.AutoCompleteWindow as acw -import idlelib.macosxSupport as mac from idlelib.idle_test.mock_idle import Func from idlelib.idle_test.mock_tk import Event @@ -27,7 +26,6 @@ def setUpClass(cls): requires('gui') cls.root = Tk() - mac.setupApp(cls.root, None) cls.text = Text(cls.root) cls.editor = DummyEditwin(cls.root, cls.text) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 21:09:11 2016 From: python-checkins at python.org (terry.reedy) Date: Sun, 21 Aug 2016 01:09:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327714=3A_Remove_unneeded_non-idempotent_call_th?= =?utf-8?q?at_fails_on_retest=2E?= Message-ID: <20160821010911.30572.32448.0540B8FA@psf.io> https://hg.python.org/cpython/rev/e5db20ad0328 changeset: 102811:e5db20ad0328 parent: 102808:8d09ec0ee934 parent: 102810:0e4bea7c8c00 user: Terry Jan Reedy date: Sat Aug 20 21:08:50 2016 -0400 summary: Issue #27714: Remove unneeded non-idempotent call that fails on retest. files: Lib/idlelib/idle_test/test_autocomplete.py | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/idle_test/test_autocomplete.py b/Lib/idlelib/idle_test/test_autocomplete.py --- a/Lib/idlelib/idle_test/test_autocomplete.py +++ b/Lib/idlelib/idle_test/test_autocomplete.py @@ -8,7 +8,6 @@ import idlelib.autocomplete as ac import idlelib.autocomplete_w as acw -from idlelib import macosx from idlelib.idle_test.mock_idle import Func from idlelib.idle_test.mock_tk import Event @@ -31,7 +30,6 @@ def setUpClass(cls): requires('gui') cls.root = Tk() - macosx.setupApp(cls.root, None) cls.text = Text(cls.root) cls.editor = DummyEditwin(cls.root, cls.text) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 21:09:11 2016 From: python-checkins at python.org (terry.reedy) Date: Sun, 21 Aug 2016 01:09:11 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NzE0?= =?utf-8?q?=3A_Remove_unneeded_non-idempotent_call_that_fails_on_retest=2E?= Message-ID: <20160821010911.921.57787.9F633D0A@psf.io> https://hg.python.org/cpython/rev/0e4bea7c8c00 changeset: 102810:0e4bea7c8c00 branch: 3.5 parent: 102802:a5b8af72e80c user: Terry Jan Reedy date: Sat Aug 20 21:07:38 2016 -0400 summary: Issue #27714: Remove unneeded non-idempotent call that fails on retest. files: Lib/idlelib/idle_test/test_autocomplete.py | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/idle_test/test_autocomplete.py b/Lib/idlelib/idle_test/test_autocomplete.py --- a/Lib/idlelib/idle_test/test_autocomplete.py +++ b/Lib/idlelib/idle_test/test_autocomplete.py @@ -4,7 +4,6 @@ import idlelib.AutoComplete as ac import idlelib.AutoCompleteWindow as acw -import idlelib.macosxSupport as mac from idlelib.idle_test.mock_idle import Func from idlelib.idle_test.mock_tk import Event @@ -27,7 +26,6 @@ def setUpClass(cls): requires('gui') cls.root = Tk() - mac.setupApp(cls.root, None) cls.text = Text(cls.root) cls.editor = DummyEditwin(cls.root, cls.text) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 21:43:18 2016 From: python-checkins at python.org (terry.reedy) Date: Sun, 21 Aug 2016 01:43:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogQWRkIDMuNS4zIElE?= =?utf-8?q?LE_NEWS_and_idlelib/NEWS=2Etxt_items=2E?= Message-ID: <20160821014318.89480.1741.D85A9776@psf.io> https://hg.python.org/cpython/rev/6c8dd4cb4ee7 changeset: 102813:6c8dd4cb4ee7 branch: 3.5 parent: 102810:0e4bea7c8c00 user: Terry Jan Reedy date: Sat Aug 20 21:41:03 2016 -0400 summary: Add 3.5.3 IDLE NEWS and idlelib/NEWS.txt items. files: Lib/idlelib/NEWS.txt | 9 +++++++++ Misc/NEWS | 9 +++++++++ 2 files changed, 18 insertions(+), 0 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,6 +2,15 @@ ========================= *Release date: 2017-01-01?* +- Issue #27714: text_textview and test_autocomplete now pass when re-run + in the same process. This occurs when test_idle fails when run with the + -w option but without -jn. Fix warning tom test_config. + +- Issue #25507: IDLE no longer runs buggy code because of its tkinter imports. + Users must include the same imports required to run directly in Python. + +- Issue #27452: add line counter and crc to IDLE configHandler test dump. + - Issue #27365: Allow non-ascii chars in IDLE NEWS.txt, for contributor names. - Issue #27245: IDLE: Cleanly delete custom themes and key bindings. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -140,6 +140,15 @@ IDLE ---- +- Issue #27714: text_textview and test_autocomplete now pass when re-run + in the same process. This occurs when test_idle fails when run with the + -w option but without -jn. Fix warning tom test_config. + +- Issue #25507: IDLE no longer runs buggy code because of its tkinter imports. + Users must include the same imports required to run directly in Python. + +- Issue #27452: add line counter and crc to IDLE configHandler test dump. + - Issue #27365: Allow non-ascii chars in IDLE NEWS.txt, for contributor names. - Issue #27245: IDLE: Cleanly delete custom themes and key bindings. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 21:43:18 2016 From: python-checkins at python.org (terry.reedy) Date: Sun, 21 Aug 2016 01:43:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Add_3=2E6=2E0a4_IDLE_NEWS_and_idlelib/NEWS=2Etxt_items?= =?utf-8?q?=2E?= Message-ID: <20160821014318.11003.11223.18580AB2@psf.io> https://hg.python.org/cpython/rev/42a10253d066 changeset: 102814:42a10253d066 parent: 102811:e5db20ad0328 parent: 102813:6c8dd4cb4ee7 user: Terry Jan Reedy date: Sat Aug 20 21:43:00 2016 -0400 summary: Add 3.6.0a4 IDLE NEWS and idlelib/NEWS.txt items. files: Lib/idlelib/NEWS.txt | 26 ++++++++++++++++++++++++++ Misc/NEWS | 29 +++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+), 0 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,6 +2,32 @@ =========================== *Release date: 2016-09-??* +- Issue #27714: text_textview and test_autocomplete now pass when re-run + in the same process. This occurs when test_idle fails when run with the + -w option but without -jn. Fix warning tom test_config. + +- Issue #27621: Put query response validation error messages in the query + box itself instead of a separate massagebox. Redo tests to match. + Add Mac OSX refinements. Original patch by Mark Roseman. + +- Issue #27620: Escape key now closes Query box as cancelled. + +- Issue #27609: IDLE: tab after initial whitespace should tab, not + autocomplete. This fixes problem with writing docstrings at lease + twice indented. + +- Issue #27609: Explicitly return None when there are also non-None + returns. In a few cases, reverse a condition and eliminate a return. + +- Issue #25507: IDLE no longer runs buggy code because of its tkinter imports. + Users must include the same imports required to run directly in Python. + +- Issue #27173: Add 'IDLE Modern Unix' to the built-in key sets. + Make the default key set depend on the platform. + Add tests for the changes to the config module. + +- Issue #27452: add line counter and crc to IDLE configHandler test dump. + - Issue #27477: IDLE search dialogs now use ttk widgets. - Issue #27173: Add 'IDLE Modern Unix' to the built-in key sets. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -191,6 +191,35 @@ - Issue #27512: Fix a segfault when os.fspath() called an __fspath__() method that raised an exception. Patch by Xiang Zhang. +IDLE +---- + +- Issue #27714: text_textview and test_autocomplete now pass when re-run + in the same process. This occurs when test_idle fails when run with the + -w option but without -jn. Fix warning tom test_config. + +- Issue #27621: Put query response validation error messages in the query + box itself instead of a separate massagebox. Redo tests to match. + Add Mac OSX refinements. Original patch by Mark Roseman. + +- Issue #27620: Escape key now closes Query box as cancelled. + +- Issue #27609: IDLE: tab after initial whitespace should tab, not + autocomplete. This fixes problem with writing docstrings at lease + twice indented. + +- Issue #27609: Explicitly return None when there are also non-None + returns. In a few cases, reverse a condition and eliminate a return. + +- Issue #25507: IDLE no longer runs buggy code because of its tkinter imports. + Users must include the same imports required to run directly in Python. + +- Issue #27173: Add 'IDLE Modern Unix' to the built-in key sets. + Make the default key set depend on the platform. + Add tests for the changes to the config module. + +- Issue #27452: add line counter and crc to IDLE configHandler test dump. + Tests ----- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Aug 20 21:43:17 2016 From: python-checkins at python.org (terry.reedy) Date: Sun, 21 Aug 2016 01:43:17 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogQWRkIDIuNy4xMyBJ?= =?utf-8?q?DLE_NEWS_and_idlelib/NEWS=2Etxt_items=2E?= Message-ID: <20160821014317.2398.84895.EF824674@psf.io> https://hg.python.org/cpython/rev/0e48480f48ad changeset: 102812:0e48480f48ad branch: 2.7 parent: 102809:1c4cb01723a0 user: Terry Jan Reedy date: Sat Aug 20 21:40:55 2016 -0400 summary: Add 2.7.13 IDLE NEWS and idlelib/NEWS.txt items. files: Lib/idlelib/NEWS.txt | 6 ++++++ Misc/NEWS | 6 ++++++ 2 files changed, 12 insertions(+), 0 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,6 +2,12 @@ ========================== *Release date: 2017-01-01?* +- Issue #27714: text_textview and test_autocomplete now pass when re-run + in the same process. This occurs when test_idle fails when run with the + -w option but without -jn. Fix warning tom test_config. + +- Issue #27452: add line counter and crc to IDLE configHandler test dump. + - Issue #27365: Allow non-ascii chars in IDLE NEWS.txt, for contributor names. - Issue #27245: IDLE: Cleanly delete custom themes and key bindings. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -82,6 +82,12 @@ IDLE ---- +- Issue #27714: text_textview and test_autocomplete now pass when re-run + in the same process. This occurs when test_idle fails when run with the + -w option but without -jn. Fix warning tom test_config. + +- Issue #27452: add line counter and crc to IDLE configHandler test dump. + - Issue #27365: Allow non-ascii chars in IDLE NEWS.txt, for contributor names. - Issue #27245: IDLE: Cleanly delete custom themes and key bindings. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 00:20:50 2016 From: python-checkins at python.org (terry.reedy) Date: Sun, 21 Aug 2016 04:20:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Fix_typo_in_IDLE_News=2E?= Message-ID: <20160821042050.11185.13411.DD850B16@psf.io> https://hg.python.org/cpython/rev/9c96e4b2e0ca changeset: 102817:9c96e4b2e0ca parent: 102814:42a10253d066 parent: 102816:37bc126f3473 user: Terry Jan Reedy date: Sun Aug 21 00:20:35 2016 -0400 summary: Fix typo in IDLE News. files: Lib/idlelib/NEWS.txt | 6 +++--- Misc/NEWS | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -4,16 +4,16 @@ - Issue #27714: text_textview and test_autocomplete now pass when re-run in the same process. This occurs when test_idle fails when run with the - -w option but without -jn. Fix warning tom test_config. + -w option but without -jn. Fix warning from test_config. - Issue #27621: Put query response validation error messages in the query - box itself instead of a separate massagebox. Redo tests to match. + box itself instead of in a separate massagebox. Redo tests to match. Add Mac OSX refinements. Original patch by Mark Roseman. - Issue #27620: Escape key now closes Query box as cancelled. - Issue #27609: IDLE: tab after initial whitespace should tab, not - autocomplete. This fixes problem with writing docstrings at lease + autocomplete. This fixes problem with writing docstrings at least twice indented. - Issue #27609: Explicitly return None when there are also non-None diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -196,16 +196,16 @@ - Issue #27714: text_textview and test_autocomplete now pass when re-run in the same process. This occurs when test_idle fails when run with the - -w option but without -jn. Fix warning tom test_config. + -w option but without -jn. Fix warning from test_config. - Issue #27621: Put query response validation error messages in the query - box itself instead of a separate massagebox. Redo tests to match. + box itself instead of in a separate massagebox. Redo tests to match. Add Mac OSX refinements. Original patch by Mark Roseman. - Issue #27620: Escape key now closes Query box as cancelled. - Issue #27609: IDLE: tab after initial whitespace should tab, not - autocomplete. This fixes problem with writing docstrings at lease + autocomplete. This fixes problem with writing docstrings at least twice indented. - Issue #27609: Explicitly return None when there are also non-None -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 00:20:50 2016 From: python-checkins at python.org (terry.reedy) Date: Sun, 21 Aug 2016 04:20:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_typo_in_ID?= =?utf-8?q?LE_News=2E?= Message-ID: <20160821042049.82978.92666.6CFB4779@psf.io> https://hg.python.org/cpython/rev/3e0f8acf786c changeset: 102815:3e0f8acf786c branch: 2.7 parent: 102812:0e48480f48ad user: Terry Jan Reedy date: Sun Aug 21 00:18:49 2016 -0400 summary: Fix typo in IDLE News. files: Lib/idlelib/NEWS.txt | 2 +- Misc/NEWS | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -4,7 +4,7 @@ - Issue #27714: text_textview and test_autocomplete now pass when re-run in the same process. This occurs when test_idle fails when run with the - -w option but without -jn. Fix warning tom test_config. + -w option but without -jn. Fix warning from test_config. - Issue #27452: add line counter and crc to IDLE configHandler test dump. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -84,7 +84,7 @@ - Issue #27714: text_textview and test_autocomplete now pass when re-run in the same process. This occurs when test_idle fails when run with the - -w option but without -jn. Fix warning tom test_config. + -w option but without -jn. Fix warning from test_config. - Issue #27452: add line counter and crc to IDLE configHandler test dump. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 00:20:50 2016 From: python-checkins at python.org (terry.reedy) Date: Sun, 21 Aug 2016 04:20:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_typo_in_ID?= =?utf-8?q?LE_News=2E?= Message-ID: <20160821042050.100495.40758.26504AF7@psf.io> https://hg.python.org/cpython/rev/37bc126f3473 changeset: 102816:37bc126f3473 branch: 3.5 parent: 102813:6c8dd4cb4ee7 user: Terry Jan Reedy date: Sun Aug 21 00:18:55 2016 -0400 summary: Fix typo in IDLE News. files: Lib/idlelib/NEWS.txt | 2 +- Misc/NEWS | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -4,7 +4,7 @@ - Issue #27714: text_textview and test_autocomplete now pass when re-run in the same process. This occurs when test_idle fails when run with the - -w option but without -jn. Fix warning tom test_config. + -w option but without -jn. Fix warning from test_config. - Issue #25507: IDLE no longer runs buggy code because of its tkinter imports. Users must include the same imports required to run directly in Python. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -142,7 +142,7 @@ - Issue #27714: text_textview and test_autocomplete now pass when re-run in the same process. This occurs when test_idle fails when run with the - -w option but without -jn. Fix warning tom test_config. + -w option but without -jn. Fix warning from test_config. - Issue #25507: IDLE no longer runs buggy code because of its tkinter imports. Users must include the same imports required to run directly in Python. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 01:37:04 2016 From: python-checkins at python.org (martin.panter) Date: Sun, 21 Aug 2016 05:37:04 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NDg3?= =?utf-8?q?=3A_Warn_if_submodule_already_imported_before_runpy_execution?= Message-ID: <20160821053704.29386.78735.22E357B7@psf.io> https://hg.python.org/cpython/rev/43ae044eaccc changeset: 102818:43ae044eaccc branch: 3.5 parent: 102816:37bc126f3473 user: Martin Panter date: Sun Aug 21 04:07:58 2016 +0000 summary: Issue #27487: Warn if submodule already imported before runpy execution Also try to clarify the find_spec() error message. files: Lib/runpy.py | 11 +++++- Lib/test/test_cmd_line_script.py | 8 ++- Lib/test/test_runpy.py | 40 ++++++++++++++++--- Misc/NEWS | 4 ++ 4 files changed, 52 insertions(+), 11 deletions(-) diff --git a/Lib/runpy.py b/Lib/runpy.py --- a/Lib/runpy.py +++ b/Lib/runpy.py @@ -114,6 +114,15 @@ if e.name is None or (e.name != pkg_name and not pkg_name.startswith(e.name + ".")): raise + # Warn if the module has already been imported under its normal name + existing = sys.modules.get(mod_name) + if existing is not None and not hasattr(existing, "__path__"): + from warnings import warn + msg = "{mod_name!r} found in sys.modules after import of " \ + "package {pkg_name!r}, but prior to execution of " \ + "{mod_name!r}; this may result in unpredictable " \ + "behaviour".format(mod_name=mod_name, pkg_name=pkg_name) + warn(RuntimeWarning(msg)) try: spec = importlib.util.find_spec(mod_name) @@ -121,7 +130,7 @@ # This hack fixes an impedance mismatch between pkgutil and # importlib, where the latter raises other errors for cases where # pkgutil previously raised ImportError - msg = "Error while finding spec for {!r} ({}: {})" + msg = "Error while finding module specification for {!r} ({}: {})" raise error(msg.format(mod_name, type(ex).__name__, ex)) from ex if spec is None: raise error("No module named %s" % mod_name) diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -426,8 +426,9 @@ # Exercise error reporting for various invalid package executions tests = ( ('builtins', br'No code object available'), - ('builtins.x', br'Error while finding spec.*AttributeError'), - ('builtins.x.y', br'Error while finding spec.*' + ('builtins.x', br'Error while finding module specification.*' + br'AttributeError'), + ('builtins.x.y', br'Error while finding module specification.*' br'ImportError.*No module named.*not a package'), ('os.path', br'loader.*cannot handle'), ('importlib', br'No module named.*' @@ -450,7 +451,8 @@ with open('test_pkg/__init__.pyc', 'wb'): pass err = self.check_dash_m_failure('test_pkg') - self.assertRegex(err, br'Error while finding spec.*' + self.assertRegex(err, + br'Error while finding module specification.*' br'ImportError.*bad magic number') self.assertNotIn(b'is a package', err) self.assertNotIn(b'Traceback', err) diff --git a/Lib/test/test_runpy.py b/Lib/test/test_runpy.py --- a/Lib/test/test_runpy.py +++ b/Lib/test/test_runpy.py @@ -7,6 +7,7 @@ import tempfile import importlib, importlib.machinery, importlib.util import py_compile +import warnings from test.support import ( forget, make_legacy_pyc, unload, verbose, no_tracing, create_empty_file, temp_dir) @@ -246,7 +247,7 @@ mod_fname) return pkg_dir, mod_fname, mod_name, mod_spec - def _del_pkg(self, top, depth, mod_name): + def _del_pkg(self, top): for entry in list(sys.modules): if entry.startswith("__runpy_pkg__"): del sys.modules[entry] @@ -320,7 +321,7 @@ self._fix_ns_for_legacy_pyc(expected_ns, alter_sys) self.check_code_execution(create_ns, expected_ns) finally: - self._del_pkg(pkg_dir, depth, mod_name) + self._del_pkg(pkg_dir) if verbose > 1: print("Module executed successfully") def _check_package(self, depth, alter_sys=False, @@ -361,7 +362,7 @@ self._fix_ns_for_legacy_pyc(expected_ns, alter_sys) self.check_code_execution(create_ns, expected_ns) finally: - self._del_pkg(pkg_dir, depth, pkg_name) + self._del_pkg(pkg_dir) if verbose > 1: print("Package executed successfully") def _add_relative_modules(self, base_dir, source, depth): @@ -424,7 +425,7 @@ self.assertIn("nephew", d2) del d2 # Ensure __loader__ entry doesn't keep file open finally: - self._del_pkg(pkg_dir, depth, mod_name) + self._del_pkg(pkg_dir) if verbose > 1: print("Module executed successfully") def test_run_module(self): @@ -447,7 +448,7 @@ result = self._make_pkg("", 1, "__main__") pkg_dir, _, mod_name, _ = result mod_name = mod_name.replace(".__main__", "") - self.addCleanup(self._del_pkg, pkg_dir, 1, mod_name) + self.addCleanup(self._del_pkg, pkg_dir) init = os.path.join(pkg_dir, "__runpy_pkg__", "__init__.py") exceptions = (ImportError, AttributeError, TypeError, ValueError) @@ -470,6 +471,31 @@ else: self.fail("Nothing raised; expected {}".format(name)) + def test_submodule_imported_warning(self): + pkg_dir, _, mod_name, _ = self._make_pkg("", 1) + try: + __import__(mod_name) + with self.assertWarnsRegex(RuntimeWarning, + r"found in sys\.modules"): + run_module(mod_name) + finally: + self._del_pkg(pkg_dir) + + def test_package_imported_no_warning(self): + pkg_dir, _, mod_name, _ = self._make_pkg("", 1, "__main__") + self.addCleanup(self._del_pkg, pkg_dir) + package = mod_name.replace(".__main__", "") + # No warning should occur if we only imported the parent package + __import__(package) + self.assertIn(package, sys.modules) + with warnings.catch_warnings(): + warnings.simplefilter("error", RuntimeWarning) + run_module(package) + # But the warning should occur if we imported the __main__ submodule + __import__(mod_name) + with self.assertWarnsRegex(RuntimeWarning, r"found in sys\.modules"): + run_module(package) + def test_run_package_in_namespace_package(self): for depth in range(1, 4): if verbose > 1: print("Testing package depth:", depth) @@ -524,7 +550,7 @@ try: self.check_code_execution(create_ns, expected_ns) finally: - self._del_pkg(pkg_dir, depth, mod_name) + self._del_pkg(pkg_dir) def test_pkgutil_walk_packages(self): # This is a dodgy hack to use the test_runpy infrastructure to test @@ -548,7 +574,7 @@ expected_modules.add(pkg_name + ".runpy_test") pkg_dir, mod_fname, mod_name, mod_spec = ( self._make_pkg("", max_depth)) - self.addCleanup(self._del_pkg, pkg_dir, max_depth, mod_name) + self.addCleanup(self._del_pkg, pkg_dir) for depth in range(2, max_depth+1): self._add_relative_modules(pkg_dir, "", depth) for finder, mod_name, ispkg in pkgutil.walk_packages([pkg_dir]): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #27487: Warn if a submodule argument to "python -m" or + runpy.run_module() is found in sys.modules after parent packages are + imported, but before the submodule is executed. + - Issue #27558: Fix a SystemError in the implementation of "raise" statement. In a brand new thread, raise a RuntimeError since there is no active exception to reraise. Patch written by Xiang Zhang. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 01:37:04 2016 From: python-checkins at python.org (martin.panter) Date: Sun, 21 Aug 2016 05:37:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327487=3A_Merge_runpy_warning_from_3=2E5?= Message-ID: <20160821053704.29291.67575.A4EB97C2@psf.io> https://hg.python.org/cpython/rev/fccd733aa78b changeset: 102819:fccd733aa78b parent: 102817:9c96e4b2e0ca parent: 102818:43ae044eaccc user: Martin Panter date: Sun Aug 21 05:07:35 2016 +0000 summary: Issue #27487: Merge runpy warning from 3.5 files: Lib/runpy.py | 11 +++++- Lib/test/test_cmd_line_script.py | 8 ++- Lib/test/test_runpy.py | 40 ++++++++++++++++--- Misc/NEWS | 4 ++ 4 files changed, 52 insertions(+), 11 deletions(-) diff --git a/Lib/runpy.py b/Lib/runpy.py --- a/Lib/runpy.py +++ b/Lib/runpy.py @@ -114,6 +114,15 @@ if e.name is None or (e.name != pkg_name and not pkg_name.startswith(e.name + ".")): raise + # Warn if the module has already been imported under its normal name + existing = sys.modules.get(mod_name) + if existing is not None and not hasattr(existing, "__path__"): + from warnings import warn + msg = "{mod_name!r} found in sys.modules after import of " \ + "package {pkg_name!r}, but prior to execution of " \ + "{mod_name!r}; this may result in unpredictable " \ + "behaviour".format(mod_name=mod_name, pkg_name=pkg_name) + warn(RuntimeWarning(msg)) try: spec = importlib.util.find_spec(mod_name) @@ -121,7 +130,7 @@ # This hack fixes an impedance mismatch between pkgutil and # importlib, where the latter raises other errors for cases where # pkgutil previously raised ImportError - msg = "Error while finding spec for {!r} ({}: {})" + msg = "Error while finding module specification for {!r} ({}: {})" raise error(msg.format(mod_name, type(ex).__name__, ex)) from ex if spec is None: raise error("No module named %s" % mod_name) diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -425,8 +425,9 @@ # Exercise error reporting for various invalid package executions tests = ( ('builtins', br'No code object available'), - ('builtins.x', br'Error while finding spec.*AttributeError'), - ('builtins.x.y', br'Error while finding spec.*' + ('builtins.x', br'Error while finding module specification.*' + br'AttributeError'), + ('builtins.x.y', br'Error while finding module specification.*' br'ImportError.*No module named.*not a package'), ('os.path', br'loader.*cannot handle'), ('importlib', br'No module named.*' @@ -449,7 +450,8 @@ with open('test_pkg/__init__.pyc', 'wb'): pass err = self.check_dash_m_failure('test_pkg') - self.assertRegex(err, br'Error while finding spec.*' + self.assertRegex(err, + br'Error while finding module specification.*' br'ImportError.*bad magic number') self.assertNotIn(b'is a package', err) self.assertNotIn(b'Traceback', err) diff --git a/Lib/test/test_runpy.py b/Lib/test/test_runpy.py --- a/Lib/test/test_runpy.py +++ b/Lib/test/test_runpy.py @@ -7,6 +7,7 @@ import tempfile import importlib, importlib.machinery, importlib.util import py_compile +import warnings from test.support import ( forget, make_legacy_pyc, unload, verbose, no_tracing, create_empty_file, temp_dir) @@ -246,7 +247,7 @@ mod_fname) return pkg_dir, mod_fname, mod_name, mod_spec - def _del_pkg(self, top, depth, mod_name): + def _del_pkg(self, top): for entry in list(sys.modules): if entry.startswith("__runpy_pkg__"): del sys.modules[entry] @@ -320,7 +321,7 @@ self._fix_ns_for_legacy_pyc(expected_ns, alter_sys) self.check_code_execution(create_ns, expected_ns) finally: - self._del_pkg(pkg_dir, depth, mod_name) + self._del_pkg(pkg_dir) if verbose > 1: print("Module executed successfully") def _check_package(self, depth, alter_sys=False, @@ -361,7 +362,7 @@ self._fix_ns_for_legacy_pyc(expected_ns, alter_sys) self.check_code_execution(create_ns, expected_ns) finally: - self._del_pkg(pkg_dir, depth, pkg_name) + self._del_pkg(pkg_dir) if verbose > 1: print("Package executed successfully") def _add_relative_modules(self, base_dir, source, depth): @@ -424,7 +425,7 @@ self.assertIn("nephew", d2) del d2 # Ensure __loader__ entry doesn't keep file open finally: - self._del_pkg(pkg_dir, depth, mod_name) + self._del_pkg(pkg_dir) if verbose > 1: print("Module executed successfully") def test_run_module(self): @@ -447,7 +448,7 @@ result = self._make_pkg("", 1, "__main__") pkg_dir, _, mod_name, _ = result mod_name = mod_name.replace(".__main__", "") - self.addCleanup(self._del_pkg, pkg_dir, 1, mod_name) + self.addCleanup(self._del_pkg, pkg_dir) init = os.path.join(pkg_dir, "__runpy_pkg__", "__init__.py") exceptions = (ImportError, AttributeError, TypeError, ValueError) @@ -470,6 +471,31 @@ else: self.fail("Nothing raised; expected {}".format(name)) + def test_submodule_imported_warning(self): + pkg_dir, _, mod_name, _ = self._make_pkg("", 1) + try: + __import__(mod_name) + with self.assertWarnsRegex(RuntimeWarning, + r"found in sys\.modules"): + run_module(mod_name) + finally: + self._del_pkg(pkg_dir) + + def test_package_imported_no_warning(self): + pkg_dir, _, mod_name, _ = self._make_pkg("", 1, "__main__") + self.addCleanup(self._del_pkg, pkg_dir) + package = mod_name.replace(".__main__", "") + # No warning should occur if we only imported the parent package + __import__(package) + self.assertIn(package, sys.modules) + with warnings.catch_warnings(): + warnings.simplefilter("error", RuntimeWarning) + run_module(package) + # But the warning should occur if we imported the __main__ submodule + __import__(mod_name) + with self.assertWarnsRegex(RuntimeWarning, r"found in sys\.modules"): + run_module(package) + def test_run_package_in_namespace_package(self): for depth in range(1, 4): if verbose > 1: print("Testing package depth:", depth) @@ -524,7 +550,7 @@ try: self.check_code_execution(create_ns, expected_ns) finally: - self._del_pkg(pkg_dir, depth, mod_name) + self._del_pkg(pkg_dir) def test_pkgutil_walk_packages(self): # This is a dodgy hack to use the test_runpy infrastructure to test @@ -548,7 +574,7 @@ expected_modules.add(pkg_name + ".runpy_test") pkg_dir, mod_fname, mod_name, mod_spec = ( self._make_pkg("", max_depth)) - self.addCleanup(self._del_pkg, pkg_dir, max_depth, mod_name) + self.addCleanup(self._del_pkg, pkg_dir) for depth in range(2, max_depth+1): self._add_relative_modules(pkg_dir, "", depth) for finder, mod_name, ispkg in pkgutil.walk_packages([pkg_dir]): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #27487: Warn if a submodule argument to "python -m" or + runpy.run_module() is found in sys.modules after parent packages are + imported, but before the submodule is executed. + - Issue #27157: Make only type() itself accept the one-argument form. Patch by Eryk Sun and Emanuel Barry. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 03:44:11 2016 From: python-checkins at python.org (nick.coghlan) Date: Sun, 21 Aug 2016 07:44:11 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3Nzgy?= =?utf-8?q?=3A_Fix_m=5Fmethods_handling_in_multiphase_init?= Message-ID: <20160821074410.10961.77984.4179B621@psf.io> https://hg.python.org/cpython/rev/913268337886 changeset: 102820:913268337886 branch: 3.5 parent: 102818:43ae044eaccc user: Nick Coghlan date: Sun Aug 21 17:41:56 2016 +1000 summary: Issue #27782: Fix m_methods handling in multiphase init Multi-phase extension module import now correctly allows the ``m_methods`` field to be used to add module level functions to instances of non-module types returned from ``Py_create_mod``. Patch by Xiang Zhang. files: Doc/c-api/module.rst | 2 +- Include/moduleobject.h | 2 +- Lib/test/test_importlib/extension/test_loader.py | 9 + Misc/ACKS | 1 + Misc/NEWS | 4 + Modules/_testmultiphase.c | 33 +++++- Objects/moduleobject.c | 64 +++++---- 7 files changed, 83 insertions(+), 32 deletions(-) diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -324,7 +324,7 @@ :c:type:`PyModule_Type`. Any type can be used, as long as it supports setting and getting import-related attributes. However, only ``PyModule_Type`` instances may be returned if the - ``PyModuleDef`` has non-*NULL* ``m_methods``, ``m_traverse``, ``m_clear``, + ``PyModuleDef`` has non-*NULL* ``m_traverse``, ``m_clear``, ``m_free``; non-zero ``m_size``; or slots other than ``Py_mod_create``. .. c:var:: Py_mod_exec diff --git a/Include/moduleobject.h b/Include/moduleobject.h --- a/Include/moduleobject.h +++ b/Include/moduleobject.h @@ -77,7 +77,7 @@ traverseproc m_traverse; inquiry m_clear; freefunc m_free; -}PyModuleDef; +} PyModuleDef; #ifdef __cplusplus } diff --git a/Lib/test/test_importlib/extension/test_loader.py b/Lib/test/test_importlib/extension/test_loader.py --- a/Lib/test/test_importlib/extension/test_loader.py +++ b/Lib/test/test_importlib/extension/test_loader.py @@ -212,6 +212,15 @@ self.assertNotEqual(type(mod), type(unittest)) self.assertEqual(mod.three, 3) + # issue 27782 + def test_nonmodule_with_methods(self): + '''Test creating a non-module object with methods defined''' + name = self.name + '_nonmodule_with_methods' + mod = self.load_module_by_name(name) + self.assertNotEqual(type(mod), type(unittest)) + self.assertEqual(mod.three, 3) + self.assertEqual(mod.bar(10, 1), 9) + def test_null_slots(self): '''Test that NULL slots aren't a problem''' name = self.name + '_null_slots' diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1657,6 +1657,7 @@ Yuxiao Zeng Uwe Zessin Cheng Zhang +Xiang Zhang Kai Zhu Tarek Ziad? Jelle Zijlstra diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #27782: Multi-phase extension module import now correctly allows the + ``m_methods`` field to be used to add module level functions to instances + of non-module types returned from ``Py_create_mod``. Patch by Xiang Zhang. + - Issue #27487: Warn if a submodule argument to "python -m" or runpy.run_module() is found in sys.modules after parent packages are imported, but before the submodule is executed. diff --git a/Modules/_testmultiphase.c b/Modules/_testmultiphase.c --- a/Modules/_testmultiphase.c +++ b/Modules/_testmultiphase.c @@ -248,6 +248,7 @@ /**** Importing a non-module object ****/ static PyModuleDef def_nonmodule; +static PyModuleDef def_nonmodule_with_methods; /* Create a SimpleNamespace(three=3) */ static PyObject* @@ -255,7 +256,7 @@ { PyObject *dct, *ns, *three; - if (def != &def_nonmodule) { + if (def != &def_nonmodule && def != &def_nonmodule_with_methods) { PyErr_SetString(PyExc_SystemError, "def does not match"); return NULL; } @@ -291,6 +292,36 @@ return PyModuleDef_Init(&def_nonmodule); } +PyDoc_STRVAR(nonmodule_bar_doc, +"bar(i,j)\n\ +\n\ +Return the difference of i - j."); + +static PyObject * +nonmodule_bar(PyObject *self, PyObject *args) +{ + long i, j; + long res; + if (!PyArg_ParseTuple(args, "ll:bar", &i, &j)) + return NULL; + res = i - j; + return PyLong_FromLong(res); +} + +static PyMethodDef nonmodule_methods[] = { + {"bar", nonmodule_bar, METH_VARARGS, nonmodule_bar_doc}, + {NULL, NULL} /* sentinel */ +}; + +static PyModuleDef def_nonmodule_with_methods = TEST_MODULE_DEF( + "_testmultiphase_nonmodule_with_methods", slots_create_nonmodule, nonmodule_methods); + +PyMODINIT_FUNC +PyInit__testmultiphase_nonmodule_with_methods(PyObject *spec) +{ + return PyModuleDef_Init(&def_nonmodule_with_methods); +} + /**** Non-ASCII-named modules ****/ static PyModuleDef def_nonascii_latin = { \ diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -130,6 +130,34 @@ return 1; } +static int +_add_methods_to_object(PyObject *module, PyObject *name, PyMethodDef *functions) +{ + PyObject *func; + PyMethodDef *fdef; + + for (fdef = functions; fdef->ml_name != NULL; fdef++) { + if ((fdef->ml_flags & METH_CLASS) || + (fdef->ml_flags & METH_STATIC)) { + PyErr_SetString(PyExc_ValueError, + "module functions cannot set" + " METH_CLASS or METH_STATIC"); + return -1; + } + func = PyCFunction_NewEx(fdef, (PyObject*)module, name); + if (func == NULL) { + return -1; + } + if (PyObject_SetAttrString(module, fdef->ml_name, func) != 0) { + Py_DECREF(func); + return -1; + } + Py_DECREF(func); + } + + return 0; +} + PyObject * PyModule_Create2(struct PyModuleDef* module, int module_api_version) { @@ -269,7 +297,7 @@ } } } else { - m = PyModule_New(name); + m = PyModule_NewObject(nameobj); if (m == NULL) { goto error; } @@ -297,7 +325,7 @@ } if (def->m_methods != NULL) { - ret = PyModule_AddFunctions(m, def->m_methods); + ret = _add_methods_to_object(m, nameobj, def->m_methods); if (ret != 0) { goto error; } @@ -331,7 +359,7 @@ return -1; } - if (PyModule_Check(module) && def->m_size >= 0) { + if (def->m_size >= 0) { PyModuleObject *md = (PyModuleObject*)module; if (md->md_state == NULL) { /* Always set a state pointer; this serves as a marker to skip @@ -387,37 +415,15 @@ int PyModule_AddFunctions(PyObject *m, PyMethodDef *functions) { - PyObject *name, *func; - PyMethodDef *fdef; - - name = PyModule_GetNameObject(m); + int res; + PyObject *name = PyModule_GetNameObject(m); if (name == NULL) { return -1; } - for (fdef = functions; fdef->ml_name != NULL; fdef++) { - if ((fdef->ml_flags & METH_CLASS) || - (fdef->ml_flags & METH_STATIC)) { - PyErr_SetString(PyExc_ValueError, - "module functions cannot set" - " METH_CLASS or METH_STATIC"); - Py_DECREF(name); - return -1; - } - func = PyCFunction_NewEx(fdef, (PyObject*)m, name); - if (func == NULL) { - Py_DECREF(name); - return -1; - } - if (PyObject_SetAttrString(m, fdef->ml_name, func) != 0) { - Py_DECREF(func); - Py_DECREF(name); - return -1; - } - Py_DECREF(func); - } + res = _add_methods_to_object(m, name, functions); Py_DECREF(name); - return 0; + return res; } int -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 03:44:12 2016 From: python-checkins at python.org (nick.coghlan) Date: Sun, 21 Aug 2016 07:44:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_=2327782_fix_from_3=2E5?= Message-ID: <20160821074411.797.14502.EA55AE71@psf.io> https://hg.python.org/cpython/rev/fb509792dffc changeset: 102821:fb509792dffc parent: 102819:fccd733aa78b parent: 102820:913268337886 user: Nick Coghlan date: Sun Aug 21 17:43:58 2016 +1000 summary: Merge #27782 fix from 3.5 files: Doc/c-api/module.rst | 2 +- Include/moduleobject.h | 2 +- Lib/test/test_importlib/extension/test_loader.py | 9 + Misc/ACKS | 1 + Misc/NEWS | 4 + Modules/_testmultiphase.c | 33 +++++- Objects/moduleobject.c | 64 +++++---- 7 files changed, 83 insertions(+), 32 deletions(-) diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -324,7 +324,7 @@ :c:type:`PyModule_Type`. Any type can be used, as long as it supports setting and getting import-related attributes. However, only ``PyModule_Type`` instances may be returned if the - ``PyModuleDef`` has non-*NULL* ``m_methods``, ``m_traverse``, ``m_clear``, + ``PyModuleDef`` has non-*NULL* ``m_traverse``, ``m_clear``, ``m_free``; non-zero ``m_size``; or slots other than ``Py_mod_create``. .. c:var:: Py_mod_exec diff --git a/Include/moduleobject.h b/Include/moduleobject.h --- a/Include/moduleobject.h +++ b/Include/moduleobject.h @@ -77,7 +77,7 @@ traverseproc m_traverse; inquiry m_clear; freefunc m_free; -}PyModuleDef; +} PyModuleDef; #ifdef __cplusplus } diff --git a/Lib/test/test_importlib/extension/test_loader.py b/Lib/test/test_importlib/extension/test_loader.py --- a/Lib/test/test_importlib/extension/test_loader.py +++ b/Lib/test/test_importlib/extension/test_loader.py @@ -212,6 +212,15 @@ self.assertNotEqual(type(mod), type(unittest)) self.assertEqual(mod.three, 3) + # issue 27782 + def test_nonmodule_with_methods(self): + '''Test creating a non-module object with methods defined''' + name = self.name + '_nonmodule_with_methods' + mod = self.load_module_by_name(name) + self.assertNotEqual(type(mod), type(unittest)) + self.assertEqual(mod.three, 3) + self.assertEqual(mod.bar(10, 1), 9) + def test_null_slots(self): '''Test that NULL slots aren't a problem''' name = self.name + '_null_slots' diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1679,6 +1679,7 @@ Yuxiao Zeng Uwe Zessin Cheng Zhang +Xiang Zhang Kai Zhu Tarek Ziad? Jelle Zijlstra diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #27782: Multi-phase extension module import now correctly allows the + ``m_methods`` field to be used to add module level functions to instances + of non-module types returned from ``Py_create_mod``. Patch by Xiang Zhang. + - Issue #27487: Warn if a submodule argument to "python -m" or runpy.run_module() is found in sys.modules after parent packages are imported, but before the submodule is executed. diff --git a/Modules/_testmultiphase.c b/Modules/_testmultiphase.c --- a/Modules/_testmultiphase.c +++ b/Modules/_testmultiphase.c @@ -248,6 +248,7 @@ /**** Importing a non-module object ****/ static PyModuleDef def_nonmodule; +static PyModuleDef def_nonmodule_with_methods; /* Create a SimpleNamespace(three=3) */ static PyObject* @@ -255,7 +256,7 @@ { PyObject *dct, *ns, *three; - if (def != &def_nonmodule) { + if (def != &def_nonmodule && def != &def_nonmodule_with_methods) { PyErr_SetString(PyExc_SystemError, "def does not match"); return NULL; } @@ -291,6 +292,36 @@ return PyModuleDef_Init(&def_nonmodule); } +PyDoc_STRVAR(nonmodule_bar_doc, +"bar(i,j)\n\ +\n\ +Return the difference of i - j."); + +static PyObject * +nonmodule_bar(PyObject *self, PyObject *args) +{ + long i, j; + long res; + if (!PyArg_ParseTuple(args, "ll:bar", &i, &j)) + return NULL; + res = i - j; + return PyLong_FromLong(res); +} + +static PyMethodDef nonmodule_methods[] = { + {"bar", nonmodule_bar, METH_VARARGS, nonmodule_bar_doc}, + {NULL, NULL} /* sentinel */ +}; + +static PyModuleDef def_nonmodule_with_methods = TEST_MODULE_DEF( + "_testmultiphase_nonmodule_with_methods", slots_create_nonmodule, nonmodule_methods); + +PyMODINIT_FUNC +PyInit__testmultiphase_nonmodule_with_methods(PyObject *spec) +{ + return PyModuleDef_Init(&def_nonmodule_with_methods); +} + /**** Non-ASCII-named modules ****/ static PyModuleDef def_nonascii_latin = { \ diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -130,6 +130,34 @@ return 1; } +static int +_add_methods_to_object(PyObject *module, PyObject *name, PyMethodDef *functions) +{ + PyObject *func; + PyMethodDef *fdef; + + for (fdef = functions; fdef->ml_name != NULL; fdef++) { + if ((fdef->ml_flags & METH_CLASS) || + (fdef->ml_flags & METH_STATIC)) { + PyErr_SetString(PyExc_ValueError, + "module functions cannot set" + " METH_CLASS or METH_STATIC"); + return -1; + } + func = PyCFunction_NewEx(fdef, (PyObject*)module, name); + if (func == NULL) { + return -1; + } + if (PyObject_SetAttrString(module, fdef->ml_name, func) != 0) { + Py_DECREF(func); + return -1; + } + Py_DECREF(func); + } + + return 0; +} + PyObject * PyModule_Create2(struct PyModuleDef* module, int module_api_version) { @@ -269,7 +297,7 @@ } } } else { - m = PyModule_New(name); + m = PyModule_NewObject(nameobj); if (m == NULL) { goto error; } @@ -297,7 +325,7 @@ } if (def->m_methods != NULL) { - ret = PyModule_AddFunctions(m, def->m_methods); + ret = _add_methods_to_object(m, nameobj, def->m_methods); if (ret != 0) { goto error; } @@ -331,7 +359,7 @@ return -1; } - if (PyModule_Check(module) && def->m_size >= 0) { + if (def->m_size >= 0) { PyModuleObject *md = (PyModuleObject*)module; if (md->md_state == NULL) { /* Always set a state pointer; this serves as a marker to skip @@ -387,37 +415,15 @@ int PyModule_AddFunctions(PyObject *m, PyMethodDef *functions) { - PyObject *name, *func; - PyMethodDef *fdef; - - name = PyModule_GetNameObject(m); + int res; + PyObject *name = PyModule_GetNameObject(m); if (name == NULL) { return -1; } - for (fdef = functions; fdef->ml_name != NULL; fdef++) { - if ((fdef->ml_flags & METH_CLASS) || - (fdef->ml_flags & METH_STATIC)) { - PyErr_SetString(PyExc_ValueError, - "module functions cannot set" - " METH_CLASS or METH_STATIC"); - Py_DECREF(name); - return -1; - } - func = PyCFunction_NewEx(fdef, (PyObject*)m, name); - if (func == NULL) { - Py_DECREF(name); - return -1; - } - if (PyObject_SetAttrString(m, fdef->ml_name, func) != 0) { - Py_DECREF(func); - Py_DECREF(name); - return -1; - } - Py_DECREF(func); - } + res = _add_methods_to_object(m, name, functions); Py_DECREF(name); - return 0; + return res; } int -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 03:55:40 2016 From: python-checkins at python.org (mark.dickinson) Date: Sun, 21 Aug 2016 07:55:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327662=3A_don=27t_?= =?utf-8?q?use_PY=5FSIZE=5FMAX_for_overflow_checking_in_List=5FNew=2E_Patc?= =?utf-8?q?h_by?= Message-ID: <20160821075524.20738.54666.B4DCD68B@psf.io> https://hg.python.org/cpython/rev/cd3d079ad2b5 changeset: 102822:cd3d079ad2b5 user: Mark Dickinson date: Sun Aug 21 08:55:15 2016 +0100 summary: Issue #27662: don't use PY_SIZE_MAX for overflow checking in List_New. Patch by Xiang Zhang. files: Objects/listobject.c | 12 +----------- 1 files changed, 1 insertions(+), 11 deletions(-) diff --git a/Objects/listobject.c b/Objects/listobject.c --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -140,7 +140,6 @@ PyList_New(Py_ssize_t size) { PyListObject *op; - size_t nbytes; #ifdef SHOW_ALLOC_COUNT static int initialized = 0; if (!initialized) { @@ -153,11 +152,6 @@ PyErr_BadInternalCall(); return NULL; } - /* Check for overflow without an actual overflow, - * which can cause compiler to optimise out */ - if ((size_t)size > PY_SIZE_MAX / sizeof(PyObject *)) - return PyErr_NoMemory(); - nbytes = size * sizeof(PyObject *); if (numfree) { numfree--; op = free_list[numfree]; @@ -176,12 +170,11 @@ if (size <= 0) op->ob_item = NULL; else { - op->ob_item = (PyObject **) PyMem_MALLOC(nbytes); + op->ob_item = (PyObject **) PyMem_Calloc(size, sizeof(PyObject *)); if (op->ob_item == NULL) { Py_DECREF(op); return PyErr_NoMemory(); } - memset(op->ob_item, 0, nbytes); } Py_SIZE(op) = size; op->allocated = size; @@ -2503,9 +2496,6 @@ step = -step; } - assert((size_t)slicelength <= - PY_SIZE_MAX / sizeof(PyObject*)); - garbage = (PyObject**) PyMem_MALLOC(slicelength*sizeof(PyObject*)); if (!garbage) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 04:32:28 2016 From: python-checkins at python.org (mark.dickinson) Date: Sun, 21 Aug 2016 08:32:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327662=3A_add_miss?= =?utf-8?q?ing_Misc/NEWS_entry=2E?= Message-ID: <20160821083154.10896.10530.0A080860@psf.io> https://hg.python.org/cpython/rev/c7f9e66826a0 changeset: 102823:c7f9e66826a0 user: Mark Dickinson date: Sun Aug 21 09:31:44 2016 +0100 summary: Issue #27662: add missing Misc/NEWS entry. files: Misc/NEWS | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #27662: Fix an overflow check in ``List_New``: the original code was + checking against ``Py_SIZE_MAX`` instead of the correct upper bound of + ``Py_SSIZE_T_MAX``. Patch by Xiang Zhang. + - Issue #27782: Multi-phase extension module import now correctly allows the ``m_methods`` field to be used to add module level functions to instances of non-module types returned from ``Py_create_mod``. Patch by Xiang Zhang. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 05:23:41 2016 From: python-checkins at python.org (mark.dickinson) Date: Sun, 21 Aug 2016 09:23:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325604=3A_Fix_mino?= =?utf-8?q?r_bug_in_integer_true_division=2C_which_could?= Message-ID: <20160821092340.83132.6588.9F47A7E4@psf.io> https://hg.python.org/cpython/rev/b9e12ca6fdb6 changeset: 102824:b9e12ca6fdb6 user: Mark Dickinson date: Sun Aug 21 10:23:23 2016 +0100 summary: Issue #25604: Fix minor bug in integer true division, which could have caused off-by-one-ulp results on certain platforms. files: Misc/NEWS | 4 ++++ Objects/longobject.c | 4 ++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #25604: Fix a minor bug in integer true division; this bug could + potentially have caused off-by-one-ulp results on platforms with + unreliable ldexp implementations. + - Issue #27662: Fix an overflow check in ``List_New``: the original code was checking against ``Py_SIZE_MAX`` instead of the correct upper bound of ``Py_SSIZE_T_MAX``. Patch by Xiang Zhang. diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -3893,9 +3893,9 @@ /* Round by directly modifying the low digit of x. */ mask = (digit)1 << (extra_bits - 1); low = x->ob_digit[0] | inexact; - if (low & mask && low & (3*mask-1)) + if ((low & mask) && (low & (3U*mask-1U))) low += mask; - x->ob_digit[0] = low & ~(mask-1U); + x->ob_digit[0] = low & ~(2U*mask-1U); /* Convert x to a double dx; the conversion is exact. */ dx = x->ob_digit[--x_size]; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 05:33:43 2016 From: python-checkins at python.org (mark.dickinson) Date: Sun, 21 Aug 2016 09:33:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Untabify_Objects/longobjec?= =?utf-8?q?t=2Ec=2E?= Message-ID: <20160821093343.30709.13409.1141FE08@psf.io> https://hg.python.org/cpython/rev/c10a8f9775a2 changeset: 102825:c10a8f9775a2 user: Mark Dickinson date: Sun Aug 21 10:33:36 2016 +0100 summary: Untabify Objects/longobject.c. files: Objects/longobject.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -2771,8 +2771,8 @@ } if (Py_ABS(Py_SIZE(v)) <= 1) { /* Fast path; single digit long (31 bits) will cast safely - to double. This improves performance of FP/long operations - by 20%. + to double. This improves performance of FP/long operations + by 20%. */ return (double)MEDIUM_VALUE((PyLongObject *)v); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 05:59:55 2016 From: python-checkins at python.org (mark.dickinson) Date: Sun, 21 Aug 2016 09:59:55 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1NjA0?= =?utf-8?q?=3A_Fix_bug_in_integer_true_division_that_could_have_resulted_i?= =?utf-8?q?n?= Message-ID: <20160821095954.89774.73199.3B65AB82@psf.io> https://hg.python.org/cpython/rev/370bbeba21b3 changeset: 102826:370bbeba21b3 branch: 2.7 parent: 102815:3e0f8acf786c user: Mark Dickinson date: Sun Aug 21 10:59:48 2016 +0100 summary: Issue #25604: Fix bug in integer true division that could have resulted in off-by-one-ulp results in unusual cases. files: Misc/NEWS | 4 ++++ Objects/longobject.c | 4 ++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #25604: Fix a minor bug in integer true division; this bug could + potentially have caused off-by-one-ulp results on platforms with + unreliable ldexp implementations. + - Issue #27473: Fixed possible integer overflow in str, unicode and bytearray concatenations and repetitions. Based on patch by Xiang Zhang. diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -3315,9 +3315,9 @@ /* Round by directly modifying the low digit of x. */ mask = (digit)1 << (extra_bits - 1); low = x->ob_digit[0] | inexact; - if (low & mask && low & (3*mask-1)) + if ((low & mask) && (low & (3U*mask-1U))) low += mask; - x->ob_digit[0] = low & ~(mask-1U); + x->ob_digit[0] = low & ~(2U*mask-1U); /* Convert x to a double dx; the conversion is exact. */ dx = x->ob_digit[--x_size]; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 12:28:28 2016 From: python-checkins at python.org (berker.peksag) Date: Sun, 21 Aug 2016 16:28:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2321718=3A_Merge_from_3=2E5?= Message-ID: <20160821162828.83132.72733.073A23FE@psf.io> https://hg.python.org/cpython/rev/cf18375732ae changeset: 102828:cf18375732ae parent: 102825:c10a8f9775a2 parent: 102827:45ef062734d6 user: Berker Peksag date: Sun Aug 21 19:41:38 2016 +0300 summary: Issue #21718: Merge from 3.5 files: Lib/sqlite3/test/types.py | 42 ++++++++++++++++++++++++++- Misc/NEWS | 2 + Modules/_sqlite/cursor.c | 11 +++--- 3 files changed, 48 insertions(+), 7 deletions(-) diff --git a/Lib/sqlite3/test/types.py b/Lib/sqlite3/test/types.py --- a/Lib/sqlite3/test/types.py +++ b/Lib/sqlite3/test/types.py @@ -274,6 +274,45 @@ self.cur.execute("select * from test where 0 = 1") self.assertEqual(self.cur.description[0][0], "x") + def CheckCursorDescriptionInsert(self): + self.cur.execute("insert into test values (1)") + self.assertIsNone(self.cur.description) + + + at unittest.skipIf(sqlite.sqlite_version_info < (3, 8, 3), "CTEs not supported") +class CommonTableExpressionTests(unittest.TestCase): + + def setUp(self): + self.con = sqlite.connect(":memory:") + self.cur = self.con.cursor() + self.cur.execute("create table test(x foo)") + + def tearDown(self): + self.cur.close() + self.con.close() + + def CheckCursorDescriptionCTESimple(self): + self.cur.execute("with one as (select 1) select * from one") + self.assertIsNotNone(self.cur.description) + self.assertEqual(self.cur.description[0][0], "1") + + def CheckCursorDescriptionCTESMultipleColumns(self): + self.cur.execute("insert into test values(1)") + self.cur.execute("insert into test values(2)") + self.cur.execute("with testCTE as (select * from test) select * from testCTE") + self.assertIsNotNone(self.cur.description) + self.assertEqual(self.cur.description[0][0], "x") + + def CheckCursorDescriptionCTE(self): + self.cur.execute("insert into test values (1)") + self.cur.execute("with bar as (select * from test) select * from test where x = 1") + self.assertIsNotNone(self.cur.description) + self.assertEqual(self.cur.description[0][0], "x") + self.cur.execute("with bar as (select * from test) select * from test where x = 2") + self.assertIsNotNone(self.cur.description) + self.assertEqual(self.cur.description[0][0], "x") + + class ObjectAdaptationTests(unittest.TestCase): def cast(obj): return float(obj) @@ -372,7 +411,8 @@ adaptation_suite = unittest.makeSuite(ObjectAdaptationTests, "Check") bin_suite = unittest.makeSuite(BinaryConverterTests, "Check") date_suite = unittest.makeSuite(DateTimeTests, "Check") - return unittest.TestSuite((sqlite_type_suite, decltypes_type_suite, colnames_type_suite, adaptation_suite, bin_suite, date_suite)) + cte_suite = unittest.makeSuite(CommonTableExpressionTests, "Check") + return unittest.TestSuite((sqlite_type_suite, decltypes_type_suite, colnames_type_suite, adaptation_suite, bin_suite, date_suite, cte_suite)) def test(): runner = unittest.TextTestRunner() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -36,6 +36,8 @@ Library ------- +- Issue #21718: cursor.description is now available for queries using CTEs. + - Issue #27819: In distutils sdists, simply produce the "gztar" (gzipped tar format) distributions on all platforms unless "formats" is supplied. diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c --- a/Modules/_sqlite/cursor.c +++ b/Modules/_sqlite/cursor.c @@ -645,12 +645,11 @@ goto error; } - if (rc == SQLITE_ROW || (rc == SQLITE_DONE && statement_type == STATEMENT_SELECT)) { - if (self->description == Py_None) { - Py_BEGIN_ALLOW_THREADS - numcols = sqlite3_column_count(self->statement->st); - Py_END_ALLOW_THREADS - + if (rc == SQLITE_ROW || rc == SQLITE_DONE) { + Py_BEGIN_ALLOW_THREADS + numcols = sqlite3_column_count(self->statement->st); + Py_END_ALLOW_THREADS + if (self->description == Py_None && numcols > 0) { Py_SETREF(self->description, PyTuple_New(numcols)); if (!self->description) { goto error; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 12:28:27 2016 From: python-checkins at python.org (berker.peksag) Date: Sun, 21 Aug 2016 16:28:27 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIxNzE4?= =?utf-8?q?=3A_cursor=2Edescription_is_now_available_for_queries_using_CTE?= =?utf-8?q?s?= Message-ID: <20160821162827.28967.21871.FD9F43D9@psf.io> https://hg.python.org/cpython/rev/45ef062734d6 changeset: 102827:45ef062734d6 branch: 3.5 parent: 102820:913268337886 user: Berker Peksag date: Sun Aug 21 19:38:47 2016 +0300 summary: Issue #21718: cursor.description is now available for queries using CTEs According to PEP 249, cursor.description must be available for any SELECT statements, such as those that use CTEs. Backported from https://github.com/ghaering/pysqlite/commit/f67fa9c898a4713850e16934046f0fe2cba8c44c Additional test cases added by me. files: Lib/sqlite3/test/types.py | 42 ++++++++++++++++++++++++++- Misc/NEWS | 2 + Modules/_sqlite/cursor.c | 11 +++--- 3 files changed, 48 insertions(+), 7 deletions(-) diff --git a/Lib/sqlite3/test/types.py b/Lib/sqlite3/test/types.py --- a/Lib/sqlite3/test/types.py +++ b/Lib/sqlite3/test/types.py @@ -274,6 +274,45 @@ self.cur.execute("select * from test where 0 = 1") self.assertEqual(self.cur.description[0][0], "x") + def CheckCursorDescriptionInsert(self): + self.cur.execute("insert into test values (1)") + self.assertIsNone(self.cur.description) + + + at unittest.skipIf(sqlite.sqlite_version_info < (3, 8, 3), "CTEs not supported") +class CommonTableExpressionTests(unittest.TestCase): + + def setUp(self): + self.con = sqlite.connect(":memory:") + self.cur = self.con.cursor() + self.cur.execute("create table test(x foo)") + + def tearDown(self): + self.cur.close() + self.con.close() + + def CheckCursorDescriptionCTESimple(self): + self.cur.execute("with one as (select 1) select * from one") + self.assertIsNotNone(self.cur.description) + self.assertEqual(self.cur.description[0][0], "1") + + def CheckCursorDescriptionCTESMultipleColumns(self): + self.cur.execute("insert into test values(1)") + self.cur.execute("insert into test values(2)") + self.cur.execute("with testCTE as (select * from test) select * from testCTE") + self.assertIsNotNone(self.cur.description) + self.assertEqual(self.cur.description[0][0], "x") + + def CheckCursorDescriptionCTE(self): + self.cur.execute("insert into test values (1)") + self.cur.execute("with bar as (select * from test) select * from test where x = 1") + self.assertIsNotNone(self.cur.description) + self.assertEqual(self.cur.description[0][0], "x") + self.cur.execute("with bar as (select * from test) select * from test where x = 2") + self.assertIsNotNone(self.cur.description) + self.assertEqual(self.cur.description[0][0], "x") + + class ObjectAdaptationTests(unittest.TestCase): def cast(obj): return float(obj) @@ -372,7 +411,8 @@ adaptation_suite = unittest.makeSuite(ObjectAdaptationTests, "Check") bin_suite = unittest.makeSuite(BinaryConverterTests, "Check") date_suite = unittest.makeSuite(DateTimeTests, "Check") - return unittest.TestSuite((sqlite_type_suite, decltypes_type_suite, colnames_type_suite, adaptation_suite, bin_suite, date_suite)) + cte_suite = unittest.makeSuite(CommonTableExpressionTests, "Check") + return unittest.TestSuite((sqlite_type_suite, decltypes_type_suite, colnames_type_suite, adaptation_suite, bin_suite, date_suite, cte_suite)) def test(): runner = unittest.TextTestRunner() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,8 @@ Library ------- +- Issue #21718: cursor.description is now available for queries using CTEs. + - Issue #2466: posixpath.ismount now correctly recognizes mount points which the user does not have permission to access. diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c --- a/Modules/_sqlite/cursor.c +++ b/Modules/_sqlite/cursor.c @@ -646,12 +646,11 @@ goto error; } - if (rc == SQLITE_ROW || (rc == SQLITE_DONE && statement_type == STATEMENT_SELECT)) { - if (self->description == Py_None) { - Py_BEGIN_ALLOW_THREADS - numcols = sqlite3_column_count(self->statement->st); - Py_END_ALLOW_THREADS - + if (rc == SQLITE_ROW || rc == SQLITE_DONE) { + Py_BEGIN_ALLOW_THREADS + numcols = sqlite3_column_count(self->statement->st); + Py_END_ALLOW_THREADS + if (self->description == Py_None && numcols > 0) { Py_SETREF(self->description, PyTuple_New(numcols)); if (!self->description) { goto error; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 13:03:31 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 21 Aug 2016 17:03:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326984=3A_int=28?= =?utf-8?q?=29_now_always_returns_an_instance_of_exact_int=2E?= Message-ID: <20160821170331.83005.63857.C2619A26@psf.io> https://hg.python.org/cpython/rev/81f229262921 changeset: 102829:81f229262921 user: Serhiy Storchaka date: Sun Aug 21 20:03:08 2016 +0300 summary: Issue #26984: int() now always returns an instance of exact int. files: Lib/test/test_int.py | 5 +++- Misc/NEWS | 2 + Objects/abstract.c | 35 +++++++++++++++++++++---------- 3 files changed, 29 insertions(+), 13 deletions(-) diff --git a/Lib/test/test_int.py b/Lib/test/test_int.py --- a/Lib/test/test_int.py +++ b/Lib/test/test_int.py @@ -430,21 +430,24 @@ with self.assertWarns(DeprecationWarning): n = int(bad_int) self.assertEqual(n, 1) + self.assertIs(type(n), int) bad_int = BadInt2() with self.assertWarns(DeprecationWarning): n = int(bad_int) self.assertEqual(n, 1) + self.assertIs(type(n), int) bad_int = TruncReturnsBadInt() with self.assertWarns(DeprecationWarning): n = int(bad_int) self.assertEqual(n, 1) + self.assertIs(type(n), int) good_int = TruncReturnsIntSubclass() n = int(good_int) self.assertEqual(n, 1) - self.assertIs(type(n), bool) + self.assertIs(type(n), int) n = IntSubclass(good_int) self.assertEqual(n, 1) self.assertIs(type(n), IntSubclass) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #26984: int() now always returns an instance of exact int. + - Issue #25604: Fix a minor bug in integer true division; this bug could potentially have caused off-by-one-ulp results on platforms with unreliable ldexp implementations. diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -1281,6 +1281,7 @@ PyObject * PyNumber_Long(PyObject *o) { + PyObject *result; PyNumberMethods *m; PyObject *trunc_func; Py_buffer view; @@ -1296,29 +1297,39 @@ } m = o->ob_type->tp_as_number; if (m && m->nb_int) { /* This should include subclasses of int */ - return (PyObject *)_PyLong_FromNbInt(o); + result = (PyObject *)_PyLong_FromNbInt(o); + if (result != NULL && !PyLong_CheckExact(result)) { + Py_SETREF(result, _PyLong_Copy((PyLongObject *)result)); + } + return result; } trunc_func = _PyObject_LookupSpecial(o, &PyId___trunc__); if (trunc_func) { - PyObject *truncated = PyEval_CallObject(trunc_func, NULL); - PyObject *int_instance; + result = PyEval_CallObject(trunc_func, NULL); Py_DECREF(trunc_func); - if (truncated == NULL || PyLong_Check(truncated)) - return truncated; + if (result == NULL || PyLong_CheckExact(result)) { + return result; + } + if (PyLong_Check(result)) { + Py_SETREF(result, _PyLong_Copy((PyLongObject *)result)); + return result; + } /* __trunc__ is specified to return an Integral type, but int() needs to return an int. */ - m = truncated->ob_type->tp_as_number; + m = result->ob_type->tp_as_number; if (m == NULL || m->nb_int == NULL) { PyErr_Format( PyExc_TypeError, "__trunc__ returned non-Integral (type %.200s)", - truncated->ob_type->tp_name); - Py_DECREF(truncated); + result->ob_type->tp_name); + Py_DECREF(result); return NULL; } - int_instance = (PyObject *)_PyLong_FromNbInt(truncated); - Py_DECREF(truncated); - return int_instance; + Py_SETREF(result, (PyObject *)_PyLong_FromNbInt(result)); + if (result != NULL && !PyLong_CheckExact(result)) { + Py_SETREF(result, _PyLong_Copy((PyLongObject *)result)); + } + return result; } if (PyErr_Occurred()) return NULL; @@ -1340,7 +1351,7 @@ PyByteArray_GET_SIZE(o), 10); if (PyObject_GetBuffer(o, &view, PyBUF_SIMPLE) == 0) { - PyObject *result, *bytes; + PyObject *bytes; /* Copy to NUL-terminated buffer. */ bytes = PyBytes_FromStringAndSize((const char *)view.buf, view.len); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 15:53:14 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 21 Aug 2016 19:53:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_remove_unused_list_of_pgen?= =?utf-8?q?_srcs?= Message-ID: <20160821195314.13731.93458.27013A8E@psf.io> https://hg.python.org/cpython/rev/81af0ab3db97 changeset: 102830:81af0ab3db97 user: Benjamin Peterson date: Sun Aug 21 12:53:09 2016 -0700 summary: remove unused list of pgen srcs files: Makefile.pre.in | 23 ----------------------- 1 files changed, 0 insertions(+), 23 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -281,18 +281,6 @@ # Parser PGEN= Parser/pgen$(EXE) -PSRCS= \ - Parser/acceler.c \ - Parser/grammar1.c \ - Parser/listnode.c \ - Parser/node.c \ - Parser/parser.c \ - Parser/bitset.c \ - Parser/metagrammar.c \ - Parser/firstsets.c \ - Parser/grammar.c \ - Parser/pgen.c - POBJS= \ Parser/acceler.o \ Parser/grammar1.o \ @@ -307,16 +295,6 @@ PARSER_OBJS= $(POBJS) Parser/myreadline.o Parser/parsetok.o Parser/tokenizer.o -PGSRCS= \ - Objects/obmalloc.c \ - Python/dynamic_annotations.c \ - Python/mysnprintf.c \ - Python/pyctype.c \ - Parser/tokenizer_pgen.c \ - Parser/printgrammar.c \ - Parser/parsetok_pgen.c \ - Parser/pgenmain.c - PGOBJS= \ Objects/obmalloc.o \ Python/dynamic_annotations.o \ @@ -332,7 +310,6 @@ $(srcdir)/Include/parsetok.h \ $(srcdir)/Parser/tokenizer.h -PGENSRCS= $(PSRCS) $(PGSRCS) PGENOBJS= $(POBJS) $(PGOBJS) ########################################################################## -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 16:09:45 2016 From: python-checkins at python.org (jason.coombs) Date: Sun, 21 Aug 2016 20:09:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327819=3A_Add_more?= =?utf-8?q?_detail_in_What=27s_New_in_3=2E6=2E?= Message-ID: <20160821200945.10896.71797.0B76C6F6@psf.io> https://hg.python.org/cpython/rev/ed9cabce9c38 changeset: 102831:ed9cabce9c38 user: Jason R. Coombs date: Sun Aug 21 16:09:27 2016 -0400 summary: Issue #27819: Add more detail in What's New in 3.6. files: Doc/whatsnew/3.6.rst | 23 +++++++++++++++++++++++ 1 files changed, 23 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -306,6 +306,16 @@ (Contributed by Ashley Anderson in :issue:`12006`.) +distutils.command.sdist +----------------------- + +The ``default_format`` attribute has been removed from +:class:`distutils.command.sdist.sdist` and the ``formats`` +attribute defaults to ``['gztar']``. Although not anticipated, +Any code relying on the presence of ``default_format`` may +need to be adapted. See :issue:`27819` for more details. + + faulthandler ------------ @@ -821,6 +831,19 @@ accepting additional keyword arguments will need to adjust their calls to :meth:`type.__new__` (whether direct or via :class:`super`) accordingly. +* In :class:`distutils.command.sdist.sdist`, the ``default_format`` + attribute has been removed and is no longer honored. Instead, the + gzipped tarfile format is the default on all platforms and no + platform-specific selection is made. + In environments where distributions are + built on Windows and zip distributions are required, configure + the project with a ``setup.cfg`` file containing the following:: + + [sdist] + formats=zip + + This behavior has also been backported to earlier Python versions + by Setuptools 26.0.0. Changes in the C API -------------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Aug 21 23:52:34 2016 From: python-checkins at python.org (raymond.hettinger) Date: Mon, 22 Aug 2016 03:52:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_main_section_that_w?= =?utf-8?q?as_only_used_during_testing_and_development?= Message-ID: <20160822035234.89774.80134.A4D014F2@psf.io> https://hg.python.org/cpython/rev/485d7c341386 changeset: 102832:485d7c341386 user: Raymond Hettinger date: Sun Aug 21 20:52:26 2016 -0700 summary: Remove main section that was only used during testing and development files: Lib/collections/__main__.py | 38 ------------------------- 1 files changed, 0 insertions(+), 38 deletions(-) diff --git a/Lib/collections/__main__.py b/Lib/collections/__main__.py deleted file mode 100644 --- a/Lib/collections/__main__.py +++ /dev/null @@ -1,38 +0,0 @@ -################################################################################ -### Simple tests -################################################################################ - -# verify that instances can be pickled -from collections import namedtuple -from pickle import loads, dumps -Point = namedtuple('Point', 'x, y', True) -p = Point(x=10, y=20) -assert p == loads(dumps(p)) - -# test and demonstrate ability to override methods -class Point(namedtuple('Point', 'x y')): - __slots__ = () - @property - def hypot(self): - return (self.x ** 2 + self.y ** 2) ** 0.5 - def __str__(self): - return 'Point: x=%6.3f y=%6.3f hypot=%6.3f' % (self.x, self.y, self.hypot) - -for p in Point(3, 4), Point(14, 5/7.): - print (p) - -class Point(namedtuple('Point', 'x y')): - 'Point class with optimized _make() and _replace() without error-checking' - __slots__ = () - _make = classmethod(tuple.__new__) - def _replace(self, _map=map, **kwds): - return self._make(_map(kwds.get, ('x', 'y'), self)) - -print(Point(11, 22)._replace(x=100)) - -Point3D = namedtuple('Point3D', Point._fields + ('z',)) -print(Point3D.__doc__) - -import doctest, collections -TestResults = namedtuple('TestResults', 'failed attempted') -print(TestResults(*doctest.testmod(collections))) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 05:56:19 2016 From: python-checkins at python.org (mark.dickinson) Date: Mon, 22 Aug 2016 09:56:19 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NTM5?= =?utf-8?q?=3A_Fix_unnormalised_Fraction=2E=5F=5Fpow=5F=5F_result_for_nega?= =?utf-8?q?tive_exponent?= Message-ID: <20160822095618.83037.50724.6F2483BA@psf.io> https://hg.python.org/cpython/rev/7eea5b87f5fa changeset: 102833:7eea5b87f5fa branch: 3.5 parent: 102827:45ef062734d6 user: Mark Dickinson date: Mon Aug 22 10:50:53 2016 +0100 summary: Issue #27539: Fix unnormalised Fraction.__pow__ result for negative exponent and base. Thanks Vedran ?a?i?. files: Lib/fractions.py | 6 +++++- Lib/test/test_fractions.py | 13 +++++++++++++ Misc/ACKS | 1 + Misc/NEWS | 3 +++ 4 files changed, 22 insertions(+), 1 deletions(-) diff --git a/Lib/fractions.py b/Lib/fractions.py --- a/Lib/fractions.py +++ b/Lib/fractions.py @@ -484,10 +484,14 @@ return Fraction(a._numerator ** power, a._denominator ** power, _normalize=False) - else: + elif a._numerator >= 0: return Fraction(a._denominator ** -power, a._numerator ** -power, _normalize=False) + else: + return Fraction((-a._denominator) ** -power, + (-a._numerator) ** -power, + _normalize=False) else: # A fractional power will generally produce an # irrational number. diff --git a/Lib/test/test_fractions.py b/Lib/test/test_fractions.py --- a/Lib/test/test_fractions.py +++ b/Lib/test/test_fractions.py @@ -356,6 +356,19 @@ z = pow(F(-1), F(1, 2)) self.assertAlmostEqual(z.real, 0) self.assertEqual(z.imag, 1) + # Regression test for #27539. + p = F(-1, 2) ** 0 + self.assertEqual(p, F(1, 1)) + self.assertEqual(p.numerator, 1) + self.assertEqual(p.denominator, 1) + p = F(-1, 2) ** -1 + self.assertEqual(p, F(-2, 1)) + self.assertEqual(p.numerator, -2) + self.assertEqual(p.denominator, 1) + p = F(-1, 2) ** -2 + self.assertEqual(p, F(4, 1)) + self.assertEqual(p.numerator, 4) + self.assertEqual(p.denominator, 1) def testMixedArithmetic(self): self.assertTypedEquals(F(11, 10), F(1, 10) + 1) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -217,6 +217,7 @@ Ralph Butler Laurent De Buyst Zach Byrne +Vedran ?a?i? Nicolas Cadou Jp Calderone Arnaud Calmettes diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,9 @@ Library ------- +- Issue #27539: Fix unnormalised ``Fraction.__pow__`` result in the case + of negative exponent and negative base. + - Issue #21718: cursor.description is now available for queries using CTEs. - Issue #2466: posixpath.ismount now correctly recognizes mount points which -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 05:56:19 2016 From: python-checkins at python.org (mark.dickinson) Date: Mon, 22 Aug 2016 09:56:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327539=3A_Merge_from_3=2E5=2E?= Message-ID: <20160822095619.89480.91464.8A9267D5@psf.io> https://hg.python.org/cpython/rev/902e82f71880 changeset: 102834:902e82f71880 parent: 102832:485d7c341386 parent: 102833:7eea5b87f5fa user: Mark Dickinson date: Mon Aug 22 10:56:06 2016 +0100 summary: Issue #27539: Merge from 3.5. files: Lib/fractions.py | 6 +++++- Lib/test/test_fractions.py | 13 +++++++++++++ Misc/ACKS | 1 + Misc/NEWS | 3 +++ 4 files changed, 22 insertions(+), 1 deletions(-) diff --git a/Lib/fractions.py b/Lib/fractions.py --- a/Lib/fractions.py +++ b/Lib/fractions.py @@ -460,10 +460,14 @@ return Fraction(a._numerator ** power, a._denominator ** power, _normalize=False) - else: + elif a._numerator >= 0: return Fraction(a._denominator ** -power, a._numerator ** -power, _normalize=False) + else: + return Fraction((-a._denominator) ** -power, + (-a._numerator) ** -power, + _normalize=False) else: # A fractional power will generally produce an # irrational number. diff --git a/Lib/test/test_fractions.py b/Lib/test/test_fractions.py --- a/Lib/test/test_fractions.py +++ b/Lib/test/test_fractions.py @@ -356,6 +356,19 @@ z = pow(F(-1), F(1, 2)) self.assertAlmostEqual(z.real, 0) self.assertEqual(z.imag, 1) + # Regression test for #27539. + p = F(-1, 2) ** 0 + self.assertEqual(p, F(1, 1)) + self.assertEqual(p.numerator, 1) + self.assertEqual(p.denominator, 1) + p = F(-1, 2) ** -1 + self.assertEqual(p, F(-2, 1)) + self.assertEqual(p.numerator, -2) + self.assertEqual(p.denominator, 1) + p = F(-1, 2) ** -2 + self.assertEqual(p, F(4, 1)) + self.assertEqual(p.numerator, 4) + self.assertEqual(p.denominator, 1) def testMixedArithmetic(self): self.assertTypedEquals(F(11, 10), F(1, 10) + 1) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -218,6 +218,7 @@ Ralph Butler Laurent De Buyst Zach Byrne +Vedran ?a?i? Nicolas Cadou Jp Calderone Arnaud Calmettes diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -38,6 +38,9 @@ Library ------- +- Issue #27539: Fix unnormalised ``Fraction.__pow__`` result in the case + of negative exponent and negative base. + - Issue #21718: cursor.description is now available for queries using CTEs. - Issue #27819: In distutils sdists, simply produce the "gztar" (gzipped tar -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 06:37:25 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 10:37:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Optimize_call_to_Python_fu?= =?utf-8?q?nction_without_argument?= Message-ID: <20160822103725.11221.74994.D0FDA170@psf.io> https://hg.python.org/cpython/rev/7dd85b19c873 changeset: 102835:7dd85b19c873 user: Victor Stinner date: Mon Aug 22 12:29:42 2016 +0200 summary: Optimize call to Python function without argument Issue #27128. When a Python function is called with no arguments, but all parameters have a default value: use default values as arguments for the fast path. files: Python/ceval.c | 42 ++++++++++++++++++++++++++++--------- 1 files changed, 31 insertions(+), 11 deletions(-) diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4801,8 +4801,8 @@ */ static PyObject* -_PyFunction_FastCallNoKw(PyObject **args, Py_ssize_t na, - PyCodeObject *co, PyObject *globals) +_PyFunction_FastCallNoKw(PyCodeObject *co, PyObject **args, Py_ssize_t na, + PyObject *globals) { PyFrameObject *f; PyThreadState *tstate = PyThreadState_GET(); @@ -4837,8 +4837,10 @@ return result; } +/* Similar to _PyFunction_FastCall() but keywords are passed a (key, value) + pairs in stack */ static PyObject * -fast_function(PyObject *func, PyObject **stack, int n, int na, int nk) +fast_function(PyObject *func, PyObject **stack, int n, int nargs, int nk) { PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func); PyObject *globals = PyFunction_GET_GLOBALS(func); @@ -4850,11 +4852,20 @@ PCALL(PCALL_FUNCTION); PCALL(PCALL_FAST_FUNCTION); - if (argdefs == NULL && co->co_argcount == na && - co->co_kwonlyargcount == 0 && nk == 0 && + if (co->co_kwonlyargcount == 0 && nk == 0 && co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) { - return _PyFunction_FastCallNoKw(stack, na, co, globals); + if (argdefs == NULL && co->co_argcount == nargs) { + return _PyFunction_FastCallNoKw(co, stack, nargs, globals); + } + else if (nargs == 0 && argdefs != NULL + && co->co_argcount == Py_SIZE(argdefs)) { + /* function called with no arguments, but all parameters have + a default value: use default values as arguments .*/ + stack = &PyTuple_GET_ITEM(argdefs, 0); + return _PyFunction_FastCallNoKw(co, stack, Py_SIZE(argdefs), + globals); + } } kwdefs = PyFunction_GET_KW_DEFAULTS(func); @@ -4871,8 +4882,8 @@ nd = 0; } return _PyEval_EvalCodeWithName((PyObject*)co, globals, (PyObject *)NULL, - stack, na, - stack + na, nk, + stack, nargs, + stack + nargs, nk, d, nd, kwdefs, closure, name, qualname); } @@ -4893,11 +4904,20 @@ /* issue #27128: support for keywords will come later */ assert(kwargs == NULL); - if (argdefs == NULL && co->co_argcount == nargs && - co->co_kwonlyargcount == 0 && + if (co->co_kwonlyargcount == 0 && kwargs == NULL && co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) { - return _PyFunction_FastCallNoKw(args, nargs, co, globals); + if (argdefs == NULL && co->co_argcount == nargs) { + return _PyFunction_FastCallNoKw(co, args, nargs, globals); + } + else if (nargs == 0 && argdefs != NULL + && co->co_argcount == Py_SIZE(argdefs)) { + /* function called with no arguments, but all parameters have + a default value: use default values as arguments .*/ + args = &PyTuple_GET_ITEM(argdefs, 0); + return _PyFunction_FastCallNoKw(co, args, Py_SIZE(argdefs), + globals); + } } kwdefs = PyFunction_GET_KW_DEFAULTS(func); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 07:24:57 2016 From: python-checkins at python.org (mark.dickinson) Date: Mon, 22 Aug 2016 11:24:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327792=3A_force_in?= =?utf-8?q?t_return_type_for_modulo_operations_involving_bools=2E?= Message-ID: <20160822112457.83397.87080.CA1FFBC6@psf.io> https://hg.python.org/cpython/rev/d998d87f0aa0 changeset: 102836:d998d87f0aa0 user: Mark Dickinson date: Mon Aug 22 12:24:46 2016 +0100 summary: Issue #27792: force int return type for modulo operations involving bools. files: Lib/test/test_bool.py | 7 +++++++ Misc/NEWS | 4 ++++ Objects/longobject.c | 7 +++++-- 3 files changed, 16 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_bool.py b/Lib/test/test_bool.py --- a/Lib/test/test_bool.py +++ b/Lib/test/test_bool.py @@ -96,6 +96,13 @@ self.assertEqual(False/1, 0) self.assertIsNot(False/1, False) + self.assertEqual(True%1, 0) + self.assertIsNot(True%1, False) + self.assertEqual(True%2, 1) + self.assertIsNot(True%2, True) + self.assertEqual(False%1, 0) + self.assertIsNot(False%1, False) + for b in False, True: for i in 0, 1, 2: self.assertEqual(b**i, int(b)**i) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #27792: The modulo operation applied to ``bool`` and other + ``int`` subclasses now always returns an ``int``. Previously + the return type depended on the input values. Patch by Xiang Zhang. + - Issue #26984: int() now always returns an instance of exact int. - Issue #25604: Fix a minor bug in integer true division; this bug could diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -2458,8 +2458,11 @@ *pdiv = (PyLongObject*)PyLong_FromLong(0); if (*pdiv == NULL) return -1; - Py_INCREF(a); - *prem = (PyLongObject *) a; + *prem = (PyLongObject *)long_long((PyObject *)a); + if (*prem == NULL) { + Py_CLEAR(*pdiv); + return -1; + } return 0; } if (size_b == 1) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 08:32:13 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 12:32:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327829=3A_libregrt?= =?utf-8?q?est=2Esave=5Fenv=3A_flush_stderr?= Message-ID: <20160822123212.130269.44002.74A51EE3@psf.io> https://hg.python.org/cpython/rev/2e23f7b9515c changeset: 102838:2e23f7b9515c user: Victor Stinner date: Mon Aug 22 14:29:54 2016 +0200 summary: Issue #27829: libregrtest.save_env: flush stderr Use flush=True to try to get a warning which is missing in buildbots. Use also f-string to make the code shorter. files: Lib/test/libregrtest/save_env.py | 10 ++++------ 1 files changed, 4 insertions(+), 6 deletions(-) diff --git a/Lib/test/libregrtest/save_env.py b/Lib/test/libregrtest/save_env.py --- a/Lib/test/libregrtest/save_env.py +++ b/Lib/test/libregrtest/save_env.py @@ -277,11 +277,9 @@ self.changed = True restore(original) if not self.quiet and not self.pgo: - print("Warning -- {} was modified by {}".format( - name, self.testname), - file=sys.stderr) + print(f"Warning -- {name} was modified by {self.testname}", + file=sys.stderr, flush=True) if self.verbose > 1: - print(" Before: {}\n After: {} ".format( - original, current), - file=sys.stderr) + print(f" Before: {original}\n After: {current} ", + file=sys.stderr, flush=True) return False -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 08:32:14 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 12:32:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Cleanup_libregrtest?= Message-ID: <20160822123212.20830.6647.50FECFA1@psf.io> https://hg.python.org/cpython/rev/13b13f4ad285 changeset: 102837:13b13f4ad285 user: Victor Stinner date: Mon Aug 22 14:28:52 2016 +0200 summary: Cleanup libregrtest * main.py: remove unused import * runtest: simplify runtest_inner() parameters, reuse ns parameter files: Lib/test/libregrtest/main.py | 1 - Lib/test/libregrtest/runtest.py | 56 +++++++------------- 2 files changed, 20 insertions(+), 37 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -1,6 +1,5 @@ import datetime import faulthandler -import math import os import platform import random diff --git a/Lib/test/libregrtest/runtest.py b/Lib/test/libregrtest/runtest.py --- a/Lib/test/libregrtest/runtest.py +++ b/Lib/test/libregrtest/runtest.py @@ -74,18 +74,12 @@ def runtest(ns, test): """Run a single test. + ns -- regrtest namespace of options test -- the name of the test - verbose -- if true, print more messages - quiet -- if true, don't print 'skipped' messages (probably redundant) - huntrleaks -- run multiple times to test for leaks; requires a debug - build; a triple corresponding to -R's three arguments - output_on_failure -- if true, display test output on failure - timeout -- dump the traceback and exit if a test takes more than - timeout seconds - failfast, match_tests -- See regrtest command-line flags for these. - pgo -- if true, suppress any info irrelevant to a generating a PGO build - Returns the tuple result, test_time, where result is one of the constants: + Returns the tuple (result, test_time), where result is one of the + constants: + INTERRUPTED KeyboardInterrupt when run under -j RESOURCE_DENIED test skipped because resource denied SKIPPED test skipped for some other reason @@ -94,21 +88,14 @@ PASSED test passed """ - verbose = ns.verbose - quiet = ns.quiet - huntrleaks = ns.huntrleaks output_on_failure = ns.verbose3 - failfast = ns.failfast - match_tests = ns.match_tests - timeout = ns.timeout - pgo = ns.pgo - use_timeout = (timeout is not None) + use_timeout = (ns.timeout is not None) if use_timeout: - faulthandler.dump_traceback_later(timeout, exit=True) + faulthandler.dump_traceback_later(ns.timeout, exit=True) try: - support.match_tests = match_tests - if failfast: + support.match_tests = ns.match_tests + if ns.failfast: support.failfast = True if output_on_failure: support.verbose = True @@ -129,8 +116,7 @@ try: sys.stdout = stream sys.stderr = stream - result = runtest_inner(ns, test, verbose, quiet, huntrleaks, - display_failure=False, pgo=pgo) + result = runtest_inner(ns, test, display_failure=False) if result[0] == FAILED: output = stream.getvalue() orig_stderr.write(output) @@ -139,19 +125,17 @@ sys.stdout = orig_stdout sys.stderr = orig_stderr else: - support.verbose = verbose # Tell tests to be moderately quiet - result = runtest_inner(ns, test, verbose, quiet, huntrleaks, - display_failure=not verbose, pgo=pgo) + support.verbose = ns.verbose # Tell tests to be moderately quiet + result = runtest_inner(ns, test, display_failure=not ns.verbose) return result finally: if use_timeout: faulthandler.cancel_dump_traceback_later() - cleanup_test_droppings(test, verbose) + cleanup_test_droppings(test, ns.verbose) runtest.stringio = None -def runtest_inner(ns, test, verbose, quiet, - huntrleaks=False, display_failure=True, *, pgo=False): +def runtest_inner(ns, test, display_failure=True): support.unload(test) test_time = 0.0 @@ -162,7 +146,7 @@ else: # Always import it from the test package abstest = 'test.' + test - with saved_test_environment(test, verbose, quiet, pgo=pgo) as environment: + with saved_test_environment(test, ns.verbose, ns.quiet, pgo=ns.pgo) as environment: start_time = time.time() the_module = importlib.import_module(abstest) # If the test has a test_main, that will run the appropriate @@ -178,21 +162,21 @@ raise Exception("errors while loading tests") support.run_unittest(tests) test_runner() - if huntrleaks: - refleak = dash_R(the_module, test, test_runner, huntrleaks) + if ns.huntrleaks: + refleak = dash_R(the_module, test, test_runner, ns.huntrleaks) test_time = time.time() - start_time except support.ResourceDenied as msg: - if not quiet and not pgo: + if not ns.quiet and not ns.pgo: print(test, "skipped --", msg, flush=True) return RESOURCE_DENIED, test_time except unittest.SkipTest as msg: - if not quiet and not pgo: + if not ns.quiet and not ns.pgo: print(test, "skipped --", msg, flush=True) return SKIPPED, test_time except KeyboardInterrupt: raise except support.TestFailed as msg: - if not pgo: + if not ns.pgo: if display_failure: print("test", test, "failed --", msg, file=sys.stderr, flush=True) @@ -201,7 +185,7 @@ return FAILED, test_time except: msg = traceback.format_exc() - if not pgo: + if not ns.pgo: print("test", test, "crashed --", msg, file=sys.stderr, flush=True) return FAILED, test_time -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Mon Aug 22 09:55:05 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Mon, 22 Aug 2016 14:55:05 +0100 Subject: [Python-checkins] UGLY Benchmark Results for Python 2.7 2016-08-22 Message-ID: <019b419b-fa04-45d4-8749-72673548008c@irsmsx103.ger.corp.intel.com> Results for project Python 2.7, build date 2016-08-22 02:46:49 +0000 commit: 370bbeba21b3 previous commit: 11a9bca71528 revision date: 2016-08-21 09:59:48 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.11% 2.01% 5.56% 3.78% :-) pybench 0.23% 0.50% 6.23% 3.92% :-( regex_v8 0.64% -0.01% -2.20% 10.36% :-) nbody 0.09% -1.11% 7.80% 2.76% :-) json_dump_v2 0.35% 0.73% 3.11% 10.32% :-| normal_startup 0.74% 0.03% -0.26% 1.89% :-) ssbench 0.17% 0.57% 2.99% 0.41% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-2-7-2016-08-22/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Mon Aug 22 09:56:33 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Mon, 22 Aug 2016 14:56:33 +0100 Subject: [Python-checkins] BAD Benchmark Results for Python Default 2016-08-22 Message-ID: <06cc89a1-23e6-481f-8ca3-c1e7b917357c@irsmsx103.ger.corp.intel.com> Results for project Python default, build date 2016-08-22 02:00:30 +0000 commit: ed9cabce9c38 previous commit: dd0f1fa809c5 revision date: 2016-08-21 20:09:27 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.26% 0.47% 11.63% 14.90% :-) pybench 0.26% -0.28% 5.96% 8.23% :-( regex_v8 2.60% 0.03% -3.14% 3.75% :-( nbody 0.10% -2.41% -0.72% 10.44% :-| json_dump_v2 0.35% 0.20% 0.14% 10.07% :-| normal_startup 1.15% 0.49% 1.55% 6.39% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-default-2016-08-22/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Mon Aug 22 11:06:11 2016 From: python-checkins at python.org (berker.peksag) Date: Mon, 22 Aug 2016 15:06:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327587=3A_Merge_from_3=2E5?= Message-ID: <20160822150611.27037.30257.2786E0D5@psf.io> https://hg.python.org/cpython/rev/7d90bf4780ff changeset: 102840:7d90bf4780ff parent: 102838:2e23f7b9515c parent: 102839:51627344fc58 user: Berker Peksag date: Mon Aug 22 18:07:02 2016 +0300 summary: Issue #27587: Merge from 3.5 files: Misc/NEWS | 4 ++++ Python/pystate.c | 6 ++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #27587: Fix another issue found by PVS-Studio: Null pointer check + after use of 'def' in _PyState_AddModule(). + Initial patch by Christian Heimes. + - Issue #27792: The modulo operation applied to ``bool`` and other ``int`` subclasses now always returns an ``int``. Previously the return type depended on the input values. Patch by Xiang Zhang. diff --git a/Python/pystate.c b/Python/pystate.c --- a/Python/pystate.c +++ b/Python/pystate.c @@ -285,14 +285,16 @@ _PyState_AddModule(PyObject* module, struct PyModuleDef* def) { PyInterpreterState *state; + if (!def) { + assert(PyErr_Occurred()); + return -1; + } if (def->m_slots) { PyErr_SetString(PyExc_SystemError, "PyState_AddModule called on module with slots"); return -1; } state = GET_INTERP_STATE(); - if (!def) - return -1; if (!state->modules_by_index) { state->modules_by_index = PyList_New(0); if (!state->modules_by_index) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 11:06:12 2016 From: python-checkins at python.org (berker.peksag) Date: Mon, 22 Aug 2016 15:06:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NTg3?= =?utf-8?q?=3A_Move_null_pointer_check_earlier_in_=5FPyState=5FAddModule?= =?utf-8?b?KCk=?= Message-ID: <20160822150611.130387.91382.9B9B7A7E@psf.io> https://hg.python.org/cpython/rev/51627344fc58 changeset: 102839:51627344fc58 branch: 3.5 parent: 102833:7eea5b87f5fa user: Berker Peksag date: Mon Aug 22 18:05:56 2016 +0300 summary: Issue #27587: Move null pointer check earlier in _PyState_AddModule() This was found by PVS-Studio: V595 The 'def' pointer was utilized before it was verified against nullptr. Check lines: 286, 292. pystate.c 286 Initial patch by Christian Heimes. files: Misc/NEWS | 4 ++++ Python/pystate.c | 6 ++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #27587: Fix another issue found by PVS-Studio: Null pointer check + after use of 'def' in _PyState_AddModule(). + Initial patch by Christian Heimes. + - Issue #27782: Multi-phase extension module import now correctly allows the ``m_methods`` field to be used to add module level functions to instances of non-module types returned from ``Py_create_mod``. Patch by Xiang Zhang. diff --git a/Python/pystate.c b/Python/pystate.c --- a/Python/pystate.c +++ b/Python/pystate.c @@ -281,14 +281,16 @@ _PyState_AddModule(PyObject* module, struct PyModuleDef* def) { PyInterpreterState *state; + if (!def) { + assert(PyErr_Occurred()); + return -1; + } if (def->m_slots) { PyErr_SetString(PyExc_SystemError, "PyState_AddModule called on module with slots"); return -1; } state = GET_INTERP_STATE(); - if (!def) - return -1; if (!state->modules_by_index) { state->modules_by_index = PyList_New(0); if (!state->modules_by_index) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 17:03:28 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 21:03:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Rename_=5FPyObject=5FFastC?= =?utf-8?b?YWxsKCkgdG8gX1B5T2JqZWN0X0Zhc3RDYWxsRGljdCgp?= Message-ID: <20160822210327.13731.82586.881C6663@psf.io> https://hg.python.org/cpython/rev/263334a652ac changeset: 102841:263334a652ac user: Victor Stinner date: Mon Aug 22 22:48:54 2016 +0200 summary: Rename _PyObject_FastCall() to _PyObject_FastCallDict() Issue #27809: * Rename _PyObject_FastCall() function to _PyObject_FastCallDict() * Add _PyObject_FastCall(), _PyObject_CallNoArg() and _PyObject_CallArg1() macros calling _PyObject_FastCallDict() files: Include/abstract.h | 17 +++++++++++++---- Modules/_elementtree.c | 2 +- Modules/_functoolsmodule.c | 2 +- Modules/_pickle.c | 6 +++--- Modules/_sre.c | 2 +- Objects/abstract.c | 12 ++++++------ Objects/fileobject.c | 2 +- Objects/iterobject.c | 2 +- Objects/typeobject.c | 14 +++++++------- Python/ceval.c | 4 ++-- Python/pythonrun.c | 2 +- Python/sysmodule.c | 4 ++-- 12 files changed, 39 insertions(+), 30 deletions(-) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -279,9 +279,18 @@ Return the result on success. Raise an exception on return NULL on error. */ - PyAPI_FUNC(PyObject *) _PyObject_FastCall(PyObject *func, - PyObject **args, int nargs, - PyObject *kwargs); + PyAPI_FUNC(PyObject *) _PyObject_FastCallDict(PyObject *func, + PyObject **args, int nargs, + PyObject *kwargs); + +#define _PyObject_FastCall(func, args, nargs) \ + _PyObject_FastCallDict((func), (args), (nargs), NULL) + +#define _PyObject_CallNoArg(func) \ + _PyObject_FastCall((func), NULL, 0) + +#define _PyObject_CallArg1(func, arg) \ + _PyObject_FastCall((func), &(arg), 1) PyAPI_FUNC(PyObject *) _Py_CheckFunctionResult(PyObject *func, PyObject *result, @@ -291,7 +300,7 @@ /* Call a callable Python object, callable_object, with arguments and keywords arguments. The 'args' argument can not be - NULL, but the 'kw' argument can be NULL. + NULL. */ PyAPI_FUNC(PyObject *) PyObject_CallObject(PyObject *callable_object, diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -858,7 +858,7 @@ stack[0] = object; stack[1] = memo; - return _PyObject_FastCall(st->deepcopy_obj, stack, 2, NULL); + return _PyObject_FastCall(st->deepcopy_obj, stack, 2); } diff --git a/Modules/_functoolsmodule.c b/Modules/_functoolsmodule.c --- a/Modules/_functoolsmodule.c +++ b/Modules/_functoolsmodule.c @@ -492,7 +492,7 @@ */ stack[0] = x; stack[1] = y; - res = _PyObject_FastCall(compare, stack, 2, NULL); + res = _PyObject_FastCall(compare, stack, 2); if (res == NULL) { return NULL; } diff --git a/Modules/_pickle.c b/Modules/_pickle.c --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -357,7 +357,7 @@ significantly reduced the number of function calls we do. Thus, the benefits became marginal at best. */ - result = _PyObject_FastCall(func, &obj, 1, NULL); + result = _PyObject_CallArg1(func, obj); Py_DECREF(obj); return result; } @@ -1151,7 +1151,7 @@ return -1; if (n == READ_WHOLE_LINE) { - data = _PyObject_FastCall(self->readline, NULL, 0, NULL); + data = _PyObject_CallNoArg(self->readline); } else { PyObject *len; @@ -3948,7 +3948,7 @@ /* Check for a __reduce__ method. */ reduce_func = _PyObject_GetAttrId(obj, &PyId___reduce__); if (reduce_func != NULL) { - reduce_value = _PyObject_FastCall(reduce_func, NULL, 0, NULL); + reduce_value = _PyObject_CallNoArg(reduce_func); } else { PyErr_Format(st->PicklingError, diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -1157,7 +1157,7 @@ match = pattern_new_match(self, &state, 1); if (!match) goto error; - item = _PyObject_FastCall(filter, &match, 1, NULL); + item = _PyObject_CallArg1(filter, match); Py_DECREF(match); if (!item) goto error; diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2255,12 +2255,12 @@ } PyObject * -_PyObject_FastCall(PyObject *func, PyObject **args, int nargs, PyObject *kwargs) +_PyObject_FastCallDict(PyObject *func, PyObject **args, int nargs, PyObject *kwargs) { ternaryfunc call; PyObject *result = NULL; - /* _PyObject_FastCall() must not be called with an exception set, + /* _PyObject_FastCallDict() must not be called with an exception set, because it may clear it (directly or indirectly) and so the caller loses its exception */ assert(!PyErr_Occurred()); @@ -2318,7 +2318,7 @@ assert(args != NULL); if (!PyTuple_Check(args)) { - result = _PyObject_FastCall(callable, &args, 1, NULL); + result = _PyObject_CallArg1(callable, args); } else { result = PyObject_Call(callable, args, NULL); @@ -2338,7 +2338,7 @@ } if (!format || !*format) { - return _PyObject_FastCall(callable, NULL, 0, NULL); + return _PyObject_CallNoArg(callable); } va_start(va, format); @@ -2364,7 +2364,7 @@ } if (!format || !*format) { - return _PyObject_FastCall(callable, NULL, 0, NULL); + return _PyObject_CallNoArg(callable); } va_start(va, format); @@ -2392,7 +2392,7 @@ } if (!format || !*format) { - return _PyObject_FastCall(func, NULL, 0, NULL); + return _PyObject_CallNoArg(func); } if (is_size_t) { diff --git a/Objects/fileobject.c b/Objects/fileobject.c --- a/Objects/fileobject.c +++ b/Objects/fileobject.c @@ -146,7 +146,7 @@ Py_DECREF(writer); return -1; } - result = _PyObject_FastCall(writer, &value, 1, NULL); + result = _PyObject_CallArg1(writer, value); Py_DECREF(value); Py_DECREF(writer); if (result == NULL) diff --git a/Objects/iterobject.c b/Objects/iterobject.c --- a/Objects/iterobject.c +++ b/Objects/iterobject.c @@ -214,7 +214,7 @@ return NULL; } - result = _PyObject_FastCall(it->it_callable, NULL, 0, NULL); + result = _PyObject_CallNoArg(it->it_callable); if (result != NULL) { int ok; diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -1450,7 +1450,7 @@ Py_DECREF(args); } else { - retval = _PyObject_FastCall(func, NULL, 0, NULL); + retval = _PyObject_CallNoArg(func); } Py_DECREF(func); @@ -1490,7 +1490,7 @@ Py_DECREF(args); } else { - retval = _PyObject_FastCall(func, NULL, 0, NULL); + retval = _PyObject_CallNoArg(func); } Py_DECREF(func); @@ -5834,7 +5834,7 @@ goto error; } - retval = _PyObject_FastCall(func, &ival, 1, NULL); + retval = _PyObject_CallArg1(func, ival); Py_DECREF(func); Py_DECREF(ival); return retval; @@ -5875,7 +5875,7 @@ return -1; } if (func != NULL) { - res = _PyObject_FastCall(func, &value, 1, NULL); + res = _PyObject_CallArg1(func, value); Py_DECREF(func); if (res != NULL) { result = PyObject_IsTrue(res); @@ -5967,7 +5967,7 @@ using_len = 1; } - value = _PyObject_FastCall(func, NULL, 0, NULL); + value = _PyObject_CallNoArg(func); if (value == NULL) { goto error; } @@ -6245,7 +6245,7 @@ PyErr_Clear(); Py_RETURN_NOTIMPLEMENTED; } - res = _PyObject_FastCall(func, &other, 1, NULL); + res = _PyObject_CallArg1(func, other); Py_DECREF(func); return res; } @@ -6266,7 +6266,7 @@ } if (func != NULL) { - res = _PyObject_FastCall(func, NULL, 0, NULL); + res = _PyObject_CallNoArg(func); Py_DECREF(func); return res; } diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4593,7 +4593,7 @@ if (args == NULL) { if (kwargs == NULL) { - return _PyObject_FastCall(func, NULL, 0, 0); + return _PyObject_CallNoArg(func); } args = PyTuple_New(0); @@ -5298,7 +5298,7 @@ stack[2] = f->f_locals == NULL ? Py_None : f->f_locals; stack[3] = fromlist; stack[4] = level; - res = _PyObject_FastCall(import_func, stack, 5, NULL); + res = _PyObject_FastCall(import_func, stack, 5); Py_DECREF(import_func); return res; } diff --git a/Python/pythonrun.c b/Python/pythonrun.c --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -636,7 +636,7 @@ stack[0] = exception; stack[1] = v; stack[2] = tb; - result = _PyObject_FastCall(hook, stack, 3, NULL); + result = _PyObject_FastCall(hook, stack, 3); if (result == NULL) { PyObject *exception2, *v2, *tb2; if (PyErr_ExceptionMatches(PyExc_SystemExit)) { diff --git a/Python/sysmodule.c b/Python/sysmodule.c --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -380,7 +380,7 @@ stack[2] = (arg != NULL) ? arg : Py_None; /* call the Python-level function */ - result = _PyObject_FastCall(callback, stack, 3, NULL); + result = _PyObject_FastCall(callback, stack, 3); PyFrame_LocalsToFast(frame, 1); if (result == NULL) { @@ -2122,7 +2122,7 @@ if (writer == NULL) goto error; - result = _PyObject_FastCall(writer, &unicode, 1, NULL); + result = _PyObject_CallArg1(writer, unicode); if (result == NULL) { goto error; } else { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 17:22:19 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 21:22:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327809=3A_Cleanup_?= =?utf-8?q?=5FPyEval=5FEvalCodeWithName=28=29?= Message-ID: <20160822212218.5843.75026.7AB1E8D5@psf.io> https://hg.python.org/cpython/rev/7924eac912fe changeset: 102843:7924eac912fe user: Victor Stinner date: Mon Aug 22 23:17:30 2016 +0200 summary: Issue #27809: Cleanup _PyEval_EvalCodeWithName() * Rename nm to name * PEP 7: add { ... } to if/else blocks files: Python/ceval.c | 28 +++++++++++++--------------- 1 files changed, 13 insertions(+), 15 deletions(-) diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -3878,28 +3878,28 @@ normally interned this should almost always hit. */ co_varnames = ((PyTupleObject *)(co->co_varnames))->ob_item; for (j = 0; j < total_args; j++) { - PyObject *nm = co_varnames[j]; - if (nm == keyword) + PyObject *name = co_varnames[j]; + if (name == keyword) { goto kw_found; + } } /* Slow fallback, just in case */ for (j = 0; j < total_args; j++) { - PyObject *nm = co_varnames[j]; - int cmp = PyObject_RichCompareBool( - keyword, nm, Py_EQ); - if (cmp > 0) + PyObject *name = co_varnames[j]; + int cmp = PyObject_RichCompareBool( keyword, name, Py_EQ); + if (cmp > 0) { goto kw_found; - else if (cmp < 0) + } + else if (cmp < 0) { goto fail; + } } if (j >= total_args && kwdict == NULL) { PyErr_Format(PyExc_TypeError, - "%U() got an unexpected " - "keyword argument '%S'", - co->co_name, - keyword); + "%U() got an unexpected keyword argument '%S'", + co->co_name, keyword); goto fail; } @@ -3911,10 +3911,8 @@ kw_found: if (GETLOCAL(j) != NULL) { PyErr_Format(PyExc_TypeError, - "%U() got multiple " - "values for argument '%S'", - co->co_name, - keyword); + "%U() got multiple values for argument '%S'", + co->co_name, keyword); goto fail; } Py_INCREF(value); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 17:22:19 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 21:22:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327809=3A_Use_=5FP?= =?utf-8?q?yObject=5FFastCallDict=28=29?= Message-ID: <20160822212219.89328.58446.40820F3C@psf.io> https://hg.python.org/cpython/rev/15eab21bf934 changeset: 102844:15eab21bf934 user: Victor Stinner date: Mon Aug 22 23:21:55 2016 +0200 summary: Issue #27809: Use _PyObject_FastCallDict() Modify: * builtin_sorted() * classmethoddescr_call() * methoddescr_call() * wrapperdescr_call() files: Objects/descrobject.c | 34 +++++++++--------------------- Python/bltinmodule.c | 15 ++++--------- 2 files changed, 15 insertions(+), 34 deletions(-) diff --git a/Objects/descrobject.c b/Objects/descrobject.c --- a/Objects/descrobject.c +++ b/Objects/descrobject.c @@ -213,7 +213,7 @@ methoddescr_call(PyMethodDescrObject *descr, PyObject *args, PyObject *kwds) { Py_ssize_t argc; - PyObject *self, *func, *result; + PyObject *self, *func, *result, **stack; /* Make sure that the first argument is acceptable as 'self' */ assert(PyTuple_Check(args)); @@ -242,13 +242,8 @@ func = PyCFunction_NewEx(descr->d_method, self, NULL); if (func == NULL) return NULL; - args = PyTuple_GetSlice(args, 1, argc); - if (args == NULL) { - Py_DECREF(func); - return NULL; - } - result = PyEval_CallObjectWithKeywords(func, args, kwds); - Py_DECREF(args); + stack = &PyTuple_GET_ITEM(args, 1); + result = _PyObject_FastCallDict(func, stack, argc - 1, kwds); Py_DECREF(func); return result; } @@ -258,7 +253,7 @@ PyObject *kwds) { Py_ssize_t argc; - PyObject *self, *func, *result; + PyObject *self, *func, *result, **stack; /* Make sure that the first argument is acceptable as 'self' */ assert(PyTuple_Check(args)); @@ -295,14 +290,9 @@ func = PyCFunction_NewEx(descr->d_method, self, NULL); if (func == NULL) return NULL; - args = PyTuple_GetSlice(args, 1, argc); - if (args == NULL) { - Py_DECREF(func); - return NULL; - } - result = PyEval_CallObjectWithKeywords(func, args, kwds); + stack = &PyTuple_GET_ITEM(args, 1); + result = _PyObject_FastCallDict(func, stack, argc - 1, kwds); Py_DECREF(func); - Py_DECREF(args); return result; } @@ -310,7 +300,7 @@ wrapperdescr_call(PyWrapperDescrObject *descr, PyObject *args, PyObject *kwds) { Py_ssize_t argc; - PyObject *self, *func, *result; + PyObject *self, *func, *result, **stack; /* Make sure that the first argument is acceptable as 'self' */ assert(PyTuple_Check(args)); @@ -339,13 +329,9 @@ func = PyWrapper_New((PyObject *)descr, self); if (func == NULL) return NULL; - args = PyTuple_GetSlice(args, 1, argc); - if (args == NULL) { - Py_DECREF(func); - return NULL; - } - result = PyEval_CallObjectWithKeywords(func, args, kwds); - Py_DECREF(args); + + stack = &PyTuple_GET_ITEM(args, 1); + result = _PyObject_FastCallDict(func, stack, argc - 1, kwds); Py_DECREF(func); return result; } diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -2087,10 +2087,11 @@ static PyObject * builtin_sorted(PyObject *self, PyObject *args, PyObject *kwds) { - PyObject *newlist, *v, *seq, *keyfunc=NULL, *newargs; + PyObject *newlist, *v, *seq, *keyfunc=NULL, **newargs; PyObject *callable; static char *kwlist[] = {"iterable", "key", "reverse", 0}; int reverse; + int nargs; /* args 1-3 should match listsort in Objects/listobject.c */ if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|Oi:sorted", @@ -2107,15 +2108,9 @@ return NULL; } - newargs = PyTuple_GetSlice(args, 1, 4); - if (newargs == NULL) { - Py_DECREF(newlist); - Py_DECREF(callable); - return NULL; - } - - v = PyObject_Call(callable, newargs, kwds); - Py_DECREF(newargs); + newargs = &PyTuple_GET_ITEM(args, 1); + nargs = PyTuple_GET_SIZE(args) - 1; + v = _PyObject_FastCallDict(callable, newargs, nargs, kwds); Py_DECREF(callable); if (v == NULL) { Py_DECREF(newlist); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 17:22:19 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 21:22:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=5FPyFunction=5FFastCallDi?= =?utf-8?q?ct=28=29_supports_keyword_args?= Message-ID: <20160822212218.83402.53309.9C86BAC4@psf.io> https://hg.python.org/cpython/rev/1aefb4c4a7b4 changeset: 102842:1aefb4c4a7b4 user: Victor Stinner date: Mon Aug 22 23:15:44 2016 +0200 summary: _PyFunction_FastCallDict() supports keyword args Issue #27809: * Rename _PyFunction_FastCall() to _PyFunction_FastCallDict() * Rename _PyCFunction_FastCall() to _PyCFunction_FastCallDict() * _PyFunction_FastCallDict() now supports keyword arguments files: Include/funcobject.h | 2 +- Include/methodobject.h | 2 +- Objects/abstract.c | 11 ++--- Objects/methodobject.c | 6 +- Python/ceval.c | 52 ++++++++++++++++++++++++----- 5 files changed, 52 insertions(+), 21 deletions(-) diff --git a/Include/funcobject.h b/Include/funcobject.h --- a/Include/funcobject.h +++ b/Include/funcobject.h @@ -59,7 +59,7 @@ PyAPI_FUNC(int) PyFunction_SetAnnotations(PyObject *, PyObject *); #ifndef Py_LIMITED_API -PyAPI_FUNC(PyObject *) _PyFunction_FastCall( +PyAPI_FUNC(PyObject *) _PyFunction_FastCallDict( PyObject *func, PyObject **args, int nargs, PyObject *kwargs); diff --git a/Include/methodobject.h b/Include/methodobject.h --- a/Include/methodobject.h +++ b/Include/methodobject.h @@ -38,7 +38,7 @@ PyAPI_FUNC(PyObject *) PyCFunction_Call(PyObject *, PyObject *, PyObject *); #ifndef Py_LIMITED_API -PyAPI_FUNC(PyObject *) _PyCFunction_FastCall(PyObject *func, +PyAPI_FUNC(PyObject *) _PyCFunction_FastCallDict(PyObject *func, PyObject **args, int nargs, PyObject *kwargs); #endif diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2255,7 +2255,8 @@ } PyObject * -_PyObject_FastCallDict(PyObject *func, PyObject **args, int nargs, PyObject *kwargs) +_PyObject_FastCallDict(PyObject *func, PyObject **args, int nargs, + PyObject *kwargs) { ternaryfunc call; PyObject *result = NULL; @@ -2268,19 +2269,17 @@ assert(func != NULL); assert(nargs >= 0); assert(nargs == 0 || args != NULL); - /* issue #27128: support for keywords will come later: - _PyFunction_FastCall() doesn't support keyword arguments yet */ - assert(kwargs == NULL); + assert(kwargs == NULL || PyDict_Check(kwargs)); if (Py_EnterRecursiveCall(" while calling a Python object")) { return NULL; } if (PyFunction_Check(func)) { - result = _PyFunction_FastCall(func, args, nargs, kwargs); + result = _PyFunction_FastCallDict(func, args, nargs, kwargs); } else if (PyCFunction_Check(func)) { - result = _PyCFunction_FastCall(func, args, nargs, kwargs); + result = _PyCFunction_FastCallDict(func, args, nargs, kwargs); } else { PyObject *tuple; diff --git a/Objects/methodobject.c b/Objects/methodobject.c --- a/Objects/methodobject.c +++ b/Objects/methodobject.c @@ -146,8 +146,8 @@ } PyObject * -_PyCFunction_FastCall(PyObject *func_obj, PyObject **args, int nargs, - PyObject *kwargs) +_PyCFunction_FastCallDict(PyObject *func_obj, PyObject **args, int nargs, + PyObject *kwargs) { PyCFunctionObject* func = (PyCFunctionObject*)func_obj; PyCFunction meth = PyCFunction_GET_FUNCTION(func); @@ -155,7 +155,7 @@ PyObject *result; int flags; - /* _PyCFunction_FastCall() must not be called with an exception set, + /* _PyCFunction_FastCallDict() must not be called with an exception set, because it may clear it (directly or indirectly) and so the caller loses its exception */ assert(!PyErr_Occurred()); diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4889,24 +4889,29 @@ } PyObject * -_PyFunction_FastCall(PyObject *func, PyObject **args, int nargs, PyObject *kwargs) +_PyFunction_FastCallDict(PyObject *func, PyObject **args, int nargs, + PyObject *kwargs) { PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func); PyObject *globals = PyFunction_GET_GLOBALS(func); PyObject *argdefs = PyFunction_GET_DEFAULTS(func); PyObject *kwdefs, *closure, *name, *qualname; + PyObject *kwtuple, **k; PyObject **d; int nd; + Py_ssize_t nk; + PyObject *result; PCALL(PCALL_FUNCTION); PCALL(PCALL_FAST_FUNCTION); - /* issue #27128: support for keywords will come later */ - assert(kwargs == NULL); - - if (co->co_kwonlyargcount == 0 && kwargs == NULL && + assert(kwargs == NULL || PyDict_Check(kwargs)); + + if (co->co_kwonlyargcount == 0 && + (kwargs == NULL || PyDict_Size(kwargs) == 0) && co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) { + /* Fast paths */ if (argdefs == NULL && co->co_argcount == nargs) { return _PyFunction_FastCallNoKw(co, args, nargs, globals); } @@ -4920,6 +4925,30 @@ } } + if (kwargs != NULL) { + Py_ssize_t pos, i; + nk = PyDict_Size(kwargs); + + kwtuple = PyTuple_New(2 * nk); + if (kwtuple == NULL) { + return NULL; + } + + k = &PyTuple_GET_ITEM(kwtuple, 0); + pos = i = 0; + while (PyDict_Next(kwargs, &pos, &k[i], &k[i+1])) { + Py_INCREF(k[i]); + Py_INCREF(k[i+1]); + i += 2; + } + nk = i / 2; + } + else { + kwtuple = NULL; + k = NULL; + nk = 0; + } + kwdefs = PyFunction_GET_KW_DEFAULTS(func); closure = PyFunction_GET_CLOSURE(func); name = ((PyFunctionObject *)func) -> func_name; @@ -4933,11 +4962,14 @@ d = NULL; nd = 0; } - return _PyEval_EvalCodeWithName((PyObject*)co, globals, (PyObject *)NULL, - args, nargs, - NULL, 0, - d, nd, kwdefs, - closure, name, qualname); + + result = _PyEval_EvalCodeWithName((PyObject*)co, globals, (PyObject *)NULL, + args, nargs, + k, (int)nk, + d, nd, kwdefs, + closure, name, qualname); + Py_XDECREF(kwtuple); + return result; } static PyObject * -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 18:32:46 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 22:32:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327809=3A_tzinfo?= =?utf-8?q?=5Freduce=28=29_uses_fast_call?= Message-ID: <20160822223246.83397.60902.2E4B109B@psf.io> https://hg.python.org/cpython/rev/614dd914c21e changeset: 102849:614dd914c21e user: Victor Stinner date: Tue Aug 23 00:11:04 2016 +0200 summary: Issue #27809: tzinfo_reduce() uses fast call files: Modules/_datetimemodule.c | 24 ++++++++++-------------- 1 files changed, 10 insertions(+), 14 deletions(-) diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -3133,37 +3133,34 @@ static PyObject * tzinfo_reduce(PyObject *self) { - PyObject *args, *state, *tmp; + PyObject *args, *state; PyObject *getinitargs, *getstate; _Py_IDENTIFIER(__getinitargs__); _Py_IDENTIFIER(__getstate__); - tmp = PyTuple_New(0); - if (tmp == NULL) - return NULL; - getinitargs = _PyObject_GetAttrId(self, &PyId___getinitargs__); if (getinitargs != NULL) { - args = PyObject_CallObject(getinitargs, tmp); + args = _PyObject_CallNoArg(getinitargs); Py_DECREF(getinitargs); if (args == NULL) { - Py_DECREF(tmp); return NULL; } } else { PyErr_Clear(); - args = tmp; - Py_INCREF(args); + + args = PyTuple_New(0); + if (args == NULL) { + return NULL; + } } getstate = _PyObject_GetAttrId(self, &PyId___getstate__); if (getstate != NULL) { - state = PyObject_CallObject(getstate, tmp); + state = _PyObject_CallNoArg(getstate); Py_DECREF(getstate); if (state == NULL) { Py_DECREF(args); - Py_DECREF(tmp); return NULL; } } @@ -3172,13 +3169,12 @@ PyErr_Clear(); state = Py_None; dictptr = _PyObject_GetDictPtr(self); - if (dictptr && *dictptr && PyDict_Size(*dictptr)) + if (dictptr && *dictptr && PyDict_Size(*dictptr)) { state = *dictptr; + } Py_INCREF(state); } - Py_DECREF(tmp); - if (state == Py_None) { Py_DECREF(state); return Py_BuildValue("(ON)", Py_TYPE(self), args); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 18:32:45 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 22:32:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_PyEval=5FCallObjectWithKey?= =?utf-8?q?words=28=29_uses_fast_call_with_kwargs?= Message-ID: <20160822223245.14208.39339.F2859C0D@psf.io> https://hg.python.org/cpython/rev/ef0110a52e24 changeset: 102845:ef0110a52e24 user: Victor Stinner date: Mon Aug 22 23:26:00 2016 +0200 summary: PyEval_CallObjectWithKeywords() uses fast call with kwargs Issue #27809. _PyObject_FastCallDict() now supports keyword arguments, and so the args==NULL fast-path can also be used when kwargs is not NULL. files: Python/ceval.c | 16 ++++------------ 1 files changed, 4 insertions(+), 12 deletions(-) diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4590,30 +4590,22 @@ #endif if (args == NULL) { - if (kwargs == NULL) { - return _PyObject_CallNoArg(func); - } - - args = PyTuple_New(0); - if (args == NULL) - return NULL; + return _PyObject_FastCallDict(func, NULL, 0, kwargs); } - else if (!PyTuple_Check(args)) { + + if (!PyTuple_Check(args)) { PyErr_SetString(PyExc_TypeError, "argument list must be a tuple"); return NULL; } - else { - Py_INCREF(args); - } if (kwargs != NULL && !PyDict_Check(kwargs)) { PyErr_SetString(PyExc_TypeError, "keyword list must be a dictionary"); - Py_DECREF(args); return NULL; } + Py_INCREF(args); result = PyObject_Call(func, args, kwargs); Py_DECREF(args); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 18:32:46 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 22:32:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_=5FPyErr=5FCreateExcep?= =?utf-8?b?dGlvbigp?= Message-ID: <20160822223245.15730.18400.62572F3D@psf.io> https://hg.python.org/cpython/rev/0c65a2089f00 changeset: 102847:0c65a2089f00 user: Victor Stinner date: Mon Aug 22 23:59:08 2016 +0200 summary: Add _PyErr_CreateException() Issue #27809: Helper function optimized to create an exception: use fastcall whenever possible. files: Python/errors.c | 61 ++++++++++++++++++------------------ 1 files changed, 30 insertions(+), 31 deletions(-) diff --git a/Python/errors.c b/Python/errors.c --- a/Python/errors.c +++ b/Python/errors.c @@ -52,6 +52,20 @@ Py_XDECREF(oldtraceback); } +static PyObject* +_PyErr_CreateException(PyObject *exception, PyObject *value) +{ + if (value == NULL || value == Py_None) { + return _PyObject_CallNoArg(exception); + } + else if (PyTuple_Check(value)) { + return PyObject_Call(exception, value, NULL); + } + else { + return _PyObject_CallArg1(exception, value); + } +} + void PyErr_SetObject(PyObject *exception, PyObject *value) { @@ -66,6 +80,7 @@ exception); return; } + Py_XINCREF(value); exc_value = tstate->exc_value; if (exc_value != NULL && exc_value != Py_None) { @@ -73,28 +88,21 @@ Py_INCREF(exc_value); if (value == NULL || !PyExceptionInstance_Check(value)) { /* We must normalize the value right now */ - PyObject *args, *fixed_value; + PyObject *fixed_value; - /* Issue #23571: PyEval_CallObject() must not be called with an + /* Issue #23571: functions must not be called with an exception set */ PyErr_Clear(); - if (value == NULL || value == Py_None) - args = PyTuple_New(0); - else if (PyTuple_Check(value)) { - Py_INCREF(value); - args = value; + fixed_value = _PyErr_CreateException(exception, value); + Py_XDECREF(value); + if (fixed_value == NULL) { + return; } - else - args = PyTuple_Pack(1, value); - fixed_value = args ? - PyEval_CallObject(exception, args) : NULL; - Py_XDECREF(args); - Py_XDECREF(value); - if (fixed_value == NULL) - return; + value = fixed_value; } + /* Avoid reference cycles through the context chain. This is O(chain length) but context chains are usually very short. Sensitive readers may try @@ -110,7 +118,8 @@ o = context; } PyException_SetContext(value, exc_value); - } else { + } + else { Py_DECREF(exc_value); } } @@ -258,25 +267,15 @@ class. */ if (!inclass || !is_subclass) { - PyObject *args, *res; + PyObject *fixed_value; - if (value == Py_None) - args = PyTuple_New(0); - else if (PyTuple_Check(value)) { - Py_INCREF(value); - args = value; + fixed_value = _PyErr_CreateException(type, value); + if (fixed_value == NULL) { + goto finally; } - else - args = PyTuple_Pack(1, value); - if (args == NULL) - goto finally; - res = PyEval_CallObject(type, args); - Py_DECREF(args); - if (res == NULL) - goto finally; Py_DECREF(value); - value = res; + value = fixed_value; } /* if the class of the instance doesn't exactly match the class of the type, believe the instance -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 18:32:46 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 22:32:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327809=3A_=5Fcsv?= =?utf-8?q?=3A_=5Fcall=5Fdialect=28=29_uses_fast_call?= Message-ID: <20160822223246.27150.79936.FA0B1C82@psf.io> https://hg.python.org/cpython/rev/f48ae71e8a8f changeset: 102850:f48ae71e8a8f user: Victor Stinner date: Tue Aug 23 00:21:34 2016 +0200 summary: Issue #27809: _csv: _call_dialect() uses fast call files: Modules/_csv.c | 16 +++++++--------- 1 files changed, 7 insertions(+), 9 deletions(-) diff --git a/Modules/_csv.c b/Modules/_csv.c --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -518,15 +518,13 @@ static PyObject * _call_dialect(PyObject *dialect_inst, PyObject *kwargs) { - PyObject *ctor_args; - PyObject *dialect; - - ctor_args = Py_BuildValue(dialect_inst ? "(O)" : "()", dialect_inst); - if (ctor_args == NULL) - return NULL; - dialect = PyObject_Call((PyObject *)&Dialect_Type, ctor_args, kwargs); - Py_DECREF(ctor_args); - return dialect; + PyObject *type = (PyObject *)&Dialect_Type; + if (dialect_inst) { + return _PyObject_FastCallDict(type, &dialect_inst, 1, kwargs); + } + else { + return _PyObject_FastCallDict(type, NULL, 0, kwargs); + } } /* -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 18:32:47 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 22:32:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327809=3A_methodca?= =?utf-8?q?ller=5Freduce=28=29_uses_fast_call?= Message-ID: <20160822223246.89362.94101.84F5EEA4@psf.io> https://hg.python.org/cpython/rev/60660890459f changeset: 102851:60660890459f user: Victor Stinner date: Tue Aug 23 00:23:23 2016 +0200 summary: Issue #27809: methodcaller_reduce() uses fast call files: Modules/_operator.c | 18 +++++++----------- 1 files changed, 7 insertions(+), 11 deletions(-) diff --git a/Modules/_operator.c b/Modules/_operator.c --- a/Modules/_operator.c +++ b/Modules/_operator.c @@ -1111,6 +1111,8 @@ PyObject *functools; PyObject *partial; PyObject *constructor; + PyObject *newargs[2]; + _Py_IDENTIFIER(partial); functools = PyImport_ImportModule("functools"); if (!functools) @@ -1119,17 +1121,11 @@ Py_DECREF(functools); if (!partial) return NULL; - newargs = PyTuple_New(2); - if (newargs == NULL) { - Py_DECREF(partial); - return NULL; - } - Py_INCREF(Py_TYPE(mc)); - PyTuple_SET_ITEM(newargs, 0, (PyObject *)Py_TYPE(mc)); - Py_INCREF(mc->name); - PyTuple_SET_ITEM(newargs, 1, mc->name); - constructor = PyObject_Call(partial, newargs, mc->kwds); - Py_DECREF(newargs); + + newargs[0] = (PyObject *)Py_TYPE(mc); + newargs[1] = mc->name; + constructor = _PyObject_FastCallDict(partial, newargs, 2, mc->kwds); + Py_DECREF(partial); return Py_BuildValue("NO", constructor, mc->args); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 18:32:46 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 22:32:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327809=3A_PyErr=5F?= =?utf-8?q?SetImportError=28=29_uses_fast_call?= Message-ID: <20160822223246.89521.90353.07D68285@psf.io> https://hg.python.org/cpython/rev/c53c532de995 changeset: 102848:c53c532de995 user: Victor Stinner date: Tue Aug 23 00:04:41 2016 +0200 summary: Issue #27809: PyErr_SetImportError() uses fast call files: Python/errors.c | 26 ++++++++++---------------- 1 files changed, 10 insertions(+), 16 deletions(-) diff --git a/Python/errors.c b/Python/errors.c --- a/Python/errors.c +++ b/Python/errors.c @@ -699,18 +699,14 @@ PyObject * PyErr_SetImportError(PyObject *msg, PyObject *name, PyObject *path) { - PyObject *args, *kwargs, *error; + PyObject *kwargs, *error; - if (msg == NULL) + if (msg == NULL) { return NULL; - - args = PyTuple_New(1); - if (args == NULL) - return NULL; + } kwargs = PyDict_New(); if (kwargs == NULL) { - Py_DECREF(args); return NULL; } @@ -722,22 +718,20 @@ path = Py_None; } - Py_INCREF(msg); - PyTuple_SET_ITEM(args, 0, msg); + if (PyDict_SetItemString(kwargs, "name", name) < 0) { + goto done; + } + if (PyDict_SetItemString(kwargs, "path", path) < 0) { + goto done; + } - if (PyDict_SetItemString(kwargs, "name", name) < 0) - goto done; - if (PyDict_SetItemString(kwargs, "path", path) < 0) - goto done; - - error = PyObject_Call(PyExc_ImportError, args, kwargs); + error = _PyObject_FastCallDict(PyExc_ImportError, &msg, 1, kwargs); if (error != NULL) { PyErr_SetObject((PyObject *)Py_TYPE(error), error); Py_DECREF(error); } done: - Py_DECREF(args); Py_DECREF(kwargs); return NULL; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 18:32:46 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 22:32:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327809=3A_Use_=5FP?= =?utf-8?q?yObject=5FFastCallDict=28=29?= Message-ID: <20160822223245.15671.61653.E2D2A1AC@psf.io> https://hg.python.org/cpython/rev/5587d0dfab4c changeset: 102846:5587d0dfab4c user: Victor Stinner date: Mon Aug 22 23:33:13 2016 +0200 summary: Issue #27809: Use _PyObject_FastCallDict() Modify: * init_subclass() * builtin___build_class__() Fix also a bug in init_subclass(): check for super() failure. files: Objects/typeobject.c | 36 +++++++++++++++---------------- Python/bltinmodule.c | 12 +-------- 2 files changed, 19 insertions(+), 29 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -7007,30 +7007,28 @@ static int init_subclass(PyTypeObject *type, PyObject *kwds) { - PyObject *super, *func, *tmp, *tuple; - - super = PyObject_CallFunctionObjArgs((PyObject *) &PySuper_Type, - type, type, NULL); + PyObject *super, *func, *result; + PyObject *args[2] = {(PyObject *)type, (PyObject *)type}; + + super = _PyObject_FastCall((PyObject *)&PySuper_Type, args, 2); + if (super == NULL) { + return -1; + } + func = _PyObject_GetAttrId(super, &PyId___init_subclass__); Py_DECREF(super); - - if (func == NULL) + if (func == NULL) { return -1; - - tuple = PyTuple_New(0); - if (tuple == NULL) { - Py_DECREF(func); - return 0; - } - - tmp = PyObject_Call(func, tuple, kwds); - Py_DECREF(tuple); + } + + + result = _PyObject_FastCallDict(func, NULL, 0, kwds); Py_DECREF(func); - - if (tmp == NULL) + if (result == NULL) { return -1; - - Py_DECREF(tmp); + } + + Py_DECREF(result); return 0; } diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -155,16 +155,8 @@ } } else { - PyObject *pargs = PyTuple_Pack(2, name, bases); - if (pargs == NULL) { - Py_DECREF(prep); - Py_DECREF(meta); - Py_XDECREF(mkw); - Py_DECREF(bases); - return NULL; - } - ns = PyEval_CallObjectWithKeywords(prep, pargs, mkw); - Py_DECREF(pargs); + PyObject *pargs[2] = {name, bases}; + ns = _PyObject_FastCallDict(prep, pargs, 2, mkw); Py_DECREF(prep); } if (ns == NULL) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 18:32:47 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 22:32:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_PyEval=5FCallObjectWithKey?= =?utf-8?q?words=28=29_doesn=27t_inc/decref?= Message-ID: <20160822223247.1987.48796.BED6D92C@psf.io> https://hg.python.org/cpython/rev/bf2d88cf039a changeset: 102852:bf2d88cf039a user: Victor Stinner date: Tue Aug 23 00:25:01 2016 +0200 summary: PyEval_CallObjectWithKeywords() doesn't inc/decref Issue #27809: PyEval_CallObjectWithKeywords() doesn't increment temporary the reference counter of the args tuple (positional arguments). The caller already holds a strong reference to it. files: Python/ceval.c | 8 +------- 1 files changed, 1 insertions(+), 7 deletions(-) diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4580,8 +4580,6 @@ PyObject * PyEval_CallObjectWithKeywords(PyObject *func, PyObject *args, PyObject *kwargs) { - PyObject *result; - #ifdef Py_DEBUG /* PyEval_CallObjectWithKeywords() must not be called with an exception set. It raises a new exception if parameters are invalid or if @@ -4605,11 +4603,7 @@ return NULL; } - Py_INCREF(args); - result = PyObject_Call(func, args, kwargs); - Py_DECREF(args); - - return result; + return PyObject_Call(func, args, kwargs); } const char * -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Aug 22 19:36:30 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 22 Aug 2016 23:36:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327809=3A_builtin?= =?utf-8?b?X19fYnVpbGRfY2xhc3NfXygpIHVzZXMgZmFzdCBjYWxs?= Message-ID: <20160822233629.130275.3536.C26B274C@psf.io> https://hg.python.org/cpython/rev/620b2e554be4 changeset: 102853:620b2e554be4 user: Victor Stinner date: Tue Aug 23 01:34:35 2016 +0200 summary: Issue #27809: builtin___build_class__() uses fast call files: Python/bltinmodule.c | 8 ++------ 1 files changed, 2 insertions(+), 6 deletions(-) diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -169,12 +169,8 @@ NULL, 0, NULL, 0, NULL, 0, NULL, PyFunction_GET_CLOSURE(func)); if (cell != NULL) { - PyObject *margs; - margs = PyTuple_Pack(3, name, bases, ns); - if (margs != NULL) { - cls = PyEval_CallObjectWithKeywords(meta, margs, mkw); - Py_DECREF(margs); - } + PyObject *margs[3] = {name, bases, ns}; + cls = _PyObject_FastCallDict(meta, margs, 3, mkw); if (cls != NULL && PyCell_Check(cell)) PyCell_Set(cell, cls); Py_DECREF(cell); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 03:43:28 2016 From: python-checkins at python.org (vinay.sajip) Date: Tue, 23 Aug 2016 07:43:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2312713=3A_reverted?= =?utf-8?q?_fix_pending_further_discussion=2E?= Message-ID: <20160823074328.24010.39941.CA5F6382@psf.io> https://hg.python.org/cpython/rev/3310ea7dbe30 changeset: 102854:3310ea7dbe30 user: Vinay Sajip date: Tue Aug 23 08:43:16 2016 +0100 summary: Issue #12713: reverted fix pending further discussion. files: Doc/library/argparse.rst | 12 --- Lib/argparse.py | 23 +----- Lib/test/test_argparse.py | 92 +++++++------------------- Misc/NEWS | 3 - 4 files changed, 32 insertions(+), 98 deletions(-) diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -1668,18 +1668,6 @@ >>> parser.parse_args(['co', 'bar']) Namespace(foo='bar') - argparse supports non-ambiguous abbreviations of subparser names. - - For example, the following three calls are all supported - and do the same thing, as the abbreviations used are not ambiguous:: - - >>> parser.parse_args(['checkout', 'bar']) - Namespace(foo='bar') - >>> parser.parse_args(['check', 'bar']) - Namespace(foo='bar') - >>> parser.parse_args(['che', 'bar']) - Namespace(foo='bar') - One particularly effective way of handling sub-commands is to combine the use of the :meth:`add_subparsers` method with calls to :meth:`set_defaults` so that each subparser knows which Python function it should execute. For diff --git a/Lib/argparse.py b/Lib/argparse.py --- a/Lib/argparse.py +++ b/Lib/argparse.py @@ -1110,12 +1110,6 @@ parser_name = values[0] arg_strings = values[1:] - # get full parser_name from (optional) abbreviated one - for p in self._name_parser_map: - if p.startswith(parser_name): - parser_name = p - break - # set the parser name if requested if self.dest is not SUPPRESS: setattr(namespace, self.dest, parser_name) @@ -2313,18 +2307,11 @@ def _check_value(self, action, value): # converted value must be one of the choices (if specified) - if action.choices is not None: - ac = [ax for ax in action.choices if str(ax).startswith(str(value))] - if len(ac) == 0: - args = {'value': value, - 'choices': ', '.join(map(repr, action.choices))} - msg = _('invalid choice: %(value)r (choose from %(choices)s)') - raise ArgumentError(action, msg % args) - elif len(ac) > 1: - args = {'value': value, - 'choices': ', '.join(ac)} - msg = _('ambiguous choice: %(value)r could match %(choices)s') - raise ArgumentError(action, msg % args) + if action.choices is not None and value not in action.choices: + args = {'value': value, + 'choices': ', '.join(map(repr, action.choices))} + msg = _('invalid choice: %(value)r (choose from %(choices)s)') + raise ArgumentError(action, msg % args) # ======================= # Help-formatting methods diff --git a/Lib/test/test_argparse.py b/Lib/test/test_argparse.py --- a/Lib/test/test_argparse.py +++ b/Lib/test/test_argparse.py @@ -1842,22 +1842,6 @@ parser3.add_argument('t', type=int, help='t help') parser3.add_argument('u', nargs='...', help='u help') - # add fourth sub-parser (to test abbreviations) - parser4_kwargs = dict(description='lost description') - if subparser_help: - parser4_kwargs['help'] = 'lost help' - parser4 = subparsers.add_parser('lost', **parser4_kwargs) - parser4.add_argument('-w', type=int, help='w help') - parser4.add_argument('x', choices='abc', help='x help') - - # add fifth sub-parser, with longer name (to test abbreviations) - parser5_kwargs = dict(description='long description') - if subparser_help: - parser5_kwargs['help'] = 'long help' - parser5 = subparsers.add_parser('long', **parser5_kwargs) - parser5.add_argument('-w', type=int, help='w help') - parser5.add_argument('x', choices='abc', help='x help') - # return the main parser return parser @@ -1873,24 +1857,6 @@ args = args_str.split() self.assertArgumentParserError(self.parser.parse_args, args) - def test_parse_args_abbreviation(self): - # check some non-failure cases: - self.assertEqual( - self.parser.parse_args('0.5 long b -w 7'.split()), - NS(foo=False, bar=0.5, w=7, x='b'), - ) - self.assertEqual( - self.parser.parse_args('0.5 lon b -w 7'.split()), - NS(foo=False, bar=0.5, w=7, x='b'), - ) - self.assertEqual( - self.parser.parse_args('0.5 los b -w 7'.split()), - NS(foo=False, bar=0.5, w=7, x='b'), - ) - # check a failure case: 'lo' is ambiguous - self.assertArgumentParserError(self.parser.parse_args, - '0.5 lo b -w 7'.split()) - def test_parse_args(self): # check some non-failure cases: self.assertEqual( @@ -1943,80 +1909,78 @@ def test_help(self): self.assertEqual(self.parser.format_usage(), - 'usage: PROG [-h] [--foo] bar {1,2,3,lost,long} ...\n') + 'usage: PROG [-h] [--foo] bar {1,2,3} ...\n') self.assertEqual(self.parser.format_help(), textwrap.dedent('''\ - usage: PROG [-h] [--foo] bar {1,2,3,lost,long} ... + usage: PROG [-h] [--foo] bar {1,2,3} ... main description positional arguments: - bar bar help - {1,2,3,lost,long} command help + bar bar help + {1,2,3} command help optional arguments: - -h, --help show this help message and exit - --foo foo help + -h, --help show this help message and exit + --foo foo help ''')) def test_help_extra_prefix_chars(self): # Make sure - is still used for help if it is a non-first prefix char parser = self._get_parser(prefix_chars='+:-') self.assertEqual(parser.format_usage(), - 'usage: PROG [-h] [++foo] bar {1,2,3,lost,long} ...\n') + 'usage: PROG [-h] [++foo] bar {1,2,3} ...\n') self.assertEqual(parser.format_help(), textwrap.dedent('''\ - usage: PROG [-h] [++foo] bar {1,2,3,lost,long} ... + usage: PROG [-h] [++foo] bar {1,2,3} ... main description positional arguments: - bar bar help - {1,2,3,lost,long} command help + bar bar help + {1,2,3} command help optional arguments: - -h, --help show this help message and exit - ++foo foo help + -h, --help show this help message and exit + ++foo foo help ''')) def test_help_alternate_prefix_chars(self): parser = self._get_parser(prefix_chars='+:/') self.assertEqual(parser.format_usage(), - 'usage: PROG [+h] [++foo] bar {1,2,3,lost,long} ...\n') + 'usage: PROG [+h] [++foo] bar {1,2,3} ...\n') self.assertEqual(parser.format_help(), textwrap.dedent('''\ - usage: PROG [+h] [++foo] bar {1,2,3,lost,long} ... + usage: PROG [+h] [++foo] bar {1,2,3} ... main description positional arguments: - bar bar help - {1,2,3,lost,long} command help + bar bar help + {1,2,3} command help optional arguments: - +h, ++help show this help message and exit - ++foo foo help + +h, ++help show this help message and exit + ++foo foo help ''')) def test_parser_command_help(self): self.assertEqual(self.command_help_parser.format_usage(), - 'usage: PROG [-h] [--foo] bar {1,2,3,lost,long} ...\n') + 'usage: PROG [-h] [--foo] bar {1,2,3} ...\n') self.assertEqual(self.command_help_parser.format_help(), textwrap.dedent('''\ - usage: PROG [-h] [--foo] bar {1,2,3,lost,long} ... + usage: PROG [-h] [--foo] bar {1,2,3} ... main description positional arguments: - bar bar help - {1,2,3,lost,long} command help - 1 1 help - 2 2 help - 3 3 help - lost lost help - long long help + bar bar help + {1,2,3} command help + 1 1 help + 2 2 help + 3 3 help optional arguments: - -h, --help show this help message and exit - --foo foo help + -h, --help show this help message and exit + --foo foo help ''')) def test_subparser_title_help(self): @@ -2119,8 +2083,6 @@ 1 help 2 2 help 3 3 help - lost lost help - long long help """)) # ============ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -60,9 +60,6 @@ - Issue #9998: On Linux, ctypes.util.find_library now looks in LD_LIBRARY_PATH for shared libraries. -- Issue #12713: Allowed abbreviation of subcommands by end-users for users of - argparse. - Tests ----- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 06:49:07 2016 From: python-checkins at python.org (martin.panter) Date: Tue, 23 Aug 2016 10:49:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327787=3A_Remove_t?= =?utf-8?q?est=5Fmain=28=29_and_hard-coded_list_of_test_classes?= Message-ID: <20160823104907.20243.25046.ED77C689@psf.io> https://hg.python.org/cpython/rev/1f73355afebb changeset: 102855:1f73355afebb user: Martin Panter date: Tue Aug 23 09:01:43 2016 +0000 summary: Issue #27787: Remove test_main() and hard-coded list of test classes The @reap_threads decorator made the test wait (for up to 1 s) until background threads have finished. Calling join() with a timeout should be equivalent. files: Lib/test/test_httplib.py | 11 ++--------- 1 files changed, 2 insertions(+), 9 deletions(-) diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -940,6 +940,7 @@ thread = threading.Thread(target=run_server) thread.start() + self.addCleanup(thread.join, float(1)) conn = client.HTTPConnection(*serv.getsockname()) conn.request("CONNECT", "dummy:1234") response = conn.getresponse() @@ -953,7 +954,7 @@ finally: response.close() conn.close() - thread.join() + thread.join() self.assertEqual(result, b"proxied data\n") class ExtendedReadTest(TestCase): @@ -1711,13 +1712,5 @@ self.assertIn('header: {}'.format(expected_header), lines) - at support.reap_threads -def test_main(verbose=None): - support.run_unittest(HeaderTests, OfflineTest, BasicTest, TimeoutTest, - PersistenceTest, - HTTPSTest, RequestBodyTest, SourceAddressTest, - HTTPResponseTest, ExtendedReadTest, - ExtendedReadTestChunked, TunnelTests) - if __name__ == '__main__': test_main() -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Tue Aug 23 08:47:09 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 23 Aug 2016 13:47:09 +0100 Subject: [Python-checkins] UGLY Benchmark Results for Python 2.7 2016-08-23 Message-ID: <08f7d481-8bdc-4e2c-9c3a-a40c83c587c9@irsmsx153.ger.corp.intel.com> No new revisions. Here are the previous results: Results for project Python 2.7, build date 2016-08-23 02:46:46 +0000 commit: 370bbeba21b3 previous commit: 11a9bca71528 revision date: 2016-08-21 09:59:48 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.11% 2.01% 5.56% 3.78% :-) pybench 0.23% 0.50% 6.23% 3.92% :-( regex_v8 0.64% -0.01% -2.20% 10.36% :-) nbody 0.09% -1.11% 7.80% 2.76% :-) json_dump_v2 0.35% 0.73% 3.11% 10.32% :-| normal_startup 0.74% 0.03% -0.26% 1.89% :-) ssbench 0.17% 0.57% 2.99% 0.41% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-2-7-2016-08-23/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Tue Aug 23 08:49:22 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 23 Aug 2016 13:49:22 +0100 Subject: [Python-checkins] UGLY Benchmark Results for Python Default 2016-08-23 Message-ID: Results for project Python default, build date 2016-08-23 02:00:28 +0000 commit: 620b2e554be4 previous commit: ed9cabce9c38 revision date: 2016-08-22 23:34:35 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.22% -2.10% 9.78% 16.93% :-) pybench 0.13% -0.17% 5.81% 8.02% :-( regex_v8 2.59% -0.16% -3.30% 3.76% :-| nbody 0.29% 1.58% 0.88% 9.59% :-| json_dump_v2 0.61% -1.87% -1.72% 12.10% :-| normal_startup 0.63% -0.89% 0.52% 6.07% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-default-2016-08-23/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Tue Aug 23 10:28:11 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 23 Aug 2016 14:28:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327809=3A_partial?= =?utf-8?q?=5Fcall=28=29_uses_fast_call_for_positional_args?= Message-ID: <20160823142801.27150.3888.B6D92ED7@psf.io> https://hg.python.org/cpython/rev/c1a698edfa1b changeset: 102856:c1a698edfa1b user: Victor Stinner date: Tue Aug 23 16:22:35 2016 +0200 summary: Issue #27809: partial_call() uses fast call for positional args files: Modules/_functoolsmodule.c | 40 ++++++++++++++++++------- 1 files changed, 28 insertions(+), 12 deletions(-) diff --git a/Modules/_functoolsmodule.c b/Modules/_functoolsmodule.c --- a/Modules/_functoolsmodule.c +++ b/Modules/_functoolsmodule.c @@ -128,44 +128,60 @@ { PyObject *ret; PyObject *argappl, *kwappl; + PyObject **stack; + Py_ssize_t nargs; assert (PyCallable_Check(pto->fn)); assert (PyTuple_Check(pto->args)); assert (PyDict_Check(pto->kw)); if (PyTuple_GET_SIZE(pto->args) == 0) { - argappl = args; - Py_INCREF(args); - } else if (PyTuple_GET_SIZE(args) == 0) { - argappl = pto->args; - Py_INCREF(pto->args); - } else { + stack = &PyTuple_GET_ITEM(args, 0); + nargs = PyTuple_GET_SIZE(args); + argappl = NULL; + } + else if (PyTuple_GET_SIZE(args) == 0) { + stack = &PyTuple_GET_ITEM(pto->args, 0); + nargs = PyTuple_GET_SIZE(pto->args); + argappl = NULL; + } + else { + stack = NULL; argappl = PySequence_Concat(pto->args, args); - if (argappl == NULL) + if (argappl == NULL) { return NULL; + } + assert(PyTuple_Check(argappl)); } if (PyDict_Size(pto->kw) == 0) { kwappl = kw; Py_XINCREF(kwappl); - } else { + } + else { kwappl = PyDict_Copy(pto->kw); if (kwappl == NULL) { - Py_DECREF(argappl); + Py_XDECREF(argappl); return NULL; } + if (kw != NULL) { if (PyDict_Merge(kwappl, kw, 1) != 0) { - Py_DECREF(argappl); + Py_XDECREF(argappl); Py_DECREF(kwappl); return NULL; } } } - ret = PyObject_Call(pto->fn, argappl, kwappl); - Py_DECREF(argappl); + if (stack) { + ret = _PyObject_FastCallDict(pto->fn, stack, nargs, kwappl); + } + else { + ret = PyObject_Call(pto->fn, argappl, kwappl); + Py_DECREF(argappl); + } Py_XDECREF(kwappl); return ret; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 11:17:10 2016 From: python-checkins at python.org (mark.dickinson) Date: Tue, 23 Aug 2016 15:17:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327832=3A_Make_=5F?= =?utf-8?q?normalize_parameter_to_Fraction=2E=5F=5Finit=5F=5F_keyword-only?= =?utf-8?q?=2E?= Message-ID: <20160823151709.82663.76219.2AAD2C48@psf.io> https://hg.python.org/cpython/rev/ea495a5ded9b changeset: 102857:ea495a5ded9b user: Mark Dickinson date: Tue Aug 23 16:16:52 2016 +0100 summary: Issue #27832: Make _normalize parameter to Fraction.__init__ keyword-only. files: Lib/fractions.py | 2 +- Lib/test/test_fractions.py | 1 + Misc/NEWS | 3 +++ 3 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/fractions.py b/Lib/fractions.py --- a/Lib/fractions.py +++ b/Lib/fractions.py @@ -81,7 +81,7 @@ __slots__ = ('_numerator', '_denominator') # We're immutable, so use __new__ not __init__ - def __new__(cls, numerator=0, denominator=None, _normalize=True): + def __new__(cls, numerator=0, denominator=None, *, _normalize=True): """Constructs a Rational. Takes a string like '3/2' or '1.5', another Rational instance, a diff --git a/Lib/test/test_fractions.py b/Lib/test/test_fractions.py --- a/Lib/test/test_fractions.py +++ b/Lib/test/test_fractions.py @@ -150,6 +150,7 @@ self.assertRaises(TypeError, F, "3/2", 3) self.assertRaises(TypeError, F, 3, 0j) self.assertRaises(TypeError, F, 3, 1j) + self.assertRaises(TypeError, F, 1, 2, 3) @requires_IEEE_754 def testInitFromFloat(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,9 @@ Library ------- +- Issue #27832: Make ``_normalize`` parameter to ``Fraction`` constuctor + keyword-only, so that ``Fraction(2, 3, 4)`` now raises ``TypeError``. + - Issue #27539: Fix unnormalised ``Fraction.__pow__`` result in the case of negative exponent and negative base. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 12:30:46 2016 From: python-checkins at python.org (r.david.murray) Date: Tue, 23 Aug 2016 16:30:46 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIyAyNDY2OiBpc21v?= =?utf-8?q?unt_now_recognizes_mount_points_user_can=27t_access=2E?= Message-ID: <20160823163046.39043.46167.92245BCB@psf.io> https://hg.python.org/cpython/rev/75111791110b changeset: 102858:75111791110b branch: 2.7 parent: 102826:370bbeba21b3 user: R David Murray date: Tue Aug 23 12:30:28 2016 -0400 summary: # 2466: ismount now recognizes mount points user can't access. Patch by Robin Roth, backport by Xiang Zhang. files: Lib/posixpath.py | 2 +- Lib/test/test_posixpath.py | 66 +++++++++++++++++++++++++- Misc/ACKS | 1 + Misc/NEWS | 3 + 4 files changed, 70 insertions(+), 2 deletions(-) diff --git a/Lib/posixpath.py b/Lib/posixpath.py --- a/Lib/posixpath.py +++ b/Lib/posixpath.py @@ -186,7 +186,7 @@ return False try: s1 = os.lstat(path) - s2 = os.lstat(join(path, '..')) + s2 = os.lstat(realpath(join(path, '..'))) except os.error: return False # It doesn't exist -- so not a mount point :-) dev1 = s1.st_dev diff --git a/Lib/test/test_posixpath.py b/Lib/test/test_posixpath.py --- a/Lib/test/test_posixpath.py +++ b/Lib/test/test_posixpath.py @@ -1,4 +1,5 @@ import unittest +from test import symlink_support from test import test_support, test_genericpath from test import test_support as support @@ -7,6 +8,11 @@ import sys from posixpath import realpath, abspath, dirname, basename +try: + import posix +except ImportError: + posix = None + # An absolute path to a temporary filename for testing. We can't rely on TESTFN # being an absolute path, so we need this. @@ -100,7 +106,7 @@ f.write("foo") f.close() self.assertIs(posixpath.islink(test_support.TESTFN + "1"), False) - if hasattr(os, "symlink"): + if symlink_support.can_symlink(): os.symlink(test_support.TESTFN + "1", test_support.TESTFN + "2") self.assertIs(posixpath.islink(test_support.TESTFN + "2"), True) os.remove(test_support.TESTFN + "1") @@ -196,6 +202,64 @@ def test_ismount(self): self.assertIs(posixpath.ismount("/"), True) + def test_ismount_non_existent(self): + # Non-existent mountpoint. + self.assertIs(posixpath.ismount(ABSTFN), False) + try: + os.mkdir(ABSTFN) + self.assertIs(posixpath.ismount(ABSTFN), False) + finally: + safe_rmdir(ABSTFN) + + @symlink_support.skip_unless_symlink + def test_ismount_symlinks(self): + # Symlinks are never mountpoints. + try: + os.symlink("/", ABSTFN) + self.assertIs(posixpath.ismount(ABSTFN), False) + finally: + os.unlink(ABSTFN) + + @unittest.skipIf(posix is None, "Test requires posix module") + def test_ismount_different_device(self): + # Simulate the path being on a different device from its parent by + # mocking out st_dev. + save_lstat = os.lstat + def fake_lstat(path): + st_ino = 0 + st_dev = 0 + if path == ABSTFN: + st_dev = 1 + st_ino = 1 + return posix.stat_result((0, st_ino, st_dev, 0, 0, 0, 0, 0, 0, 0)) + try: + os.lstat = fake_lstat + self.assertIs(posixpath.ismount(ABSTFN), True) + finally: + os.lstat = save_lstat + + @unittest.skipIf(posix is None, "Test requires posix module") + def test_ismount_directory_not_readable(self): + # issue #2466: Simulate ismount run on a directory that is not + # readable, which used to return False. + save_lstat = os.lstat + def fake_lstat(path): + st_ino = 0 + st_dev = 0 + if path.startswith(ABSTFN) and path != ABSTFN: + # ismount tries to read something inside the ABSTFN directory; + # simulate this being forbidden (no read permission). + raise OSError("Fake [Errno 13] Permission denied") + if path == ABSTFN: + st_dev = 1 + st_ino = 1 + return posix.stat_result((0, st_ino, st_dev, 0, 0, 0, 0, 0, 0, 0)) + try: + os.lstat = fake_lstat + self.assertIs(posixpath.ismount(ABSTFN), True) + finally: + os.lstat = save_lstat + def test_expanduser(self): self.assertEqual(posixpath.expanduser("foo"), "foo") with test_support.EnvironmentVarGuard() as env: diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1186,6 +1186,7 @@ Just van Rossum Hugo van Rossum Saskia van Rossum +Robin Roth Clement Rouault Donald Wallace Rouse II Liam Routt diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -33,6 +33,9 @@ Library ------- +- Issue #2466: posixpath.ismount now correctly recognizes mount points which + the user does not have permission to access. + - Issue #27783: Fix possible usage of uninitialized memory in operator.methodcaller. - Issue #27774: Fix possible Py_DECREF on unowned object in _sre. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 12:34:04 2016 From: python-checkins at python.org (mark.dickinson) Date: Tue, 23 Aug 2016 16:34:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326040_=28part_1?= =?utf-8?q?=29=3A_add_new_testcases_to_cmath=5Ftestcases=2Etxt=2E_Thanks_J?= =?utf-8?q?eff?= Message-ID: <20160823163404.67844.27727.D011A5F4@psf.io> https://hg.python.org/cpython/rev/1017215f5492 changeset: 102859:1017215f5492 parent: 102857:ea495a5ded9b user: Mark Dickinson date: Tue Aug 23 17:33:54 2016 +0100 summary: Issue #26040 (part 1): add new testcases to cmath_testcases.txt. Thanks Jeff Allen. files: Lib/test/cmath_testcases.txt | 141 +++++++++++++++++++++++ Misc/NEWS | 3 + 2 files changed, 144 insertions(+), 0 deletions(-) diff --git a/Lib/test/cmath_testcases.txt b/Lib/test/cmath_testcases.txt --- a/Lib/test/cmath_testcases.txt +++ b/Lib/test/cmath_testcases.txt @@ -53,6 +53,12 @@ -- MPFR homepage at http://www.mpfr.org for more information about the -- MPFR project. +-- A minority of the test cases were generated with the help of +-- mpmath 0.19 at 100 bit accuracy (http://mpmath.org) to improve +-- coverage of real functions with real-valued arguments. These are +-- used in test.test_math.MathTests.test_testfile, as well as in +-- test_cmath. + -------------------------- -- acos: Inverse cosine -- @@ -848,6 +854,18 @@ atan0303 atan -1e-165 1.0 -> -0.78539816339744828 190.30984376228875 atan0304 atan -9.9998886718268301e-321 -1.0 -> -0.78539816339744828 -368.76019403576692 +-- Additional real values (mpmath) +atan0400 atan 1.7976931348623157e+308 0.0 -> 1.5707963267948966192 0.0 +atan0401 atan -1.7976931348623157e+308 0.0 -> -1.5707963267948966192 0.0 +atan0402 atan 1e-17 0.0 -> 1.0000000000000000715e-17 0.0 +atan0403 atan -1e-17 0.0 -> -1.0000000000000000715e-17 0.0 +atan0404 atan 0.0001 0.0 -> 0.000099999999666666673459 0.0 +atan0405 atan -0.0001 0.0 -> -0.000099999999666666673459 0.0 +atan0406 atan 0.999999999999999 0.0 -> 0.78539816339744781002 0.0 +atan0407 atan 1.000000000000001 0.0 -> 0.78539816339744886473 0.0 +atan0408 atan 14.101419947171719 0.0 -> 1.4999999999999999969 0.0 +atan0409 atan 1255.7655915007897 0.0 -> 1.5700000000000000622 0.0 + -- special values atan1000 atan -0.0 0.0 -> -0.0 0.0 atan1001 atan nan 0.0 -> nan 0.0 @@ -1514,6 +1532,11 @@ sqrt0140 sqrt 1.6999999999999999e+308 -1.6999999999999999e+308 -> 1.4325088230154573e+154 -5.9336458271212207e+153 sqrt0141 sqrt -1.797e+308 -9.9999999999999999e+306 -> 3.7284476432057307e+152 -1.3410406899802901e+154 +-- Additional real values (mpmath) +sqrt0150 sqrt 1.7976931348623157e+308 0.0 -> 1.3407807929942596355e+154 0.0 +sqrt0151 sqrt 2.2250738585072014e-308 0.0 -> 1.4916681462400413487e-154 0.0 +sqrt0152 sqrt 5e-324 0.0 -> 2.2227587494850774834e-162 0.0 + -- special values sqrt1000 sqrt 0.0 0.0 -> 0.0 0.0 sqrt1001 sqrt -0.0 0.0 -> 0.0 0.0 @@ -1616,6 +1639,20 @@ exp0053 exp 710.0 1.6 -> -6.5231579995501372e+306 inf overflow exp0054 exp 710.0 2.8 -> -inf 7.4836177417448528e+307 overflow +-- Additional real values (mpmath) +exp0070 exp 1e-08 0.0 -> 1.00000001000000005 0.0 +exp0071 exp 0.0003 0.0 -> 1.0003000450045003375 0.0 +exp0072 exp 0.2 0.0 -> 1.2214027581601698475 0.0 +exp0073 exp 1.0 0.0 -> 2.7182818284590452354 0.0 +exp0074 exp -1e-08 0.0 -> 0.99999999000000005 0.0 +exp0075 exp -0.0003 0.0 -> 0.99970004499550033751 0.0 +exp0076 exp -1.0 0.0 -> 0.3678794411714423216 0.0 +exp0077 exp 2.220446049250313e-16 0.0 -> 1.000000000000000222 0.0 +exp0078 exp -1.1102230246251565e-16 0.0 -> 0.99999999999999988898 0.0 +exp0079 exp 2.302585092994046 0.0 -> 10.000000000000002171 0.0 +exp0080 exp -2.302585092994046 0.0 -> 0.099999999999999978292 0.0 +exp0081 exp 709.7827 0.0 -> 1.7976699566638014654e+308 0.0 + -- special values exp1000 exp 0.0 0.0 -> 1.0 0.0 exp1001 exp -0.0 0.0 -> 1.0 0.0 @@ -1708,6 +1745,23 @@ cosh0030 cosh 710.5 2.3519999999999999 -> -1.2967465239355998e+308 1.3076707908857333e+308 cosh0031 cosh -710.5 0.69999999999999996 -> 1.4085466381392499e+308 -1.1864024666450239e+308 +-- Additional real values (mpmath) +cosh0050 cosh 1e-150 0.0 -> 1.0 0.0 +cosh0051 cosh 1e-18 0.0 -> 1.0 0.0 +cosh0052 cosh 1e-09 0.0 -> 1.0000000000000000005 0.0 +cosh0053 cosh 0.0003 0.0 -> 1.0000000450000003375 0.0 +cosh0054 cosh 0.2 0.0 -> 1.0200667556190758485 0.0 +cosh0055 cosh 1.0 0.0 -> 1.5430806348152437785 0.0 +cosh0056 cosh -1e-18 0.0 -> 1.0 -0.0 +cosh0057 cosh -0.0003 0.0 -> 1.0000000450000003375 -0.0 +cosh0058 cosh -1.0 0.0 -> 1.5430806348152437785 -0.0 +cosh0059 cosh 1.3169578969248168 0.0 -> 2.0000000000000001504 0.0 +cosh0060 cosh -1.3169578969248168 0.0 -> 2.0000000000000001504 -0.0 +cosh0061 cosh 17.328679513998633 0.0 -> 16777216.000000021938 0.0 +cosh0062 cosh 18.714973875118524 0.0 -> 67108864.000000043662 0.0 +cosh0063 cosh 709.7827 0.0 -> 8.9883497833190073272e+307 0.0 +cosh0064 cosh -709.7827 0.0 -> 8.9883497833190073272e+307 -0.0 + -- special values cosh1000 cosh 0.0 0.0 -> 1.0 0.0 cosh1001 cosh 0.0 inf -> nan 0.0 invalid ignore-imag-sign @@ -1800,6 +1854,24 @@ sinh0030 sinh 710.5 -2.3999999999999999 -> -1.3579970564885919e+308 -1.24394470907798e+308 sinh0031 sinh -710.5 0.80000000000000004 -> -1.2830671601735164e+308 1.3210954193997678e+308 +-- Additional real values (mpmath) +sinh0050 sinh 1e-100 0.0 -> 1.00000000000000002e-100 0.0 +sinh0051 sinh 5e-17 0.0 -> 4.9999999999999998955e-17 0.0 +sinh0052 sinh 1e-16 0.0 -> 9.999999999999999791e-17 0.0 +sinh0053 sinh 3.7e-08 0.0 -> 3.7000000000000008885e-8 0.0 +sinh0054 sinh 0.001 0.0 -> 0.0010000001666666750208 0.0 +sinh0055 sinh 0.2 0.0 -> 0.20133600254109399895 0.0 +sinh0056 sinh 1.0 0.0 -> 1.1752011936438014569 0.0 +sinh0057 sinh -3.7e-08 0.0 -> -3.7000000000000008885e-8 0.0 +sinh0058 sinh -0.001 0.0 -> -0.0010000001666666750208 0.0 +sinh0059 sinh -1.0 0.0 -> -1.1752011936438014569 0.0 +sinh0060 sinh 1.4436354751788103 0.0 -> 1.9999999999999999078 0.0 +sinh0061 sinh -1.4436354751788103 0.0 -> -1.9999999999999999078 0.0 +sinh0062 sinh 17.328679513998633 0.0 -> 16777215.999999992136 0.0 +sinh0063 sinh 18.714973875118524 0.0 -> 67108864.000000036211 0.0 +sinh0064 sinh 709.7827 0.0 -> 8.9883497833190073272e+307 0.0 +sinh0065 sinh -709.7827 0.0 -> -8.9883497833190073272e+307 0.0 + -- special values sinh1000 sinh 0.0 0.0 -> 0.0 0.0 sinh1001 sinh 0.0 inf -> 0.0 nan invalid ignore-real-sign @@ -1897,6 +1969,24 @@ tanh0032 tanh 1000 -2.3199999999999998 -> 1.0 0.0 tanh0033 tanh -1.0000000000000001e+300 -9.6699999999999999 -> -1.0 -0.0 +-- Additional real values (mpmath) +tanh0050 tanh 1e-100 0.0 -> 1.00000000000000002e-100 0.0 +tanh0051 tanh 5e-17 0.0 -> 4.9999999999999998955e-17 0.0 +tanh0052 tanh 1e-16 0.0 -> 9.999999999999999791e-17 0.0 +tanh0053 tanh 3.7e-08 0.0 -> 3.6999999999999983559e-8 0.0 +tanh0054 tanh 0.001 0.0 -> 0.00099999966666680002076 0.0 +tanh0055 tanh 0.2 0.0 -> 0.19737532022490401141 0.0 +tanh0056 tanh 1.0 0.0 -> 0.76159415595576488812 0.0 +tanh0057 tanh -3.7e-08 0.0 -> -3.6999999999999983559e-8 0.0 +tanh0058 tanh -0.001 0.0 -> -0.00099999966666680002076 0.0 +tanh0059 tanh -1.0 0.0 -> -0.76159415595576488812 0.0 +tanh0060 tanh 0.5493061443340549 0.0 -> 0.50000000000000003402 0.0 +tanh0061 tanh -0.5493061443340549 0.0 -> -0.50000000000000003402 0.0 +tanh0062 tanh 17.328679513998633 0.0 -> 0.99999999999999822364 0.0 +tanh0063 tanh 18.714973875118524 0.0 -> 0.99999999999999988898 0.0 +tanh0064 tanh 711 0.0 -> 1.0 0.0 +tanh0065 tanh 1.797e+308 0.0 -> 1.0 0.0 + --special values tanh1000 tanh 0.0 0.0 -> 0.0 0.0 tanh1001 tanh 0.0 inf -> nan nan invalid @@ -1985,6 +2075,22 @@ cos0022 cos 7.9914515433858515 0.71659966615501436 -> -0.17375439906936566 -0.77217043527294582 cos0023 cos 0.45124351152540226 1.6992693993812158 -> 2.543477948972237 -1.1528193694875477 +-- Additional real values (mpmath) +cos0050 cos 1e-150 0.0 -> 1.0 -0.0 +cos0051 cos 1e-18 0.0 -> 1.0 -0.0 +cos0052 cos 1e-09 0.0 -> 0.9999999999999999995 -0.0 +cos0053 cos 0.0003 0.0 -> 0.9999999550000003375 -0.0 +cos0054 cos 0.2 0.0 -> 0.98006657784124162892 -0.0 +cos0055 cos 1.0 0.0 -> 0.5403023058681397174 -0.0 +cos0056 cos -1e-18 0.0 -> 1.0 0.0 +cos0057 cos -0.0003 0.0 -> 0.9999999550000003375 0.0 +cos0058 cos -1.0 0.0 -> 0.5403023058681397174 0.0 +cos0059 cos 1.0471975511965976 0.0 -> 0.50000000000000009945 -0.0 +cos0060 cos 2.5707963267948966 0.0 -> -0.84147098480789647357 -0.0 +cos0061 cos -2.5707963267948966 0.0 -> -0.84147098480789647357 0.0 +cos0062 cos 7.225663103256523 0.0 -> 0.58778525229247407559 -0.0 +cos0063 cos -8.79645943005142 0.0 -> -0.80901699437494722255 0.0 + -- special values cos1000 cos -0.0 0.0 -> 1.0 0.0 cos1001 cos -inf 0.0 -> nan 0.0 invalid ignore-imag-sign @@ -2073,6 +2179,22 @@ sin0022 sin 1.1518087354403725 4.8597235966150558 -> 58.919141989603041 26.237003403758852 sin0023 sin 0.00087773078406649192 34.792379211312095 -> 565548145569.38245 644329685822700.62 +-- Additional real values (mpmath) +sin0050 sin 1e-100 0.0 -> 1.00000000000000002e-100 0.0 +sin0051 sin 3.7e-08 0.0 -> 3.6999999999999992001e-8 0.0 +sin0052 sin 0.001 0.0 -> 0.00099999983333334168748 0.0 +sin0053 sin 0.2 0.0 -> 0.19866933079506122634 0.0 +sin0054 sin 1.0 0.0 -> 0.84147098480789650665 0.0 +sin0055 sin -3.7e-08 0.0 -> -3.6999999999999992001e-8 0.0 +sin0056 sin -0.001 0.0 -> -0.00099999983333334168748 0.0 +sin0057 sin -1.0 0.0 -> -0.84147098480789650665 0.0 +sin0058 sin 0.5235987755982989 0.0 -> 0.50000000000000004642 0.0 +sin0059 sin -0.5235987755982989 0.0 -> -0.50000000000000004642 0.0 +sin0060 sin 2.6179938779914944 0.0 -> 0.49999999999999996018 -0.0 +sin0061 sin -2.6179938779914944 0.0 -> -0.49999999999999996018 -0.0 +sin0062 sin 7.225663103256523 0.0 -> 0.80901699437494673648 0.0 +sin0063 sin -8.79645943005142 0.0 -> -0.58778525229247340658 -0.0 + -- special values sin1000 sin -0.0 0.0 -> -0.0 0.0 sin1001 sin -inf 0.0 -> nan 0.0 invalid ignore-imag-sign @@ -2161,6 +2283,25 @@ tan0022 tan 1.1615313900880577 1.7956298728647107 -> 0.041793186826390362 1.0375339546034792 tan0023 tan 0.067014779477908945 5.8517361577457097 -> 2.2088639754800034e-06 0.9999836182420061 +-- Additional real values (mpmath) +tan0050 tan 1e-100 0.0 -> 1.00000000000000002e-100 0.0 +tan0051 tan 3.7e-08 0.0 -> 3.7000000000000017328e-8 0.0 +tan0052 tan 0.001 0.0 -> 0.0010000003333334666875 0.0 +tan0053 tan 0.2 0.0 -> 0.20271003550867249488 0.0 +tan0054 tan 1.0 0.0 -> 1.5574077246549022305 0.0 +tan0055 tan -3.7e-08 0.0 -> -3.7000000000000017328e-8 0.0 +tan0056 tan -0.001 0.0 -> -0.0010000003333334666875 0.0 +tan0057 tan -1.0 0.0 -> -1.5574077246549022305 0.0 +tan0058 tan 0.4636476090008061 0.0 -> 0.49999999999999997163 0.0 +tan0059 tan -0.4636476090008061 0.0 -> -0.49999999999999997163 0.0 +tan0060 tan 1.1071487177940904 0.0 -> 1.9999999999999995298 0.0 +tan0061 tan -1.1071487177940904 0.0 -> -1.9999999999999995298 0.0 +tan0062 tan 1.5 0.0 -> 14.101419947171719388 0.0 +tan0063 tan 1.57 0.0 -> 1255.7655915007896475 0.0 +tan0064 tan 1.5707963267948961 0.0 -> 1978937966095219.0538 0.0 +tan0065 tan 7.225663103256523 0.0 -> 1.3763819204711701522 0.0 +tan0066 tan -8.79645943005142 0.0 -> 0.7265425280053614098 0.0 + -- special values tan1000 tan -0.0 0.0 -> -0.0 0.0 tan1001 tan -inf 0.0 -> nan nan invalid diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -66,6 +66,9 @@ Tests ----- +- Issue #26040: Improve math and cmath test coverage and rigour. Patch by + Jeff Allen. + - Issue #27787: Call gc.collect() before checking each test for "dangling threads", since the dangling threads are weak references. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 12:39:45 2016 From: python-checkins at python.org (guido.van.rossum) Date: Tue, 23 Aug 2016 16:39:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_In_asyncio=2Elocks=2ELock=2Eacquire=28=29=3A_Avoid_deadl?= =?utf-8?q?ock_when_a_cancelled_future_is_in?= Message-ID: <20160823163945.2191.66103.223CDBCB@psf.io> https://hg.python.org/cpython/rev/2339eb860cba changeset: 102861:2339eb860cba parent: 102859:1017215f5492 parent: 102860:e3466a556d81 user: Guido van Rossum date: Tue Aug 23 09:39:26 2016 -0700 summary: In asyncio.locks.Lock.acquire(): Avoid deadlock when a cancelled future is in self._waiters. (Merge 3.5->3.6) files: Lib/asyncio/locks.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/locks.py b/Lib/asyncio/locks.py --- a/Lib/asyncio/locks.py +++ b/Lib/asyncio/locks.py @@ -166,7 +166,7 @@ This method blocks until the lock is unlocked, then sets it to locked and returns True. """ - if not self._waiters and not self._locked: + if not self._locked and all(w.cancelled() for w in self._waiters): self._locked = True return True -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 12:39:45 2016 From: python-checkins at python.org (guido.van.rossum) Date: Tue, 23 Aug 2016 16:39:45 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSW4gYXN5bmNpby5s?= =?utf-8?q?ocks=2ELock=2Eacquire=28=29=3A_Avoid_deadlock_when_a_cancelled_?= =?utf-8?q?future_is_in?= Message-ID: <20160823163945.1773.24498.82A3095C@psf.io> https://hg.python.org/cpython/rev/e3466a556d81 changeset: 102860:e3466a556d81 branch: 3.5 parent: 102839:51627344fc58 user: Guido van Rossum date: Tue Aug 23 09:39:03 2016 -0700 summary: In asyncio.locks.Lock.acquire(): Avoid deadlock when a cancelled future is in self._waiters. files: Lib/asyncio/locks.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/locks.py b/Lib/asyncio/locks.py --- a/Lib/asyncio/locks.py +++ b/Lib/asyncio/locks.py @@ -166,7 +166,7 @@ This method blocks until the lock is unlocked, then sets it to locked and returns True. """ - if not self._waiters and not self._locked: + if not self._locked and all(w.cancelled() for w in self._waiters): self._locked = True return True -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 12:41:00 2016 From: python-checkins at python.org (steven.daprano) Date: Tue, 23 Aug 2016 16:41:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_documentation_for_geom?= =?utf-8?q?etric_and_harmonic_means=2E?= Message-ID: <20160823164100.1819.5449.8C3C37F8@psf.io> https://hg.python.org/cpython/rev/2c02284f2ce8 changeset: 102863:2c02284f2ce8 user: Steven D'Aprano date: Wed Aug 24 02:34:25 2016 +1000 summary: Add documentation for geometric and harmonic means. files: Doc/library/statistics.rst | 58 ++++++++++++++++++++++++++ 1 files changed, 58 insertions(+), 0 deletions(-) diff --git a/Doc/library/statistics.rst b/Doc/library/statistics.rst --- a/Doc/library/statistics.rst +++ b/Doc/library/statistics.rst @@ -39,6 +39,8 @@ ======================= ============================================= :func:`mean` Arithmetic mean ("average") of data. +:func:`geometric_mean` Geometric mean of data. +:func:`harmonic_mean` Harmonic mean of data. :func:`median` Median (middle value) of data. :func:`median_low` Low median of data. :func:`median_high` High median of data. @@ -111,6 +113,62 @@ ``mean(data)`` is equivalent to calculating the true population mean ?. +.. function:: geometric_mean(data) + + Return the geometric mean of *data*, a sequence or iterator of + real-valued numbers. + + The geometric mean is the *n*-th root of the product of *n* data points. + It is a type of average, a measure of the central location of the data. + + The geometric mean is appropriate when averaging quantities which + are multiplied together rather than added, for example growth rates. + Suppose an investment grows by 10% in the first year, falls by 5% in + the second, then grows by 12% in the third, what is the average rate + of growth over the three years? + + .. doctest:: + + >>> geometric_mean([1.10, 0.95, 1.12]) + 1.0538483123382172 + + giving an average growth of 5.385%. Using the arithmetic mean will + give approximately 5.667%, which is too high. + + :exc:``StatisticsError`` is raised if *data* is empty, or any + element is less than zero. + + +.. function:: harmonic_mean(data) + + Return the harmonic mean of *data*, a sequence or iterator of + real-valued numbers. + + The harmonic mean, sometimes called the subcontrary mean, is the + reciprocal of the arithmetic :func:``mean`` of the reciprocals of the + data. For example, the harmonic mean of three values *a*, *b* and *c* + will be equivalent to ``3/(1/a + 1/b + 1/c)``. + + The harmonic mean is a type of average, a measure of the central + location of the data. It is often appropriate when averaging quantities + which are rates or ratios, for example speeds. For example: + + Suppose an investor purchases an equal value of shares in each of + three companies, with P/E (price/earning) ratios of 2.5, 3 and 10. + What is the average P/E ratio for the investor's portfolio? + + .. doctest:: + + >>> harmonic_mean([2.5, 3, 10]) # For an equal investment portfolio. + 3.6 + + Using the arithmetic mean would give an average of about 5.167, which + is too high. + + :exc:``StatisticsError`` is raised if *data* is empty, or any element + is less than zero. + + .. function:: median(data) Return the median (middle value) of numeric data, using the common "mean of -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 12:41:00 2016 From: python-checkins at python.org (steven.daprano) Date: Tue, 23 Aug 2016 16:41:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327573_make_the_ex?= =?utf-8?q?it_message_configurable=2E?= Message-ID: <20160823164100.45820.6891.87880A54@psf.io> https://hg.python.org/cpython/rev/c4863530cfd6 changeset: 102862:c4863530cfd6 parent: 102856:c1a698edfa1b user: Steven D'Aprano date: Wed Aug 24 01:42:15 2016 +1000 summary: Issue #27573 make the exit message configurable. files: Doc/library/code.rst | 18 +++++++++++++----- Lib/code.py | 17 +++++++++++++---- Lib/test/test_code_module.py | 20 ++++++++++++++++++++ 3 files changed, 46 insertions(+), 9 deletions(-) diff --git a/Doc/library/code.rst b/Doc/library/code.rst --- a/Doc/library/code.rst +++ b/Doc/library/code.rst @@ -30,15 +30,19 @@ ``sys.ps1`` and ``sys.ps2``, and input buffering. -.. function:: interact(banner=None, readfunc=None, local=None) +.. function:: interact(banner=None, readfunc=None, local=None, exitmsg=None) Convenience function to run a read-eval-print loop. This creates a new instance of :class:`InteractiveConsole` and sets *readfunc* to be used as the :meth:`InteractiveConsole.raw_input` method, if provided. If *local* is provided, it is passed to the :class:`InteractiveConsole` constructor for use as the default namespace for the interpreter loop. The :meth:`interact` - method of the instance is then run with *banner* passed as the banner to - use, if provided. The console object is discarded after use. + method of the instance is then run with *banner* and *exitmsg* passed as the + banner and exit message to use, if provided. The console object is discarded + after use. + + .. versionchanged:: 3.6 + Added *exitmsg* parameter. .. function:: compile_command(source, filename="", symbol="single") @@ -136,7 +140,7 @@ interpreter objects as well as the following additions. -.. method:: InteractiveConsole.interact(banner=None) +.. method:: InteractiveConsole.interact(banner=None, exitmsg=None) Closely emulate the interactive Python console. The optional *banner* argument specify the banner to print before the first interaction; by default it prints a @@ -144,11 +148,15 @@ by the class name of the console object in parentheses (so as not to confuse this with the real interpreter -- since it's so close!). + The optional *exitmsg* argument specifies an exit message printed when exiting. + Pass the empty string to suppress the exit message. If *exitmsg* is not given or + None, a default message is printed. + .. versionchanged:: 3.4 To suppress printing any banner, pass an empty string. .. versionchanged:: 3.6 - Now prints a brief message when exiting. + Print an exit message when exiting. .. method:: InteractiveConsole.push(line) diff --git a/Lib/code.py b/Lib/code.py --- a/Lib/code.py +++ b/Lib/code.py @@ -186,7 +186,7 @@ """Reset the input buffer.""" self.buffer = [] - def interact(self, banner=None): + def interact(self, banner=None, exitmsg=None): """Closely emulate the interactive Python console. The optional banner argument specifies the banner to print @@ -196,6 +196,11 @@ to confuse this with the real interpreter -- since it's so close!). + The optional exitmsg argument specifies the exit message + printed when exiting. Pass the empty string to suppress + printing an exit message. If exitmsg is not given or None, + a default message is printed. + """ try: sys.ps1 @@ -230,7 +235,10 @@ self.write("\nKeyboardInterrupt\n") self.resetbuffer() more = 0 - self.write('now exiting %s...\n' % self.__class__.__name__) + if exitmsg is None: + self.write('now exiting %s...\n' % self.__class__.__name__) + elif exitmsg != '': + self.write('%s\n' % exitmsg) def push(self, line): """Push a line to the interpreter. @@ -268,7 +276,7 @@ -def interact(banner=None, readfunc=None, local=None): +def interact(banner=None, readfunc=None, local=None, exitmsg=None): """Closely emulate the interactive Python interpreter. This is a backwards compatible interface to the InteractiveConsole @@ -280,6 +288,7 @@ banner -- passed to InteractiveConsole.interact() readfunc -- if not None, replaces InteractiveConsole.raw_input() local -- passed to InteractiveInterpreter.__init__() + exitmsg -- passed to InteractiveConsole.interact() """ console = InteractiveConsole(local) @@ -290,7 +299,7 @@ import readline except ImportError: pass - console.interact(banner) + console.interact(banner, exitmsg) if __name__ == "__main__": diff --git a/Lib/test/test_code_module.py b/Lib/test/test_code_module.py --- a/Lib/test/test_code_module.py +++ b/Lib/test/test_code_module.py @@ -80,6 +80,7 @@ self.assertEqual(len(self.stderr.method_calls), 2) def test_exit_msg(self): + # default exit message self.infunc.side_effect = EOFError('Finished') self.console.interact(banner='') self.assertEqual(len(self.stderr.method_calls), 2) @@ -87,6 +88,25 @@ expected = 'now exiting InteractiveConsole...\n' self.assertEqual(err_msg, ['write', (expected,), {}]) + # no exit message + self.stderr.reset_mock() + self.infunc.side_effect = EOFError('Finished') + self.console.interact(banner='', exitmsg='') + self.assertEqual(len(self.stderr.method_calls), 1) + + # custom exit message + self.stderr.reset_mock() + message = ( + 'bye! \N{GREEK SMALL LETTER ZETA}\N{CYRILLIC SMALL LETTER ZHE}' + ) + self.infunc.side_effect = EOFError('Finished') + self.console.interact(banner='', exitmsg=message) + self.assertEqual(len(self.stderr.method_calls), 2) + err_msg = self.stderr.method_calls[1] + expected = message + '\n' + self.assertEqual(err_msg, ['write', (expected,), {}]) + + def test_cause_tb(self): self.infunc.side_effect = ["raise ValueError('') from AttributeError", EOFError('Finished')] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 12:41:00 2016 From: python-checkins at python.org (steven.daprano) Date: Tue, 23 Aug 2016 16:41:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Re-licence_statistics=2Epy?= =?utf-8?q?_under_the_standard_Python_licence=2E?= Message-ID: <20160823164100.27166.2063.357B247D@psf.io> https://hg.python.org/cpython/rev/f7375e89b424 changeset: 102864:f7375e89b424 user: Steven D'Aprano date: Wed Aug 24 02:40:03 2016 +1000 summary: Re-licence statistics.py under the standard Python licence. files: Lib/statistics.py | 17 ----------------- 1 files changed, 0 insertions(+), 17 deletions(-) diff --git a/Lib/statistics.py b/Lib/statistics.py --- a/Lib/statistics.py +++ b/Lib/statistics.py @@ -1,20 +1,3 @@ -## Module statistics.py -## -## Copyright (c) 2013 Steven D'Aprano . -## -## Licensed under the Apache License, Version 2.0 (the "License"); -## you may not use this file except in compliance with the License. -## You may obtain a copy of the License at -## -## http://www.apache.org/licenses/LICENSE-2.0 -## -## Unless required by applicable law or agreed to in writing, software -## distributed under the License is distributed on an "AS IS" BASIS, -## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -## See the License for the specific language governing permissions and -## limitations under the License. - - """ Basic statistics module. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 12:41:01 2016 From: python-checkins at python.org (steven.daprano) Date: Tue, 23 Aug 2016 16:41:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Automated_merge_with_ssh=3A//hg=2Epython=2Eorg/cpython?= Message-ID: <20160823164101.45723.2292.A452B791@psf.io> https://hg.python.org/cpython/rev/d43a77b6f865 changeset: 102865:d43a77b6f865 parent: 102861:2339eb860cba parent: 102864:f7375e89b424 user: Steven D'Aprano date: Wed Aug 24 02:40:31 2016 +1000 summary: Automated merge with ssh://hg.python.org/cpython files: Doc/library/code.rst | 18 +++++-- Doc/library/statistics.rst | 58 ++++++++++++++++++++++++ Lib/code.py | 17 +++++- Lib/statistics.py | 17 ------- Lib/test/test_code_module.py | 20 ++++++++ 5 files changed, 104 insertions(+), 26 deletions(-) diff --git a/Doc/library/code.rst b/Doc/library/code.rst --- a/Doc/library/code.rst +++ b/Doc/library/code.rst @@ -30,15 +30,19 @@ ``sys.ps1`` and ``sys.ps2``, and input buffering. -.. function:: interact(banner=None, readfunc=None, local=None) +.. function:: interact(banner=None, readfunc=None, local=None, exitmsg=None) Convenience function to run a read-eval-print loop. This creates a new instance of :class:`InteractiveConsole` and sets *readfunc* to be used as the :meth:`InteractiveConsole.raw_input` method, if provided. If *local* is provided, it is passed to the :class:`InteractiveConsole` constructor for use as the default namespace for the interpreter loop. The :meth:`interact` - method of the instance is then run with *banner* passed as the banner to - use, if provided. The console object is discarded after use. + method of the instance is then run with *banner* and *exitmsg* passed as the + banner and exit message to use, if provided. The console object is discarded + after use. + + .. versionchanged:: 3.6 + Added *exitmsg* parameter. .. function:: compile_command(source, filename="", symbol="single") @@ -136,7 +140,7 @@ interpreter objects as well as the following additions. -.. method:: InteractiveConsole.interact(banner=None) +.. method:: InteractiveConsole.interact(banner=None, exitmsg=None) Closely emulate the interactive Python console. The optional *banner* argument specify the banner to print before the first interaction; by default it prints a @@ -144,11 +148,15 @@ by the class name of the console object in parentheses (so as not to confuse this with the real interpreter -- since it's so close!). + The optional *exitmsg* argument specifies an exit message printed when exiting. + Pass the empty string to suppress the exit message. If *exitmsg* is not given or + None, a default message is printed. + .. versionchanged:: 3.4 To suppress printing any banner, pass an empty string. .. versionchanged:: 3.6 - Now prints a brief message when exiting. + Print an exit message when exiting. .. method:: InteractiveConsole.push(line) diff --git a/Doc/library/statistics.rst b/Doc/library/statistics.rst --- a/Doc/library/statistics.rst +++ b/Doc/library/statistics.rst @@ -39,6 +39,8 @@ ======================= ============================================= :func:`mean` Arithmetic mean ("average") of data. +:func:`geometric_mean` Geometric mean of data. +:func:`harmonic_mean` Harmonic mean of data. :func:`median` Median (middle value) of data. :func:`median_low` Low median of data. :func:`median_high` High median of data. @@ -111,6 +113,62 @@ ``mean(data)`` is equivalent to calculating the true population mean ?. +.. function:: geometric_mean(data) + + Return the geometric mean of *data*, a sequence or iterator of + real-valued numbers. + + The geometric mean is the *n*-th root of the product of *n* data points. + It is a type of average, a measure of the central location of the data. + + The geometric mean is appropriate when averaging quantities which + are multiplied together rather than added, for example growth rates. + Suppose an investment grows by 10% in the first year, falls by 5% in + the second, then grows by 12% in the third, what is the average rate + of growth over the three years? + + .. doctest:: + + >>> geometric_mean([1.10, 0.95, 1.12]) + 1.0538483123382172 + + giving an average growth of 5.385%. Using the arithmetic mean will + give approximately 5.667%, which is too high. + + :exc:``StatisticsError`` is raised if *data* is empty, or any + element is less than zero. + + +.. function:: harmonic_mean(data) + + Return the harmonic mean of *data*, a sequence or iterator of + real-valued numbers. + + The harmonic mean, sometimes called the subcontrary mean, is the + reciprocal of the arithmetic :func:``mean`` of the reciprocals of the + data. For example, the harmonic mean of three values *a*, *b* and *c* + will be equivalent to ``3/(1/a + 1/b + 1/c)``. + + The harmonic mean is a type of average, a measure of the central + location of the data. It is often appropriate when averaging quantities + which are rates or ratios, for example speeds. For example: + + Suppose an investor purchases an equal value of shares in each of + three companies, with P/E (price/earning) ratios of 2.5, 3 and 10. + What is the average P/E ratio for the investor's portfolio? + + .. doctest:: + + >>> harmonic_mean([2.5, 3, 10]) # For an equal investment portfolio. + 3.6 + + Using the arithmetic mean would give an average of about 5.167, which + is too high. + + :exc:``StatisticsError`` is raised if *data* is empty, or any element + is less than zero. + + .. function:: median(data) Return the median (middle value) of numeric data, using the common "mean of diff --git a/Lib/code.py b/Lib/code.py --- a/Lib/code.py +++ b/Lib/code.py @@ -186,7 +186,7 @@ """Reset the input buffer.""" self.buffer = [] - def interact(self, banner=None): + def interact(self, banner=None, exitmsg=None): """Closely emulate the interactive Python console. The optional banner argument specifies the banner to print @@ -196,6 +196,11 @@ to confuse this with the real interpreter -- since it's so close!). + The optional exitmsg argument specifies the exit message + printed when exiting. Pass the empty string to suppress + printing an exit message. If exitmsg is not given or None, + a default message is printed. + """ try: sys.ps1 @@ -230,7 +235,10 @@ self.write("\nKeyboardInterrupt\n") self.resetbuffer() more = 0 - self.write('now exiting %s...\n' % self.__class__.__name__) + if exitmsg is None: + self.write('now exiting %s...\n' % self.__class__.__name__) + elif exitmsg != '': + self.write('%s\n' % exitmsg) def push(self, line): """Push a line to the interpreter. @@ -268,7 +276,7 @@ -def interact(banner=None, readfunc=None, local=None): +def interact(banner=None, readfunc=None, local=None, exitmsg=None): """Closely emulate the interactive Python interpreter. This is a backwards compatible interface to the InteractiveConsole @@ -280,6 +288,7 @@ banner -- passed to InteractiveConsole.interact() readfunc -- if not None, replaces InteractiveConsole.raw_input() local -- passed to InteractiveInterpreter.__init__() + exitmsg -- passed to InteractiveConsole.interact() """ console = InteractiveConsole(local) @@ -290,7 +299,7 @@ import readline except ImportError: pass - console.interact(banner) + console.interact(banner, exitmsg) if __name__ == "__main__": diff --git a/Lib/statistics.py b/Lib/statistics.py --- a/Lib/statistics.py +++ b/Lib/statistics.py @@ -1,20 +1,3 @@ -## Module statistics.py -## -## Copyright (c) 2013 Steven D'Aprano . -## -## Licensed under the Apache License, Version 2.0 (the "License"); -## you may not use this file except in compliance with the License. -## You may obtain a copy of the License at -## -## http://www.apache.org/licenses/LICENSE-2.0 -## -## Unless required by applicable law or agreed to in writing, software -## distributed under the License is distributed on an "AS IS" BASIS, -## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -## See the License for the specific language governing permissions and -## limitations under the License. - - """ Basic statistics module. diff --git a/Lib/test/test_code_module.py b/Lib/test/test_code_module.py --- a/Lib/test/test_code_module.py +++ b/Lib/test/test_code_module.py @@ -80,6 +80,7 @@ self.assertEqual(len(self.stderr.method_calls), 2) def test_exit_msg(self): + # default exit message self.infunc.side_effect = EOFError('Finished') self.console.interact(banner='') self.assertEqual(len(self.stderr.method_calls), 2) @@ -87,6 +88,25 @@ expected = 'now exiting InteractiveConsole...\n' self.assertEqual(err_msg, ['write', (expected,), {}]) + # no exit message + self.stderr.reset_mock() + self.infunc.side_effect = EOFError('Finished') + self.console.interact(banner='', exitmsg='') + self.assertEqual(len(self.stderr.method_calls), 1) + + # custom exit message + self.stderr.reset_mock() + message = ( + 'bye! \N{GREEK SMALL LETTER ZETA}\N{CYRILLIC SMALL LETTER ZHE}' + ) + self.infunc.side_effect = EOFError('Finished') + self.console.interact(banner='', exitmsg=message) + self.assertEqual(len(self.stderr.method_calls), 2) + err_msg = self.stderr.method_calls[1] + expected = message + '\n' + self.assertEqual(err_msg, ['write', (expected,), {}]) + + def test_cause_tb(self): self.infunc.side_effect = ["raise ValueError('') from AttributeError", EOFError('Finished')] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 13:47:22 2016 From: python-checkins at python.org (guido.van.rossum) Date: Tue, 23 Aug 2016 17:47:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_27598=3A_Add_Collect?= =?utf-8?q?ions_to_collections=2Eabc=2E?= Message-ID: <20160823174722.20475.26647.CF7E1AE5@psf.io> https://hg.python.org/cpython/rev/acd9a465b373 changeset: 102866:acd9a465b373 user: Guido van Rossum date: Tue Aug 23 10:47:07 2016 -0700 summary: Issue 27598: Add Collections to collections.abc. Patch by Ivan Levkivskyi, docs by Neil Girdhar. files: Doc/library/collections.abc.rst | 26 +++-- Lib/_collections_abc.py | 17 +++- Lib/test/test_collections.py | 90 ++++++++++++++++++++- Lib/test/test_functools.py | 17 ++- Misc/NEWS | 3 + 5 files changed, 132 insertions(+), 21 deletions(-) diff --git a/Doc/library/collections.abc.rst b/Doc/library/collections.abc.rst --- a/Doc/library/collections.abc.rst +++ b/Doc/library/collections.abc.rst @@ -45,10 +45,12 @@ :class:`Generator` :class:`Iterator` ``send``, ``throw`` ``close``, ``__iter__``, ``__next__`` :class:`Sized` ``__len__`` :class:`Callable` ``__call__`` +:class:`Collection` :class:`Sized`, ``__contains__``, + :class:`Iterable`, ``__iter__``, + :class:`Container` ``__len__`` -:class:`Sequence` :class:`Sized`, ``__getitem__``, ``__contains__``, ``__iter__``, ``__reversed__``, - :class:`Reversible`, ``__len__`` ``index``, and ``count`` - :class:`Container` +:class:`Sequence` :class:`Reversible`, ``__getitem__``, ``__contains__``, ``__iter__``, ``__reversed__``, + :class:`Collection` ``__len__`` ``index``, and ``count`` :class:`MutableSequence` :class:`Sequence` ``__getitem__``, Inherited :class:`Sequence` methods and ``__setitem__``, ``append``, ``reverse``, ``extend``, ``pop``, @@ -59,9 +61,9 @@ :class:`ByteString` :class:`Sequence` ``__getitem__``, Inherited :class:`Sequence` methods ``__len__`` -:class:`Set` :class:`Sized`, ``__contains__``, ``__le__``, ``__lt__``, ``__eq__``, ``__ne__``, - :class:`Iterable`, ``__iter__``, ``__gt__``, ``__ge__``, ``__and__``, ``__or__``, - :class:`Container` ``__len__`` ``__sub__``, ``__xor__``, and ``isdisjoint`` +:class:`Set` :class:`Collection` ``__contains__``, ``__le__``, ``__lt__``, ``__eq__``, ``__ne__``, + ``__iter__``, ``__gt__``, ``__ge__``, ``__and__``, ``__or__``, + ``__len__`` ``__sub__``, ``__xor__``, and ``isdisjoint`` :class:`MutableSet` :class:`Set` ``__contains__``, Inherited :class:`Set` methods and ``__iter__``, ``clear``, ``pop``, ``remove``, ``__ior__``, @@ -69,9 +71,9 @@ ``add``, ``discard`` -:class:`Mapping` :class:`Sized`, ``__getitem__``, ``__contains__``, ``keys``, ``items``, ``values``, - :class:`Iterable`, ``__iter__``, ``get``, ``__eq__``, and ``__ne__`` - :class:`Container` ``__len__`` +:class:`Mapping` :class:`Collection` ``__getitem__``, ``__contains__``, ``keys``, ``items``, ``values``, + ``__iter__``, ``get``, ``__eq__``, and ``__ne__`` + ``__len__`` :class:`MutableMapping` :class:`Mapping` ``__getitem__``, Inherited :class:`Mapping` methods and ``__setitem__``, ``pop``, ``popitem``, ``clear``, ``update``, @@ -106,6 +108,12 @@ ABC for classes that provide the :meth:`__iter__` method. See also the definition of :term:`iterable`. +.. class:: Collection + + ABC for sized iterable container classes. + + .. versionadded:: 3.6 + .. class:: Iterator ABC for classes that provide the :meth:`~iterator.__iter__` and diff --git a/Lib/_collections_abc.py b/Lib/_collections_abc.py --- a/Lib/_collections_abc.py +++ b/Lib/_collections_abc.py @@ -11,7 +11,7 @@ __all__ = ["Awaitable", "Coroutine", "AsyncIterable", "AsyncIterator", "Hashable", "Iterable", "Iterator", "Generator", "Reversible", - "Sized", "Container", "Callable", + "Sized", "Container", "Callable", "Collection", "Set", "MutableSet", "Mapping", "MutableMapping", "MappingView", "KeysView", "ItemsView", "ValuesView", @@ -326,6 +326,15 @@ return _check_methods(C, "__contains__") return NotImplemented +class Collection(Sized, Iterable, Container): + + __slots__ = () + + @classmethod + def __subclasshook__(cls, C): + if cls is Collection: + return _check_methods(C, "__len__", "__iter__", "__contains__") + return NotImplemented class Callable(metaclass=ABCMeta): @@ -345,7 +354,7 @@ ### SETS ### -class Set(Sized, Iterable, Container): +class Set(Collection): """A set is a finite, iterable container. @@ -570,7 +579,7 @@ ### MAPPINGS ### -class Mapping(Sized, Iterable, Container): +class Mapping(Collection): __slots__ = () @@ -794,7 +803,7 @@ ### SEQUENCES ### -class Sequence(Sized, Reversible, Container): +class Sequence(Reversible, Collection): """All the operations on a read-only sequence. diff --git a/Lib/test/test_collections.py b/Lib/test/test_collections.py --- a/Lib/test/test_collections.py +++ b/Lib/test/test_collections.py @@ -21,7 +21,7 @@ from collections import deque from collections.abc import Awaitable, Coroutine, AsyncIterator, AsyncIterable from collections.abc import Hashable, Iterable, Iterator, Generator, Reversible -from collections.abc import Sized, Container, Callable +from collections.abc import Sized, Container, Callable, Collection from collections.abc import Set, MutableSet from collections.abc import Mapping, MutableMapping, KeysView, ItemsView, ValuesView from collections.abc import Sequence, MutableSequence @@ -771,6 +771,94 @@ self.assertFalse(issubclass(RevRevBlocked, Reversible)) self.assertFalse(isinstance(RevRevBlocked(), Reversible)) + def test_Collection(self): + # Check some non-collections + non_collections = [None, 42, 3.14, 1j, lambda x: 2*x] + for x in non_collections: + self.assertNotIsInstance(x, Collection) + self.assertFalse(issubclass(type(x), Collection), repr(type(x))) + # Check some non-collection iterables + non_col_iterables = [_test_gen(), iter(b''), iter(bytearray()), + (x for x in []), dict().values()] + for x in non_col_iterables: + self.assertNotIsInstance(x, Collection) + self.assertFalse(issubclass(type(x), Collection), repr(type(x))) + # Check some collections + samples = [set(), frozenset(), dict(), bytes(), str(), tuple(), + list(), dict().keys(), dict().items()] + for x in samples: + self.assertIsInstance(x, Collection) + self.assertTrue(issubclass(type(x), Collection), repr(type(x))) + # Check also Mapping, MutableMapping, etc. + self.assertTrue(issubclass(Sequence, Collection), repr(Sequence)) + self.assertTrue(issubclass(Mapping, Collection), repr(Mapping)) + self.assertTrue(issubclass(MutableMapping, Collection), + repr(MutableMapping)) + self.assertTrue(issubclass(Set, Collection), repr(Set)) + self.assertTrue(issubclass(MutableSet, Collection), repr(MutableSet)) + self.assertTrue(issubclass(Sequence, Collection), repr(MutableSet)) + # Check direct subclassing + class Col(Collection): + def __iter__(self): + return iter(list()) + def __len__(self): + return 0 + def __contains__(self, item): + return False + class DerCol(Col): pass + self.assertEqual(list(iter(Col())), []) + self.assertFalse(issubclass(list, Col)) + self.assertFalse(issubclass(set, Col)) + self.assertFalse(issubclass(float, Col)) + self.assertEqual(list(iter(DerCol())), []) + self.assertFalse(issubclass(list, DerCol)) + self.assertFalse(issubclass(set, DerCol)) + self.assertFalse(issubclass(float, DerCol)) + self.validate_abstract_methods(Collection, '__len__', '__iter__', + '__contains__') + # Check sized container non-iterable (which is not Collection) etc. + class ColNoIter: + def __len__(self): return 0 + def __contains__(self, item): return False + class ColNoSize: + def __iter__(self): return iter([]) + def __contains__(self, item): return False + class ColNoCont: + def __iter__(self): return iter([]) + def __len__(self): return 0 + self.assertFalse(issubclass(ColNoIter, Collection)) + self.assertFalse(isinstance(ColNoIter(), Collection)) + self.assertFalse(issubclass(ColNoSize, Collection)) + self.assertFalse(isinstance(ColNoSize(), Collection)) + self.assertFalse(issubclass(ColNoCont, Collection)) + self.assertFalse(isinstance(ColNoCont(), Collection)) + # Check None blocking + class SizeBlock: + def __iter__(self): return iter([]) + def __contains__(self): return False + __len__ = None + class IterBlock: + def __len__(self): return 0 + def __contains__(self): return True + __iter__ = None + self.assertFalse(issubclass(SizeBlock, Collection)) + self.assertFalse(isinstance(SizeBlock(), Collection)) + self.assertFalse(issubclass(IterBlock, Collection)) + self.assertFalse(isinstance(IterBlock(), Collection)) + # Check None blocking in subclass + class ColImpl: + def __iter__(self): + return iter(list()) + def __len__(self): + return 0 + def __contains__(self, item): + return False + class NonCol(ColImpl): + __contains__ = None + self.assertFalse(issubclass(NonCol, Collection)) + self.assertFalse(isinstance(NonCol(), Collection)) + + def test_Iterator(self): non_samples = [None, 42, 3.14, 1j, b"", "", (), [], {}, set()] for x in non_samples: diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py --- a/Lib/test/test_functools.py +++ b/Lib/test/test_functools.py @@ -1548,13 +1548,15 @@ bases = [c.Sequence, c.MutableMapping, c.Mapping, c.Set] for haystack in permutations(bases): m = mro(dict, haystack) - self.assertEqual(m, [dict, c.MutableMapping, c.Mapping, c.Sized, - c.Iterable, c.Container, object]) + self.assertEqual(m, [dict, c.MutableMapping, c.Mapping, + c.Collection, c.Sized, c.Iterable, + c.Container, object]) bases = [c.Container, c.Mapping, c.MutableMapping, c.OrderedDict] for haystack in permutations(bases): m = mro(c.ChainMap, haystack) self.assertEqual(m, [c.ChainMap, c.MutableMapping, c.Mapping, - c.Sized, c.Iterable, c.Container, object]) + c.Collection, c.Sized, c.Iterable, + c.Container, object]) # If there's a generic function with implementations registered for # both Sized and Container, passing a defaultdict to it results in an @@ -1575,9 +1577,9 @@ bases = [c.MutableSequence, c.MutableMapping] for haystack in permutations(bases): m = mro(D, bases) - self.assertEqual(m, [D, c.MutableSequence, c.Sequence, - c.defaultdict, dict, c.MutableMapping, - c.Mapping, c.Sized, c.Reversible, c.Iterable, c.Container, + self.assertEqual(m, [D, c.MutableSequence, c.Sequence, c.Reversible, + c.defaultdict, dict, c.MutableMapping, c.Mapping, + c.Collection, c.Sized, c.Iterable, c.Container, object]) # Container and Callable are registered on different base classes and @@ -1590,7 +1592,8 @@ for haystack in permutations(bases): m = mro(C, haystack) self.assertEqual(m, [C, c.Callable, c.defaultdict, dict, c.Mapping, - c.Sized, c.Iterable, c.Container, object]) + c.Collection, c.Sized, c.Iterable, + c.Container, object]) def test_register_abc(self): c = collections diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -126,6 +126,9 @@ Library ------- +- Issue 27598: Add Collections to collections.abc. + Patch by Ivan Levkivskyi, docs by Neil Girdhar. + - Issue #25958: Support "anti-registration" of special methods from various ABCs, like __hash__, __iter__ or __len__. All these (and several more) can be set to None in an implementation class and the -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 14:06:49 2016 From: python-checkins at python.org (guido.van.rossum) Date: Tue, 23 Aug 2016 18:06:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_A_new_version_?= =?utf-8?q?of_typing=2Epy_from_https=3A//github=2Ecom/python/typing=2E?= Message-ID: <20160823180649.20228.98731.7A61BCDA@psf.io> https://hg.python.org/cpython/rev/355fa8bd8d9d changeset: 102867:355fa8bd8d9d branch: 3.5 parent: 102860:e3466a556d81 user: Guido van Rossum date: Tue Aug 23 11:01:50 2016 -0700 summary: A new version of typing.py from https://github.com/python/typing. files: Lib/test/test_typing.py | 13 +++++- Lib/typing.py | 68 ++++++++++++++++++---------- Misc/NEWS | 6 ++ 3 files changed, 61 insertions(+), 26 deletions(-) diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -512,6 +512,10 @@ self.assertEqual(get_type_hints(foo, globals(), locals()), {'a': Callable[..., T]}) + def test_ellipsis_in_generic(self): + # Shouldn't crash; see https://github.com/python/typing/issues/259 + typing.List[Callable[..., str]] + XK = TypeVar('XK', str, bytes) XV = TypeVar('XV') @@ -852,7 +856,7 @@ def test_covariance_sequence(self): # Check covariance for Sequence (which is just a generic class - # for this purpose, but using a covariant type variable). + # for this purpose, but using a type variable with covariant=True). self.assertIsSubclass(typing.Sequence[Manager], typing.Sequence[Employee]) self.assertNotIsSubclass(typing.Sequence[Employee], @@ -1185,6 +1189,13 @@ self.assertIsInstance([], typing.Container) self.assertNotIsInstance(42, typing.Container) + def test_collection(self): + if hasattr(typing, 'Collection'): + self.assertIsInstance(tuple(), typing.Collection) + self.assertIsInstance(frozenset(), typing.Collection) + self.assertIsSubclass(dict, typing.Collection) + self.assertNotIsInstance(42, typing.Collection) + def test_abstractset(self): self.assertIsInstance(set(), typing.AbstractSet) self.assertNotIsInstance(42, typing.AbstractSet) diff --git a/Lib/typing.py b/Lib/typing.py --- a/Lib/typing.py +++ b/Lib/typing.py @@ -57,6 +57,7 @@ 'DefaultDict', 'List', 'Set', + 'FrozenSet', 'NamedTuple', # Not really a type. 'Generator', @@ -160,12 +161,6 @@ return self def _eval_type(self, globalns, localns): - if not isinstance(localns, dict): - raise TypeError('ForwardRef localns must be a dict -- got %r' % - (localns,)) - if not isinstance(globalns, dict): - raise TypeError('ForwardRef globalns must be a dict -- got %r' % - (globalns,)) if not self.__forward_evaluated__: if globalns is None and localns is None: globalns = localns = {} @@ -388,9 +383,10 @@ and issubclass(bytes, A) are true, and issubclass(int, A) is false. (TODO: Why is this needed? This may change. See #136.) - Type variables may be marked covariant or contravariant by passing - covariant=True or contravariant=True. See PEP 484 for more - details. By default type variables are invariant. + Type variables defined with covariant=True or contravariant=True + can be used do declare covariant or contravariant generic types. + See PEP 484 for more details. By default generic types are invariant + in all type variables. Type variables can be introspected. e.g.: @@ -405,7 +401,7 @@ covariant=False, contravariant=False): self = super().__new__(cls, name, (Final,), {}, _root=True) if covariant and contravariant: - raise ValueError("Bivariant type variables are not supported.") + raise ValueError("Bivariant types are not supported.") self.__covariant__ = bool(covariant) self.__contravariant__ = bool(contravariant) if constraints and bound is not None: @@ -782,7 +778,7 @@ return self def _get_type_vars(self, tvars): - if self.__args__: + if self.__args__ and self.__args__ is not Ellipsis: _get_type_vars(self.__args__, tvars) def _eval_type(self, globalns, localns): @@ -1044,7 +1040,7 @@ if cls is Any: return True if isinstance(cls, GenericMeta): - # For a class C(Generic[T]) where T is co-variant, + # For a covariant class C(Generic[T]), # C[X] is a subclass of C[Y] iff X is a subclass of Y. origin = self.__origin__ if origin is not None and origin is cls.__origin__: @@ -1434,31 +1430,53 @@ __slots__ = () +if hasattr(collections_abc, 'Collection'): + class Collection(Sized, Iterable[T_co], Container[T_co], + extra=collections_abc.Collection): + __slots__ = () + + __all__.append('Collection') + + # Callable was defined earlier. - -class AbstractSet(Sized, Iterable[T_co], Container[T_co], - extra=collections_abc.Set): - pass +if hasattr(collections_abc, 'Collection'): + class AbstractSet(Collection[T_co], + extra=collections_abc.Set): + pass +else: + class AbstractSet(Sized, Iterable[T_co], Container[T_co], + extra=collections_abc.Set): + pass class MutableSet(AbstractSet[T], extra=collections_abc.MutableSet): pass -# NOTE: Only the value type is covariant. -class Mapping(Sized, Iterable[KT], Container[KT], Generic[KT, VT_co], - extra=collections_abc.Mapping): - pass +# NOTE: It is only covariant in the value type. +if hasattr(collections_abc, 'Collection'): + class Mapping(Collection[KT], Generic[KT, VT_co], + extra=collections_abc.Mapping): + pass +else: + class Mapping(Sized, Iterable[KT], Container[KT], Generic[KT, VT_co], + extra=collections_abc.Mapping): + pass class MutableMapping(Mapping[KT, VT], extra=collections_abc.MutableMapping): pass if hasattr(collections_abc, 'Reversible'): - class Sequence(Sized, Reversible[T_co], Container[T_co], - extra=collections_abc.Sequence): - pass + if hasattr(collections_abc, 'Collection'): + class Sequence(Reversible[T_co], Collection[T_co], + extra=collections_abc.Sequence): + pass + else: + class Sequence(Sized, Reversible[T_co], Container[T_co], + extra=collections_abc.Sequence): + pass else: class Sequence(Sized, Iterable[T_co], Container[T_co], extra=collections_abc.Sequence): @@ -1583,11 +1601,11 @@ # Internal type variable used for Type[]. -CT = TypeVar('CT', covariant=True, bound=type) +CT_co = TypeVar('CT_co', covariant=True, bound=type) # This is not a real generic class. Don't use outside annotations. -class Type(type, Generic[CT], extra=type): +class Type(type, Generic[CT_co], extra=type): """A special construct usable to annotate class objects. For example, suppose we have the following classes:: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -50,6 +50,12 @@ Library ------- +- A new version of typing.py from https://github.com/python/typing: + - Collection (only for 3.6) (Issue #27598) + - Add FrozenSet to __all__ (upstream #261) + - fix crash in _get_type_vars() (upstream #259) + - Remove the dict constraint in ForwardRef._eval_type (upstream #252) + - Issue #27539: Fix unnormalised ``Fraction.__pow__`` result in the case of negative exponent and negative base. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 14:06:49 2016 From: python-checkins at python.org (guido.van.rossum) Date: Tue, 23 Aug 2016 18:06:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_A_new_version_of_typing=2Epy_from_https=3A//github=2Ecom?= =?utf-8?q?/python/typing=2E_=28Merge?= Message-ID: <20160823180649.27029.46569.7F1677BF@psf.io> https://hg.python.org/cpython/rev/013bb690c24a changeset: 102868:013bb690c24a parent: 102866:acd9a465b373 parent: 102867:355fa8bd8d9d user: Guido van Rossum date: Tue Aug 23 11:06:30 2016 -0700 summary: A new version of typing.py from https://github.com/python/typing. (Merge 3.5->3.6) files: Lib/test/test_typing.py | 13 +++++- Lib/typing.py | 68 ++++++++++++++++++---------- Misc/NEWS | 6 ++ 3 files changed, 61 insertions(+), 26 deletions(-) diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -512,6 +512,10 @@ self.assertEqual(get_type_hints(foo, globals(), locals()), {'a': Callable[..., T]}) + def test_ellipsis_in_generic(self): + # Shouldn't crash; see https://github.com/python/typing/issues/259 + typing.List[Callable[..., str]] + XK = TypeVar('XK', str, bytes) XV = TypeVar('XV') @@ -852,7 +856,7 @@ def test_covariance_sequence(self): # Check covariance for Sequence (which is just a generic class - # for this purpose, but using a covariant type variable). + # for this purpose, but using a type variable with covariant=True). self.assertIsSubclass(typing.Sequence[Manager], typing.Sequence[Employee]) self.assertNotIsSubclass(typing.Sequence[Employee], @@ -1185,6 +1189,13 @@ self.assertIsInstance([], typing.Container) self.assertNotIsInstance(42, typing.Container) + def test_collection(self): + if hasattr(typing, 'Collection'): + self.assertIsInstance(tuple(), typing.Collection) + self.assertIsInstance(frozenset(), typing.Collection) + self.assertIsSubclass(dict, typing.Collection) + self.assertNotIsInstance(42, typing.Collection) + def test_abstractset(self): self.assertIsInstance(set(), typing.AbstractSet) self.assertNotIsInstance(42, typing.AbstractSet) diff --git a/Lib/typing.py b/Lib/typing.py --- a/Lib/typing.py +++ b/Lib/typing.py @@ -57,6 +57,7 @@ 'DefaultDict', 'List', 'Set', + 'FrozenSet', 'NamedTuple', # Not really a type. 'Generator', @@ -160,12 +161,6 @@ return self def _eval_type(self, globalns, localns): - if not isinstance(localns, dict): - raise TypeError('ForwardRef localns must be a dict -- got %r' % - (localns,)) - if not isinstance(globalns, dict): - raise TypeError('ForwardRef globalns must be a dict -- got %r' % - (globalns,)) if not self.__forward_evaluated__: if globalns is None and localns is None: globalns = localns = {} @@ -388,9 +383,10 @@ and issubclass(bytes, A) are true, and issubclass(int, A) is false. (TODO: Why is this needed? This may change. See #136.) - Type variables may be marked covariant or contravariant by passing - covariant=True or contravariant=True. See PEP 484 for more - details. By default type variables are invariant. + Type variables defined with covariant=True or contravariant=True + can be used do declare covariant or contravariant generic types. + See PEP 484 for more details. By default generic types are invariant + in all type variables. Type variables can be introspected. e.g.: @@ -405,7 +401,7 @@ covariant=False, contravariant=False): self = super().__new__(cls, name, (Final,), {}, _root=True) if covariant and contravariant: - raise ValueError("Bivariant type variables are not supported.") + raise ValueError("Bivariant types are not supported.") self.__covariant__ = bool(covariant) self.__contravariant__ = bool(contravariant) if constraints and bound is not None: @@ -782,7 +778,7 @@ return self def _get_type_vars(self, tvars): - if self.__args__: + if self.__args__ and self.__args__ is not Ellipsis: _get_type_vars(self.__args__, tvars) def _eval_type(self, globalns, localns): @@ -1044,7 +1040,7 @@ if cls is Any: return True if isinstance(cls, GenericMeta): - # For a class C(Generic[T]) where T is co-variant, + # For a covariant class C(Generic[T]), # C[X] is a subclass of C[Y] iff X is a subclass of Y. origin = self.__origin__ if origin is not None and origin is cls.__origin__: @@ -1434,31 +1430,53 @@ __slots__ = () +if hasattr(collections_abc, 'Collection'): + class Collection(Sized, Iterable[T_co], Container[T_co], + extra=collections_abc.Collection): + __slots__ = () + + __all__.append('Collection') + + # Callable was defined earlier. - -class AbstractSet(Sized, Iterable[T_co], Container[T_co], - extra=collections_abc.Set): - pass +if hasattr(collections_abc, 'Collection'): + class AbstractSet(Collection[T_co], + extra=collections_abc.Set): + pass +else: + class AbstractSet(Sized, Iterable[T_co], Container[T_co], + extra=collections_abc.Set): + pass class MutableSet(AbstractSet[T], extra=collections_abc.MutableSet): pass -# NOTE: Only the value type is covariant. -class Mapping(Sized, Iterable[KT], Container[KT], Generic[KT, VT_co], - extra=collections_abc.Mapping): - pass +# NOTE: It is only covariant in the value type. +if hasattr(collections_abc, 'Collection'): + class Mapping(Collection[KT], Generic[KT, VT_co], + extra=collections_abc.Mapping): + pass +else: + class Mapping(Sized, Iterable[KT], Container[KT], Generic[KT, VT_co], + extra=collections_abc.Mapping): + pass class MutableMapping(Mapping[KT, VT], extra=collections_abc.MutableMapping): pass if hasattr(collections_abc, 'Reversible'): - class Sequence(Sized, Reversible[T_co], Container[T_co], - extra=collections_abc.Sequence): - pass + if hasattr(collections_abc, 'Collection'): + class Sequence(Reversible[T_co], Collection[T_co], + extra=collections_abc.Sequence): + pass + else: + class Sequence(Sized, Reversible[T_co], Container[T_co], + extra=collections_abc.Sequence): + pass else: class Sequence(Sized, Iterable[T_co], Container[T_co], extra=collections_abc.Sequence): @@ -1583,11 +1601,11 @@ # Internal type variable used for Type[]. -CT = TypeVar('CT', covariant=True, bound=type) +CT_co = TypeVar('CT_co', covariant=True, bound=type) # This is not a real generic class. Don't use outside annotations. -class Type(type, Generic[CT], extra=type): +class Type(type, Generic[CT_co], extra=type): """A special construct usable to annotate class objects. For example, suppose we have the following classes:: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,12 @@ Library ------- +- A new version of typing.py from https://github.com/python/typing: + - Collection (only for 3.6) (Issue #27598) + - Add FrozenSet to __all__ (upstream #261) + - fix crash in _get_type_vars() (upstream #259) + - Remove the dict constraint in ForwardRef._eval_type (upstream #252) + - Issue #27832: Make ``_normalize`` parameter to ``Fraction`` constuctor keyword-only, so that ``Fraction(2, 3, 4)`` now raises ``TypeError``. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 14:20:53 2016 From: python-checkins at python.org (terry.reedy) Date: Tue, 23 Aug 2016 18:20:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327787=3A_No_longe?= =?utf-8?q?r_call_deleted_test=5Fmain=28=29=2E?= Message-ID: <20160823182052.2059.25873.8C4B63C7@psf.io> https://hg.python.org/cpython/rev/31e4495a34ce changeset: 102869:31e4495a34ce user: Terry Jan Reedy date: Tue Aug 23 14:20:37 2016 -0400 summary: Issue #27787: No longer call deleted test_main(). files: Lib/test/test_httplib.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -1713,4 +1713,4 @@ if __name__ == '__main__': - test_main() + unittest.main(verbosity=2) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 14:24:17 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 23 Aug 2016 18:24:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_markup=2C_add_versiona?= =?utf-8?q?dded_tags?= Message-ID: <20160823182417.27166.38148.F9F76B32@psf.io> https://hg.python.org/cpython/rev/d4c5a7883bbc changeset: 102870:d4c5a7883bbc user: Zachary Ware date: Tue Aug 23 13:23:31 2016 -0500 summary: Fix markup, add versionadded tags files: Doc/library/statistics.rst | 10 +++++++--- 1 files changed, 7 insertions(+), 3 deletions(-) diff --git a/Doc/library/statistics.rst b/Doc/library/statistics.rst --- a/Doc/library/statistics.rst +++ b/Doc/library/statistics.rst @@ -135,9 +135,11 @@ giving an average growth of 5.385%. Using the arithmetic mean will give approximately 5.667%, which is too high. - :exc:``StatisticsError`` is raised if *data* is empty, or any + :exc:`StatisticsError` is raised if *data* is empty, or any element is less than zero. + .. versionadded:: 3.6 + .. function:: harmonic_mean(data) @@ -145,7 +147,7 @@ real-valued numbers. The harmonic mean, sometimes called the subcontrary mean, is the - reciprocal of the arithmetic :func:``mean`` of the reciprocals of the + reciprocal of the arithmetic :func:`mean` of the reciprocals of the data. For example, the harmonic mean of three values *a*, *b* and *c* will be equivalent to ``3/(1/a + 1/b + 1/c)``. @@ -165,9 +167,11 @@ Using the arithmetic mean would give an average of about 5.167, which is too high. - :exc:``StatisticsError`` is raised if *data* is empty, or any element + :exc:`StatisticsError` is raised if *data* is empty, or any element is less than zero. + .. versionadded:: 3.6 + .. function:: median(data) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 14:44:53 2016 From: python-checkins at python.org (alexander.belopolsky) Date: Tue, 23 Aug 2016 18:44:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327834=3A_Avoid_ov?= =?utf-8?q?erflow_error_in_ZoneInfo=2Einvert=28=29=2E?= Message-ID: <20160823184453.20309.54077.F825DF02@psf.io> https://hg.python.org/cpython/rev/a13c51bde2b4 changeset: 102871:a13c51bde2b4 user: Alexander Belopolsky date: Tue Aug 23 14:44:51 2016 -0400 summary: Issue #27834: Avoid overflow error in ZoneInfo.invert(). files: Lib/test/datetimetester.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -4477,11 +4477,11 @@ @staticmethod def invert(ut, ti): - lt = (ut.__copy__(), ut.__copy__()) + lt = (array('q', ut), array('q', ut)) if ut: offset = ti[0][0] // SEC - lt[0][0] = max(-2**31, lt[0][0] + offset) - lt[1][0] = max(-2**31, lt[1][0] + offset) + lt[0][0] += offset + lt[1][0] += offset for i in range(1, len(ut)): lt[0][i] += ti[i-1][0] // SEC lt[1][i] += ti[i][0] // SEC -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 15:01:00 2016 From: python-checkins at python.org (mark.dickinson) Date: Tue, 23 Aug 2016 19:01:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Backed_out_changeset_10172?= =?utf-8?q?15f5492?= Message-ID: <20160823190059.1773.46121.62BF9C04@psf.io> https://hg.python.org/cpython/rev/4389aa3507c5 changeset: 102872:4389aa3507c5 user: Mark Dickinson date: Tue Aug 23 20:00:49 2016 +0100 summary: Backed out changeset 1017215f5492 files: Lib/test/cmath_testcases.txt | 127 ----------------------- Misc/NEWS | 3 - 2 files changed, 0 insertions(+), 130 deletions(-) diff --git a/Lib/test/cmath_testcases.txt b/Lib/test/cmath_testcases.txt --- a/Lib/test/cmath_testcases.txt +++ b/Lib/test/cmath_testcases.txt @@ -53,12 +53,6 @@ -- MPFR homepage at http://www.mpfr.org for more information about the -- MPFR project. --- A minority of the test cases were generated with the help of --- mpmath 0.19 at 100 bit accuracy (http://mpmath.org) to improve --- coverage of real functions with real-valued arguments. These are --- used in test.test_math.MathTests.test_testfile, as well as in --- test_cmath. - -------------------------- -- acos: Inverse cosine -- @@ -854,18 +848,6 @@ atan0303 atan -1e-165 1.0 -> -0.78539816339744828 190.30984376228875 atan0304 atan -9.9998886718268301e-321 -1.0 -> -0.78539816339744828 -368.76019403576692 --- Additional real values (mpmath) -atan0400 atan 1.7976931348623157e+308 0.0 -> 1.5707963267948966192 0.0 -atan0401 atan -1.7976931348623157e+308 0.0 -> -1.5707963267948966192 0.0 -atan0402 atan 1e-17 0.0 -> 1.0000000000000000715e-17 0.0 -atan0403 atan -1e-17 0.0 -> -1.0000000000000000715e-17 0.0 -atan0404 atan 0.0001 0.0 -> 0.000099999999666666673459 0.0 -atan0405 atan -0.0001 0.0 -> -0.000099999999666666673459 0.0 -atan0406 atan 0.999999999999999 0.0 -> 0.78539816339744781002 0.0 -atan0407 atan 1.000000000000001 0.0 -> 0.78539816339744886473 0.0 -atan0408 atan 14.101419947171719 0.0 -> 1.4999999999999999969 0.0 -atan0409 atan 1255.7655915007897 0.0 -> 1.5700000000000000622 0.0 - -- special values atan1000 atan -0.0 0.0 -> -0.0 0.0 atan1001 atan nan 0.0 -> nan 0.0 @@ -1532,11 +1514,6 @@ sqrt0140 sqrt 1.6999999999999999e+308 -1.6999999999999999e+308 -> 1.4325088230154573e+154 -5.9336458271212207e+153 sqrt0141 sqrt -1.797e+308 -9.9999999999999999e+306 -> 3.7284476432057307e+152 -1.3410406899802901e+154 --- Additional real values (mpmath) -sqrt0150 sqrt 1.7976931348623157e+308 0.0 -> 1.3407807929942596355e+154 0.0 -sqrt0151 sqrt 2.2250738585072014e-308 0.0 -> 1.4916681462400413487e-154 0.0 -sqrt0152 sqrt 5e-324 0.0 -> 2.2227587494850774834e-162 0.0 - -- special values sqrt1000 sqrt 0.0 0.0 -> 0.0 0.0 sqrt1001 sqrt -0.0 0.0 -> 0.0 0.0 @@ -1639,20 +1616,6 @@ exp0053 exp 710.0 1.6 -> -6.5231579995501372e+306 inf overflow exp0054 exp 710.0 2.8 -> -inf 7.4836177417448528e+307 overflow --- Additional real values (mpmath) -exp0070 exp 1e-08 0.0 -> 1.00000001000000005 0.0 -exp0071 exp 0.0003 0.0 -> 1.0003000450045003375 0.0 -exp0072 exp 0.2 0.0 -> 1.2214027581601698475 0.0 -exp0073 exp 1.0 0.0 -> 2.7182818284590452354 0.0 -exp0074 exp -1e-08 0.0 -> 0.99999999000000005 0.0 -exp0075 exp -0.0003 0.0 -> 0.99970004499550033751 0.0 -exp0076 exp -1.0 0.0 -> 0.3678794411714423216 0.0 -exp0077 exp 2.220446049250313e-16 0.0 -> 1.000000000000000222 0.0 -exp0078 exp -1.1102230246251565e-16 0.0 -> 0.99999999999999988898 0.0 -exp0079 exp 2.302585092994046 0.0 -> 10.000000000000002171 0.0 -exp0080 exp -2.302585092994046 0.0 -> 0.099999999999999978292 0.0 -exp0081 exp 709.7827 0.0 -> 1.7976699566638014654e+308 0.0 - -- special values exp1000 exp 0.0 0.0 -> 1.0 0.0 exp1001 exp -0.0 0.0 -> 1.0 0.0 @@ -1745,23 +1708,6 @@ cosh0030 cosh 710.5 2.3519999999999999 -> -1.2967465239355998e+308 1.3076707908857333e+308 cosh0031 cosh -710.5 0.69999999999999996 -> 1.4085466381392499e+308 -1.1864024666450239e+308 --- Additional real values (mpmath) -cosh0050 cosh 1e-150 0.0 -> 1.0 0.0 -cosh0051 cosh 1e-18 0.0 -> 1.0 0.0 -cosh0052 cosh 1e-09 0.0 -> 1.0000000000000000005 0.0 -cosh0053 cosh 0.0003 0.0 -> 1.0000000450000003375 0.0 -cosh0054 cosh 0.2 0.0 -> 1.0200667556190758485 0.0 -cosh0055 cosh 1.0 0.0 -> 1.5430806348152437785 0.0 -cosh0056 cosh -1e-18 0.0 -> 1.0 -0.0 -cosh0057 cosh -0.0003 0.0 -> 1.0000000450000003375 -0.0 -cosh0058 cosh -1.0 0.0 -> 1.5430806348152437785 -0.0 -cosh0059 cosh 1.3169578969248168 0.0 -> 2.0000000000000001504 0.0 -cosh0060 cosh -1.3169578969248168 0.0 -> 2.0000000000000001504 -0.0 -cosh0061 cosh 17.328679513998633 0.0 -> 16777216.000000021938 0.0 -cosh0062 cosh 18.714973875118524 0.0 -> 67108864.000000043662 0.0 -cosh0063 cosh 709.7827 0.0 -> 8.9883497833190073272e+307 0.0 -cosh0064 cosh -709.7827 0.0 -> 8.9883497833190073272e+307 -0.0 - -- special values cosh1000 cosh 0.0 0.0 -> 1.0 0.0 cosh1001 cosh 0.0 inf -> nan 0.0 invalid ignore-imag-sign @@ -1854,24 +1800,6 @@ sinh0030 sinh 710.5 -2.3999999999999999 -> -1.3579970564885919e+308 -1.24394470907798e+308 sinh0031 sinh -710.5 0.80000000000000004 -> -1.2830671601735164e+308 1.3210954193997678e+308 --- Additional real values (mpmath) -sinh0050 sinh 1e-100 0.0 -> 1.00000000000000002e-100 0.0 -sinh0051 sinh 5e-17 0.0 -> 4.9999999999999998955e-17 0.0 -sinh0052 sinh 1e-16 0.0 -> 9.999999999999999791e-17 0.0 -sinh0053 sinh 3.7e-08 0.0 -> 3.7000000000000008885e-8 0.0 -sinh0054 sinh 0.001 0.0 -> 0.0010000001666666750208 0.0 -sinh0055 sinh 0.2 0.0 -> 0.20133600254109399895 0.0 -sinh0056 sinh 1.0 0.0 -> 1.1752011936438014569 0.0 -sinh0057 sinh -3.7e-08 0.0 -> -3.7000000000000008885e-8 0.0 -sinh0058 sinh -0.001 0.0 -> -0.0010000001666666750208 0.0 -sinh0059 sinh -1.0 0.0 -> -1.1752011936438014569 0.0 -sinh0060 sinh 1.4436354751788103 0.0 -> 1.9999999999999999078 0.0 -sinh0061 sinh -1.4436354751788103 0.0 -> -1.9999999999999999078 0.0 -sinh0062 sinh 17.328679513998633 0.0 -> 16777215.999999992136 0.0 -sinh0063 sinh 18.714973875118524 0.0 -> 67108864.000000036211 0.0 -sinh0064 sinh 709.7827 0.0 -> 8.9883497833190073272e+307 0.0 -sinh0065 sinh -709.7827 0.0 -> -8.9883497833190073272e+307 0.0 - -- special values sinh1000 sinh 0.0 0.0 -> 0.0 0.0 sinh1001 sinh 0.0 inf -> 0.0 nan invalid ignore-real-sign @@ -1969,24 +1897,6 @@ tanh0032 tanh 1000 -2.3199999999999998 -> 1.0 0.0 tanh0033 tanh -1.0000000000000001e+300 -9.6699999999999999 -> -1.0 -0.0 --- Additional real values (mpmath) -tanh0050 tanh 1e-100 0.0 -> 1.00000000000000002e-100 0.0 -tanh0051 tanh 5e-17 0.0 -> 4.9999999999999998955e-17 0.0 -tanh0052 tanh 1e-16 0.0 -> 9.999999999999999791e-17 0.0 -tanh0053 tanh 3.7e-08 0.0 -> 3.6999999999999983559e-8 0.0 -tanh0054 tanh 0.001 0.0 -> 0.00099999966666680002076 0.0 -tanh0055 tanh 0.2 0.0 -> 0.19737532022490401141 0.0 -tanh0056 tanh 1.0 0.0 -> 0.76159415595576488812 0.0 -tanh0057 tanh -3.7e-08 0.0 -> -3.6999999999999983559e-8 0.0 -tanh0058 tanh -0.001 0.0 -> -0.00099999966666680002076 0.0 -tanh0059 tanh -1.0 0.0 -> -0.76159415595576488812 0.0 -tanh0060 tanh 0.5493061443340549 0.0 -> 0.50000000000000003402 0.0 -tanh0061 tanh -0.5493061443340549 0.0 -> -0.50000000000000003402 0.0 -tanh0062 tanh 17.328679513998633 0.0 -> 0.99999999999999822364 0.0 -tanh0063 tanh 18.714973875118524 0.0 -> 0.99999999999999988898 0.0 -tanh0064 tanh 711 0.0 -> 1.0 0.0 -tanh0065 tanh 1.797e+308 0.0 -> 1.0 0.0 - --special values tanh1000 tanh 0.0 0.0 -> 0.0 0.0 tanh1001 tanh 0.0 inf -> nan nan invalid @@ -2075,22 +1985,6 @@ cos0022 cos 7.9914515433858515 0.71659966615501436 -> -0.17375439906936566 -0.77217043527294582 cos0023 cos 0.45124351152540226 1.6992693993812158 -> 2.543477948972237 -1.1528193694875477 --- Additional real values (mpmath) -cos0050 cos 1e-150 0.0 -> 1.0 -0.0 -cos0051 cos 1e-18 0.0 -> 1.0 -0.0 -cos0052 cos 1e-09 0.0 -> 0.9999999999999999995 -0.0 -cos0053 cos 0.0003 0.0 -> 0.9999999550000003375 -0.0 -cos0054 cos 0.2 0.0 -> 0.98006657784124162892 -0.0 -cos0055 cos 1.0 0.0 -> 0.5403023058681397174 -0.0 -cos0056 cos -1e-18 0.0 -> 1.0 0.0 -cos0057 cos -0.0003 0.0 -> 0.9999999550000003375 0.0 -cos0058 cos -1.0 0.0 -> 0.5403023058681397174 0.0 -cos0059 cos 1.0471975511965976 0.0 -> 0.50000000000000009945 -0.0 -cos0060 cos 2.5707963267948966 0.0 -> -0.84147098480789647357 -0.0 -cos0061 cos -2.5707963267948966 0.0 -> -0.84147098480789647357 0.0 -cos0062 cos 7.225663103256523 0.0 -> 0.58778525229247407559 -0.0 -cos0063 cos -8.79645943005142 0.0 -> -0.80901699437494722255 0.0 - -- special values cos1000 cos -0.0 0.0 -> 1.0 0.0 cos1001 cos -inf 0.0 -> nan 0.0 invalid ignore-imag-sign @@ -2179,22 +2073,6 @@ sin0022 sin 1.1518087354403725 4.8597235966150558 -> 58.919141989603041 26.237003403758852 sin0023 sin 0.00087773078406649192 34.792379211312095 -> 565548145569.38245 644329685822700.62 --- Additional real values (mpmath) -sin0050 sin 1e-100 0.0 -> 1.00000000000000002e-100 0.0 -sin0051 sin 3.7e-08 0.0 -> 3.6999999999999992001e-8 0.0 -sin0052 sin 0.001 0.0 -> 0.00099999983333334168748 0.0 -sin0053 sin 0.2 0.0 -> 0.19866933079506122634 0.0 -sin0054 sin 1.0 0.0 -> 0.84147098480789650665 0.0 -sin0055 sin -3.7e-08 0.0 -> -3.6999999999999992001e-8 0.0 -sin0056 sin -0.001 0.0 -> -0.00099999983333334168748 0.0 -sin0057 sin -1.0 0.0 -> -0.84147098480789650665 0.0 -sin0058 sin 0.5235987755982989 0.0 -> 0.50000000000000004642 0.0 -sin0059 sin -0.5235987755982989 0.0 -> -0.50000000000000004642 0.0 -sin0060 sin 2.6179938779914944 0.0 -> 0.49999999999999996018 -0.0 -sin0061 sin -2.6179938779914944 0.0 -> -0.49999999999999996018 -0.0 -sin0062 sin 7.225663103256523 0.0 -> 0.80901699437494673648 0.0 -sin0063 sin -8.79645943005142 0.0 -> -0.58778525229247340658 -0.0 - -- special values sin1000 sin -0.0 0.0 -> -0.0 0.0 sin1001 sin -inf 0.0 -> nan 0.0 invalid ignore-imag-sign @@ -2283,25 +2161,6 @@ tan0022 tan 1.1615313900880577 1.7956298728647107 -> 0.041793186826390362 1.0375339546034792 tan0023 tan 0.067014779477908945 5.8517361577457097 -> 2.2088639754800034e-06 0.9999836182420061 --- Additional real values (mpmath) -tan0050 tan 1e-100 0.0 -> 1.00000000000000002e-100 0.0 -tan0051 tan 3.7e-08 0.0 -> 3.7000000000000017328e-8 0.0 -tan0052 tan 0.001 0.0 -> 0.0010000003333334666875 0.0 -tan0053 tan 0.2 0.0 -> 0.20271003550867249488 0.0 -tan0054 tan 1.0 0.0 -> 1.5574077246549022305 0.0 -tan0055 tan -3.7e-08 0.0 -> -3.7000000000000017328e-8 0.0 -tan0056 tan -0.001 0.0 -> -0.0010000003333334666875 0.0 -tan0057 tan -1.0 0.0 -> -1.5574077246549022305 0.0 -tan0058 tan 0.4636476090008061 0.0 -> 0.49999999999999997163 0.0 -tan0059 tan -0.4636476090008061 0.0 -> -0.49999999999999997163 0.0 -tan0060 tan 1.1071487177940904 0.0 -> 1.9999999999999995298 0.0 -tan0061 tan -1.1071487177940904 0.0 -> -1.9999999999999995298 0.0 -tan0062 tan 1.5 0.0 -> 14.101419947171719388 0.0 -tan0063 tan 1.57 0.0 -> 1255.7655915007896475 0.0 -tan0064 tan 1.5707963267948961 0.0 -> 1978937966095219.0538 0.0 -tan0065 tan 7.225663103256523 0.0 -> 1.3763819204711701522 0.0 -tan0066 tan -8.79645943005142 0.0 -> 0.7265425280053614098 0.0 - -- special values tan1000 tan -0.0 0.0 -> -0.0 0.0 tan1001 tan -inf 0.0 -> nan nan invalid diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -72,9 +72,6 @@ Tests ----- -- Issue #26040: Improve math and cmath test coverage and rigour. Patch by - Jeff Allen. - - Issue #27787: Call gc.collect() before checking each test for "dangling threads", since the dangling threads are weak references. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 18:09:09 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 23 Aug 2016 22:09:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327809=3A_map=5Fne?= =?utf-8?q?xt=28=29_uses_fast_call?= Message-ID: <20160823220907.45873.80992.0C8610E1@psf.io> https://hg.python.org/cpython/rev/70f88b097f60 changeset: 102873:70f88b097f60 user: Victor Stinner date: Tue Aug 23 17:56:06 2016 +0200 summary: Issue #27809: map_next() uses fast call Use a small stack allocated in the C stack for up to 5 iterator functions, otherwise allocates a stack on the heap memory. files: Python/bltinmodule.c | 50 +++++++++++++++++++++---------- 1 files changed, 33 insertions(+), 17 deletions(-) diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -1156,27 +1156,43 @@ static PyObject * map_next(mapobject *lz) { - PyObject *val; - PyObject *argtuple; - PyObject *result; - Py_ssize_t numargs, i; + PyObject *small_stack[5]; + PyObject **stack; + Py_ssize_t niters, nargs, i; + PyObject *result = NULL; - numargs = PyTuple_GET_SIZE(lz->iters); - argtuple = PyTuple_New(numargs); - if (argtuple == NULL) - return NULL; - - for (i=0 ; iiters, i); - val = Py_TYPE(it)->tp_iternext(it); - if (val == NULL) { - Py_DECREF(argtuple); + niters = PyTuple_GET_SIZE(lz->iters); + if (niters <= (Py_ssize_t)Py_ARRAY_LENGTH(small_stack)) { + stack = small_stack; + } + else { + stack = PyMem_Malloc(niters * sizeof(PyObject*)); + if (stack == NULL) { + PyErr_NoMemory(); return NULL; } - PyTuple_SET_ITEM(argtuple, i, val); } - result = PyObject_Call(lz->func, argtuple, NULL); - Py_DECREF(argtuple); + + nargs = 0; + for (i=0; i < niters; i++) { + PyObject *it = PyTuple_GET_ITEM(lz->iters, i); + PyObject *val = Py_TYPE(it)->tp_iternext(it); + if (val == NULL) { + goto exit; + } + stack[i] = val; + nargs++; + } + + result = _PyObject_FastCall(lz->func, stack, nargs); + +exit: + for (i=0; i < nargs; i++) { + Py_DECREF(stack[i]); + } + if (stack != small_stack) { + PyMem_Free(stack); + } return result; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 18:09:09 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 23 Aug 2016 22:09:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_PyObject=5FCallMethodObjAr?= =?utf-8?q?gs=28=29_now_uses_fast_call?= Message-ID: <20160823220907.113257.39117.5AA5144D@psf.io> https://hg.python.org/cpython/rev/0e4f26083bbb changeset: 102874:0e4f26083bbb user: Victor Stinner date: Wed Aug 24 00:01:56 2016 +0200 summary: PyObject_CallMethodObjArgs() now uses fast call Issue #27809: * PyObject_CallMethodObjArgs(), _PyObject_CallMethodIdObjArgs() and PyObject_CallFunctionObjArgs() now use fast call to avoid the creation of a temporary tuple * Rename objargs_mktuple() to objargs_mkstack() * objargs_mkstack() now stores objects in a C array using borrowed references, instead of storing arguments into a tuple objargs_mkstack() uses a small buffer allocated on the C stack for 5 arguments or less, or allocates a buffer in the heap memory. files: Objects/abstract.c | 113 ++++++++++++++++++++++---------- 1 files changed, 76 insertions(+), 37 deletions(-) diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2499,32 +2499,52 @@ return retval; } -static PyObject * -objargs_mktuple(va_list va) +static PyObject ** +objargs_mkstack(PyObject **small_stack, Py_ssize_t small_stack_size, + va_list va, Py_ssize_t *p_nargs) { - int i, n = 0; + Py_ssize_t i, n; va_list countva; - PyObject *result, *tmp; - - Py_VA_COPY(countva, va); - - while (((PyObject *)va_arg(countva, PyObject *)) != NULL) - ++n; - result = PyTuple_New(n); - if (result != NULL && n > 0) { - for (i = 0; i < n; ++i) { - tmp = (PyObject *)va_arg(va, PyObject *); - PyTuple_SET_ITEM(result, i, tmp); - Py_INCREF(tmp); + PyObject **stack; + + /* Count the number of arguments */ + Py_VA_COPY(countva, va); + + n = 0; + while (1) { + PyObject *arg = (PyObject *)va_arg(countva, PyObject *); + if (arg == NULL) { + break; + } + n++; + } + *p_nargs = n; + + /* Copy arguments */ + if (small_stack_size <= n) { + stack = small_stack; + } + else { + stack = PyMem_Malloc(n * sizeof(PyObject**)); + if (stack == NULL) { + PyErr_NoMemory(); + return NULL; } } - return result; + + for (i = 0; i < n; ++i) { + stack[i] = va_arg(va, PyObject *); + } + return stack; } PyObject * PyObject_CallMethodObjArgs(PyObject *callable, PyObject *name, ...) { - PyObject *args, *tmp; + PyObject *small_stack[5]; + PyObject **stack; + Py_ssize_t nargs; + PyObject *result; va_list vargs; if (callable == NULL || name == NULL) { @@ -2537,24 +2557,31 @@ /* count the args */ va_start(vargs, name); - args = objargs_mktuple(vargs); + stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack), + vargs, &nargs); va_end(vargs); - if (args == NULL) { + if (stack == NULL) { Py_DECREF(callable); return NULL; } - tmp = PyObject_Call(callable, args, NULL); - Py_DECREF(args); + + result = _PyObject_FastCall(callable, stack, nargs); Py_DECREF(callable); - - return tmp; + if (stack != small_stack) { + PyMem_Free(stack); + } + + return result; } PyObject * _PyObject_CallMethodIdObjArgs(PyObject *callable, struct _Py_Identifier *name, ...) { - PyObject *args, *tmp; + PyObject *small_stack[5]; + PyObject **stack; + Py_ssize_t nargs; + PyObject *result; va_list vargs; if (callable == NULL || name == NULL) { @@ -2567,23 +2594,30 @@ /* count the args */ va_start(vargs, name); - args = objargs_mktuple(vargs); + stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack), + vargs, &nargs); va_end(vargs); - if (args == NULL) { + if (stack == NULL) { Py_DECREF(callable); return NULL; } - tmp = PyObject_Call(callable, args, NULL); - Py_DECREF(args); + + result = _PyObject_FastCall(callable, stack, nargs); Py_DECREF(callable); - - return tmp; + if (stack != small_stack) { + PyMem_Free(stack); + } + + return result; } PyObject * PyObject_CallFunctionObjArgs(PyObject *callable, ...) { - PyObject *args, *tmp; + PyObject *small_stack[5]; + PyObject **stack; + Py_ssize_t nargs; + PyObject *result; va_list vargs; if (callable == NULL) { @@ -2592,14 +2626,19 @@ /* count the args */ va_start(vargs, callable); - args = objargs_mktuple(vargs); + stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack), + vargs, &nargs); va_end(vargs); - if (args == NULL) + if (stack == NULL) { return NULL; - tmp = PyObject_Call(callable, args, NULL); - Py_DECREF(args); - - return tmp; + } + + result = _PyObject_FastCall(callable, stack, nargs); + if (stack != small_stack) { + PyMem_Free(stack); + } + + return result; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 18:55:43 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 23 Aug 2016 22:55:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Backed_out_changeset_70f88?= =?utf-8?q?b097f60_=28map=5Fnext=29?= Message-ID: <20160823225543.39362.76451.A4BF92C4@psf.io> https://hg.python.org/cpython/rev/c647c4f1db9e changeset: 102875:c647c4f1db9e user: Victor Stinner date: Wed Aug 24 00:54:47 2016 +0200 summary: Backed out changeset 70f88b097f60 (map_next) files: Python/bltinmodule.c | 50 ++++++++++--------------------- 1 files changed, 17 insertions(+), 33 deletions(-) diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -1156,43 +1156,27 @@ static PyObject * map_next(mapobject *lz) { - PyObject *small_stack[5]; - PyObject **stack; - Py_ssize_t niters, nargs, i; - PyObject *result = NULL; + PyObject *val; + PyObject *argtuple; + PyObject *result; + Py_ssize_t numargs, i; - niters = PyTuple_GET_SIZE(lz->iters); - if (niters <= (Py_ssize_t)Py_ARRAY_LENGTH(small_stack)) { - stack = small_stack; - } - else { - stack = PyMem_Malloc(niters * sizeof(PyObject*)); - if (stack == NULL) { - PyErr_NoMemory(); + numargs = PyTuple_GET_SIZE(lz->iters); + argtuple = PyTuple_New(numargs); + if (argtuple == NULL) + return NULL; + + for (i=0 ; iiters, i); + val = Py_TYPE(it)->tp_iternext(it); + if (val == NULL) { + Py_DECREF(argtuple); return NULL; } + PyTuple_SET_ITEM(argtuple, i, val); } - - nargs = 0; - for (i=0; i < niters; i++) { - PyObject *it = PyTuple_GET_ITEM(lz->iters, i); - PyObject *val = Py_TYPE(it)->tp_iternext(it); - if (val == NULL) { - goto exit; - } - stack[i] = val; - nargs++; - } - - result = _PyObject_FastCall(lz->func, stack, nargs); - -exit: - for (i=0; i < nargs; i++) { - Py_DECREF(stack[i]); - } - if (stack != small_stack) { - PyMem_Free(stack); - } + result = PyObject_Call(lz->func, argtuple, NULL); + Py_DECREF(argtuple); return result; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 19:00:54 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 23 Aug 2016 23:00:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Backed_out_changeset_0e4f2?= =?utf-8?q?6083bbb_=28PyObject=5FCallMethodObjArgs=29?= Message-ID: <20160823230022.24236.6524.F70EFA42@psf.io> https://hg.python.org/cpython/rev/46f7a27bf436 changeset: 102876:46f7a27bf436 user: Victor Stinner date: Wed Aug 24 00:59:40 2016 +0200 summary: Backed out changeset 0e4f26083bbb (PyObject_CallMethodObjArgs) files: Objects/abstract.c | 113 ++++++++++---------------------- 1 files changed, 37 insertions(+), 76 deletions(-) diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2499,52 +2499,32 @@ return retval; } -static PyObject ** -objargs_mkstack(PyObject **small_stack, Py_ssize_t small_stack_size, - va_list va, Py_ssize_t *p_nargs) +static PyObject * +objargs_mktuple(va_list va) { - Py_ssize_t i, n; + int i, n = 0; va_list countva; - PyObject **stack; - - /* Count the number of arguments */ - Py_VA_COPY(countva, va); - - n = 0; - while (1) { - PyObject *arg = (PyObject *)va_arg(countva, PyObject *); - if (arg == NULL) { - break; - } - n++; - } - *p_nargs = n; - - /* Copy arguments */ - if (small_stack_size <= n) { - stack = small_stack; - } - else { - stack = PyMem_Malloc(n * sizeof(PyObject**)); - if (stack == NULL) { - PyErr_NoMemory(); - return NULL; + PyObject *result, *tmp; + + Py_VA_COPY(countva, va); + + while (((PyObject *)va_arg(countva, PyObject *)) != NULL) + ++n; + result = PyTuple_New(n); + if (result != NULL && n > 0) { + for (i = 0; i < n; ++i) { + tmp = (PyObject *)va_arg(va, PyObject *); + PyTuple_SET_ITEM(result, i, tmp); + Py_INCREF(tmp); } } - - for (i = 0; i < n; ++i) { - stack[i] = va_arg(va, PyObject *); - } - return stack; + return result; } PyObject * PyObject_CallMethodObjArgs(PyObject *callable, PyObject *name, ...) { - PyObject *small_stack[5]; - PyObject **stack; - Py_ssize_t nargs; - PyObject *result; + PyObject *args, *tmp; va_list vargs; if (callable == NULL || name == NULL) { @@ -2557,31 +2537,24 @@ /* count the args */ va_start(vargs, name); - stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack), - vargs, &nargs); + args = objargs_mktuple(vargs); va_end(vargs); - if (stack == NULL) { + if (args == NULL) { Py_DECREF(callable); return NULL; } - - result = _PyObject_FastCall(callable, stack, nargs); + tmp = PyObject_Call(callable, args, NULL); + Py_DECREF(args); Py_DECREF(callable); - if (stack != small_stack) { - PyMem_Free(stack); - } - - return result; + + return tmp; } PyObject * _PyObject_CallMethodIdObjArgs(PyObject *callable, struct _Py_Identifier *name, ...) { - PyObject *small_stack[5]; - PyObject **stack; - Py_ssize_t nargs; - PyObject *result; + PyObject *args, *tmp; va_list vargs; if (callable == NULL || name == NULL) { @@ -2594,30 +2567,23 @@ /* count the args */ va_start(vargs, name); - stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack), - vargs, &nargs); + args = objargs_mktuple(vargs); va_end(vargs); - if (stack == NULL) { + if (args == NULL) { Py_DECREF(callable); return NULL; } - - result = _PyObject_FastCall(callable, stack, nargs); + tmp = PyObject_Call(callable, args, NULL); + Py_DECREF(args); Py_DECREF(callable); - if (stack != small_stack) { - PyMem_Free(stack); - } - - return result; + + return tmp; } PyObject * PyObject_CallFunctionObjArgs(PyObject *callable, ...) { - PyObject *small_stack[5]; - PyObject **stack; - Py_ssize_t nargs; - PyObject *result; + PyObject *args, *tmp; va_list vargs; if (callable == NULL) { @@ -2626,19 +2592,14 @@ /* count the args */ va_start(vargs, callable); - stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack), - vargs, &nargs); + args = objargs_mktuple(vargs); va_end(vargs); - if (stack == NULL) { + if (args == NULL) return NULL; - } - - result = _PyObject_FastCall(callable, stack, nargs); - if (stack != small_stack) { - PyMem_Free(stack); - } - - return result; + tmp = PyObject_Call(callable, args, NULL); + Py_DECREF(args); + + return tmp; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 19:25:30 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 23 Aug 2016 23:25:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_PyObject=5FCallMethodObjAr?= =?utf-8?q?gs=28=29_now_uses_fast_call?= Message-ID: <20160823232528.24285.6975.22D293DA@psf.io> https://hg.python.org/cpython/rev/401f59a7020b changeset: 102877:401f59a7020b user: Victor Stinner date: Wed Aug 24 01:14:54 2016 +0200 summary: PyObject_CallMethodObjArgs() now uses fast call Issue #27809: * PyObject_CallMethodObjArgs(), _PyObject_CallMethodIdObjArgs() and PyObject_CallFunctionObjArgs() now use fast call to avoid the creation of a temporary tuple * Rename objargs_mktuple() to objargs_mkstack() * objargs_mkstack() now stores objects in a C array using borrowed references, instead of storing arguments into a tuple objargs_mkstack() uses a small buffer allocated on the C stack for 5 arguments or less, or allocates a buffer in the heap memory. Note: this change is different than the change 0e4f26083bbb, I fixed the test to decide if the small stack can be used or not. sizeof(PyObject**) was also replaced with sizeof(stack[0]) since the sizeof() was wrong (but gave the same result). files: Objects/abstract.c | 113 ++++++++++++++++++++++---------- 1 files changed, 76 insertions(+), 37 deletions(-) diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2499,32 +2499,52 @@ return retval; } -static PyObject * -objargs_mktuple(va_list va) +static PyObject ** +objargs_mkstack(PyObject **small_stack, Py_ssize_t small_stack_size, + va_list va, Py_ssize_t *p_nargs) { - int i, n = 0; + Py_ssize_t i, n; va_list countva; - PyObject *result, *tmp; - - Py_VA_COPY(countva, va); - - while (((PyObject *)va_arg(countva, PyObject *)) != NULL) - ++n; - result = PyTuple_New(n); - if (result != NULL && n > 0) { - for (i = 0; i < n; ++i) { - tmp = (PyObject *)va_arg(va, PyObject *); - PyTuple_SET_ITEM(result, i, tmp); - Py_INCREF(tmp); + PyObject **stack; + + /* Count the number of arguments */ + Py_VA_COPY(countva, va); + + n = 0; + while (1) { + PyObject *arg = (PyObject *)va_arg(countva, PyObject *); + if (arg == NULL) { + break; + } + n++; + } + *p_nargs = n; + + /* Copy arguments */ + if (n <= small_stack_size) { + stack = small_stack; + } + else { + stack = PyMem_Malloc(n * sizeof(stack[0])); + if (stack == NULL) { + PyErr_NoMemory(); + return NULL; } } - return result; + + for (i = 0; i < n; ++i) { + stack[i] = va_arg(va, PyObject *); + } + return stack; } PyObject * PyObject_CallMethodObjArgs(PyObject *callable, PyObject *name, ...) { - PyObject *args, *tmp; + PyObject *small_stack[5]; + PyObject **stack; + Py_ssize_t nargs; + PyObject *result; va_list vargs; if (callable == NULL || name == NULL) { @@ -2537,24 +2557,31 @@ /* count the args */ va_start(vargs, name); - args = objargs_mktuple(vargs); + stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack), + vargs, &nargs); va_end(vargs); - if (args == NULL) { + if (stack == NULL) { Py_DECREF(callable); return NULL; } - tmp = PyObject_Call(callable, args, NULL); - Py_DECREF(args); + + result = _PyObject_FastCall(callable, stack, nargs); Py_DECREF(callable); - - return tmp; + if (stack != small_stack) { + PyMem_Free(stack); + } + + return result; } PyObject * _PyObject_CallMethodIdObjArgs(PyObject *callable, struct _Py_Identifier *name, ...) { - PyObject *args, *tmp; + PyObject *small_stack[5]; + PyObject **stack; + Py_ssize_t nargs; + PyObject *result; va_list vargs; if (callable == NULL || name == NULL) { @@ -2567,23 +2594,30 @@ /* count the args */ va_start(vargs, name); - args = objargs_mktuple(vargs); + stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack), + vargs, &nargs); va_end(vargs); - if (args == NULL) { + if (stack == NULL) { Py_DECREF(callable); return NULL; } - tmp = PyObject_Call(callable, args, NULL); - Py_DECREF(args); + + result = _PyObject_FastCall(callable, stack, nargs); Py_DECREF(callable); - - return tmp; + if (stack != small_stack) { + PyMem_Free(stack); + } + + return result; } PyObject * PyObject_CallFunctionObjArgs(PyObject *callable, ...) { - PyObject *args, *tmp; + PyObject *small_stack[5]; + PyObject **stack; + Py_ssize_t nargs; + PyObject *result; va_list vargs; if (callable == NULL) { @@ -2592,14 +2626,19 @@ /* count the args */ va_start(vargs, callable); - args = objargs_mktuple(vargs); + stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack), + vargs, &nargs); va_end(vargs); - if (args == NULL) + if (stack == NULL) { return NULL; - tmp = PyObject_Call(callable, args, NULL); - Py_DECREF(args); - - return tmp; + } + + result = _PyObject_FastCall(callable, stack, nargs); + if (stack != small_stack) { + PyMem_Free(stack); + } + + return result; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 19:52:04 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 23 Aug 2016 23:52:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327809=3A_map=5Fne?= =?utf-8?q?xt=28=29_uses_fast_call?= Message-ID: <20160823235204.1819.96487.4F793885@psf.io> https://hg.python.org/cpython/rev/1b241e761f8f changeset: 102878:1b241e761f8f user: Victor Stinner date: Wed Aug 24 01:45:13 2016 +0200 summary: Issue #27809: map_next() uses fast call Use a small stack allocated in the C stack for up to 5 iterator functions, otherwise allocates a stack on the heap memory. files: Python/bltinmodule.c | 50 +++++++++++++++++++++---------- 1 files changed, 33 insertions(+), 17 deletions(-) diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -1156,27 +1156,43 @@ static PyObject * map_next(mapobject *lz) { - PyObject *val; - PyObject *argtuple; - PyObject *result; - Py_ssize_t numargs, i; + PyObject *small_stack[5]; + PyObject **stack; + Py_ssize_t niters, nargs, i; + PyObject *result = NULL; - numargs = PyTuple_GET_SIZE(lz->iters); - argtuple = PyTuple_New(numargs); - if (argtuple == NULL) - return NULL; - - for (i=0 ; iiters, i); - val = Py_TYPE(it)->tp_iternext(it); - if (val == NULL) { - Py_DECREF(argtuple); + niters = PyTuple_GET_SIZE(lz->iters); + if (niters <= (Py_ssize_t)Py_ARRAY_LENGTH(small_stack)) { + stack = small_stack; + } + else { + stack = PyMem_Malloc(niters * sizeof(stack[0])); + if (stack == NULL) { + PyErr_NoMemory(); return NULL; } - PyTuple_SET_ITEM(argtuple, i, val); } - result = PyObject_Call(lz->func, argtuple, NULL); - Py_DECREF(argtuple); + + nargs = 0; + for (i=0; i < niters; i++) { + PyObject *it = PyTuple_GET_ITEM(lz->iters, i); + PyObject *val = Py_TYPE(it)->tp_iternext(it); + if (val == NULL) { + goto exit; + } + stack[i] = val; + nargs++; + } + + result = _PyObject_FastCall(lz->func, stack, nargs); + +exit: + for (i=0; i < nargs; i++) { + Py_DECREF(stack[i]); + } + if (stack != small_stack) { + PyMem_Free(stack); + } return result; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 20:44:48 2016 From: python-checkins at python.org (r.david.murray) Date: Wed, 24 Aug 2016 00:44:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=2325916=3A_fix_a_few_=27string_of_bytes=27_ref?= =?utf-8?q?erences=2E?= Message-ID: <20160824004448.68217.26269.7518EA16@psf.io> https://hg.python.org/cpython/rev/089f7d761a9c changeset: 102880:089f7d761a9c parent: 102878:1b241e761f8f parent: 102879:7ac8e8568da2 user: R David Murray date: Tue Aug 23 20:44:31 2016 -0400 summary: Merge: #25916: fix a few 'string of bytes' references. files: Doc/howto/unicode.rst | 2 +- Doc/library/logging.config.rst | 2 +- Doc/library/sunau.rst | 2 +- Doc/library/wave.rst | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst --- a/Doc/howto/unicode.rst +++ b/Doc/howto/unicode.rst @@ -192,7 +192,7 @@ UTF-8 has several convenient properties: 1. It can handle any Unicode code point. -2. A Unicode string is turned into a string of bytes containing no embedded zero +2. A Unicode string is turned into a sequence of bytes containing no embedded zero bytes. This avoids byte-ordering issues, and means UTF-8 strings can be processed by C functions such as ``strcpy()`` and sent through protocols that can't handle zero bytes. diff --git a/Doc/library/logging.config.rst b/Doc/library/logging.config.rst --- a/Doc/library/logging.config.rst +++ b/Doc/library/logging.config.rst @@ -144,7 +144,7 @@ (perhaps if decryption were performed). To send a configuration to the socket, read in the configuration file and - send it to the socket as a string of bytes preceded by a four-byte length + send it to the socket as a sequence of bytes preceded by a four-byte length string packed in binary using ``struct.pack('>L', n)``. .. note:: diff --git a/Doc/library/sunau.rst b/Doc/library/sunau.rst --- a/Doc/library/sunau.rst +++ b/Doc/library/sunau.rst @@ -158,7 +158,7 @@ .. method:: AU_read.readframes(n) - Reads and returns at most *n* frames of audio, as a string of bytes. The data + Reads and returns at most *n* frames of audio, as a :class:`bytes` object. The data will be returned in linear format. If the original data is in u-LAW format, it will be converted. diff --git a/Doc/library/wave.rst b/Doc/library/wave.rst --- a/Doc/library/wave.rst +++ b/Doc/library/wave.rst @@ -112,7 +112,7 @@ .. method:: Wave_read.readframes(n) - Reads and returns at most *n* frames of audio, as a string of bytes. + Reads and returns at most *n* frames of audio, as a :class:`bytes` object. .. method:: Wave_read.rewind() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 20:44:48 2016 From: python-checkins at python.org (r.david.murray) Date: Wed, 24 Aug 2016 00:44:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzI1OTE2OiBmaXgg?= =?utf-8?q?a_few_=27string_of_bytes=27_references=2E?= Message-ID: <20160824004448.27675.76737.C8325BAE@psf.io> https://hg.python.org/cpython/rev/7ac8e8568da2 changeset: 102879:7ac8e8568da2 branch: 3.5 parent: 102867:355fa8bd8d9d user: R David Murray date: Tue Aug 23 20:43:56 2016 -0400 summary: #25916: fix a few 'string of bytes' references. Patch by SilengGhost. files: Doc/howto/unicode.rst | 2 +- Doc/library/logging.config.rst | 2 +- Doc/library/sunau.rst | 2 +- Doc/library/wave.rst | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst --- a/Doc/howto/unicode.rst +++ b/Doc/howto/unicode.rst @@ -192,7 +192,7 @@ UTF-8 has several convenient properties: 1. It can handle any Unicode code point. -2. A Unicode string is turned into a string of bytes containing no embedded zero +2. A Unicode string is turned into a sequence of bytes containing no embedded zero bytes. This avoids byte-ordering issues, and means UTF-8 strings can be processed by C functions such as ``strcpy()`` and sent through protocols that can't handle zero bytes. diff --git a/Doc/library/logging.config.rst b/Doc/library/logging.config.rst --- a/Doc/library/logging.config.rst +++ b/Doc/library/logging.config.rst @@ -144,7 +144,7 @@ (perhaps if decryption were performed). To send a configuration to the socket, read in the configuration file and - send it to the socket as a string of bytes preceded by a four-byte length + send it to the socket as a sequence of bytes preceded by a four-byte length string packed in binary using ``struct.pack('>L', n)``. .. note:: diff --git a/Doc/library/sunau.rst b/Doc/library/sunau.rst --- a/Doc/library/sunau.rst +++ b/Doc/library/sunau.rst @@ -158,7 +158,7 @@ .. method:: AU_read.readframes(n) - Reads and returns at most *n* frames of audio, as a string of bytes. The data + Reads and returns at most *n* frames of audio, as a :class:`bytes` object. The data will be returned in linear format. If the original data is in u-LAW format, it will be converted. diff --git a/Doc/library/wave.rst b/Doc/library/wave.rst --- a/Doc/library/wave.rst +++ b/Doc/library/wave.rst @@ -112,7 +112,7 @@ .. method:: Wave_read.readframes(n) - Reads and returns at most *n* frames of audio, as a string of bytes. + Reads and returns at most *n* frames of audio, as a :class:`bytes` object. .. method:: Wave_read.rewind() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 21:12:51 2016 From: python-checkins at python.org (r.david.murray) Date: Wed, 24 Aug 2016 01:12:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2326907=3A_add_some_missi?= =?utf-8?q?ng_getsockopt_constants=2E?= Message-ID: <20160824011251.20534.19761.7DDAEF9C@psf.io> https://hg.python.org/cpython/rev/ed5f5d490490 changeset: 102881:ed5f5d490490 user: R David Murray date: Tue Aug 23 21:12:40 2016 -0400 summary: #26907: add some missing getsockopt constants. Patch by Christian Heimes, reviewed by Martin Panter. files: Doc/library/socket.rst | 4 ++++ Doc/whatsnew/3.6.rst | 4 ++++ Modules/socketmodule.c | 12 ++++++++++++ 3 files changed, 20 insertions(+), 0 deletions(-) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -281,6 +281,10 @@ in the Unix header files are defined; for a few symbols, default values are provided. + .. versionchanged:: 3.6 + ``SO_DOMAIN``, ``SO_PROTOCOL``, ``SO_PEERSEC``, ``SO_PASSSEC`` + were added. + .. data:: AF_CAN PF_CAN SOL_CAN_* diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -406,6 +406,10 @@ control code. (Contributed by Daniel Stokes in :issue:`26536`.) +The :meth:`~socket.socket.getsockopt` constants ``SO_DOMAIN``, +``SO_PROTOCOL``, ``SO_PEERSEC``, and ``SO_PASSSEC`` are now supported. +(Contributed by Christian Heimes in :issue:`26907`.) + socketserver ------------ diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -6582,6 +6582,12 @@ #ifdef LOCAL_PEERCRED PyModule_AddIntMacro(m, LOCAL_PEERCRED); #endif +#ifdef SO_PASSSEC + PyModule_AddIntMacro(m, SO_PASSSEC); +#endif +#ifdef SO_PEERSEC + PyModule_AddIntMacro(m, SO_PEERSEC); +#endif #ifdef SO_BINDTODEVICE PyModule_AddIntMacro(m, SO_BINDTODEVICE); #endif @@ -6591,6 +6597,12 @@ #ifdef SO_MARK PyModule_AddIntMacro(m, SO_MARK); #endif +#ifdef SO_DOMAIN + PyModule_AddIntMacro(m, SO_DOMAIN); +#endif +#ifdef SO_PROTOCOL + PyModule_AddIntMacro(m, SO_PROTOCOL); +#endif /* Maximum number of connections for "listen" */ #ifdef SOMAXCONN -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 22:17:27 2016 From: python-checkins at python.org (steven.daprano) Date: Wed, 24 Aug 2016 02:17:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Update_NEWS=2E?= Message-ID: <20160824021726.68217.30488.3E4622EB@psf.io> https://hg.python.org/cpython/rev/be0ec8b16b91 changeset: 102882:be0ec8b16b91 user: Steven D'Aprano date: Wed Aug 24 12:14:58 2016 +1000 summary: Update NEWS. files: Misc/NEWS | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -69,6 +69,8 @@ - Issue #9998: On Linux, ctypes.util.find_library now looks in LD_LIBRARY_PATH for shared libraries. +- Issue #27573: exit message for code.interact is now configurable. + Tests ----- @@ -259,7 +261,7 @@ Add tests for the changes to the config module. - Issue #27452: add line counter and crc to IDLE configHandler test dump. - + Tests ----- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 22:17:27 2016 From: python-checkins at python.org (steven.daprano) Date: Wed, 24 Aug 2016 02:17:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_geometric=5Fmean_to_?= =?utf-8?b?X19hbGxfXw==?= Message-ID: <20160824021726.27029.76661.BF101595@psf.io> https://hg.python.org/cpython/rev/0695f0bdd6d8 changeset: 102883:0695f0bdd6d8 user: Steven D'Aprano date: Wed Aug 24 12:17:00 2016 +1000 summary: Add geometric_mean to __all__ files: Lib/statistics.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Lib/statistics.py b/Lib/statistics.py --- a/Lib/statistics.py +++ b/Lib/statistics.py @@ -11,6 +11,7 @@ Function Description ================== ============================================= mean Arithmetic mean (average) of data. +geometric_mean Geometric mean of data. harmonic_mean Harmonic mean of data. median Median (middle value) of data. median_low Low median of data. @@ -79,7 +80,7 @@ __all__ = [ 'StatisticsError', 'pstdev', 'pvariance', 'stdev', 'variance', 'median', 'median_low', 'median_high', 'median_grouped', - 'mean', 'mode', 'harmonic_mean', + 'mean', 'mode', 'geometric_mean', 'harmonic_mean', ] import collections -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 22:49:29 2016 From: python-checkins at python.org (steven.daprano) Date: Wed, 24 Aug 2016 02:49:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_support_for_nth_roo?= =?utf-8?q?t_of_negative_numbers_with_odd_powers=2E?= Message-ID: <20160824024929.18061.43789.90D1205A@psf.io> https://hg.python.org/cpython/rev/69de5c95fd53 changeset: 102884:69de5c95fd53 user: Steven D'Aprano date: Wed Aug 24 12:48:12 2016 +1000 summary: Remove support for nth root of negative numbers with odd powers. Although nth roots of negative numbers are real for odd n, the statistics module doesn't make use of this. Remove support for negative roots from the private _nth_root function, which simplifies the test suite. files: Lib/statistics.py | 7 ++--- Lib/test/test_statistics.py | 29 ++++-------------------- 2 files changed, 8 insertions(+), 28 deletions(-) diff --git a/Lib/statistics.py b/Lib/statistics.py --- a/Lib/statistics.py +++ b/Lib/statistics.py @@ -335,10 +335,7 @@ """Handle nth root of Reals, treated as a float.""" assert isinstance(n, int) and n > 1 if x < 0: - if n%2 == 0: - raise ValueError('domain error: even root of negative number') - else: - return -_nroot_NS.nroot(-x, n) + raise ValueError('domain error: root of negative number') elif x == 0: return math.copysign(0.0, x) elif x > 0: @@ -433,6 +430,8 @@ else: # Preserve the input NAN. return x + if x < 0: + raise ValueError('domain error: root of negative number') if x.is_infinite(): return x # FIXME this hasn't had the extensive testing of the float diff --git a/Lib/test/test_statistics.py b/Lib/test/test_statistics.py --- a/Lib/test/test_statistics.py +++ b/Lib/test/test_statistics.py @@ -1095,13 +1095,6 @@ with self.subTest(n=n, inf=INF): self.assertEqual(self.nroot(INF, n), INF) - def testNInf(self): - # Test that the root of -inf is -inf for odd n. - for NINF in (float('-inf'), decimal.Decimal('-inf')): - for n in range(3, 11, 2): - with self.subTest(n=n, inf=NINF): - self.assertEqual(self.nroot(NINF, n), NINF) - # FIXME: need to check Decimal zeroes too. def test_zero(self): # Test that the root of +0.0 is +0.0. @@ -1157,13 +1150,15 @@ with self.subTest(x=x): self.assertRaises(TypeError, self.nroot, x, 3) - def testNegativeEvenPower(self): - # Test negative x with even n raises correctly. + def testNegativeError(self): + # Test negative x raises correctly. x = random.uniform(-20.0, -0.1) assert x < 0 - for n in range(2, 9, 2): + for n in range(3, 7): with self.subTest(x=x, n=n): self.assertRaises(ValueError, self.nroot, x, n) + # And Decimal. + self.assertRaises(ValueError, self.nroot, Decimal(-27), 3) # --- Test that nroot is never worse than calling math.pow() --- @@ -1216,25 +1211,11 @@ x = i**n self.assertEqual(self.nroot(x, n), i) - def testExactPowersNegatives(self): - # Test that small negative integer powers are calculated exactly. - for i in range(-1, -51, -1): - for n in range(3, 16, 2): - if (i, n) == (-35, 13): - # See testExpectedFailure35p13 - continue - with self.subTest(i=i, n=n): - x = i**n - assert sign(x) == -1 - self.assertEqual(self.nroot(x, n), i) - def testExpectedFailure35p13(self): # Test the expected failure 35**13 is almost exact. x = 35**13 err = abs(self.nroot(x, 13) - 35) self.assertLessEqual(err, 0.000000001) - err = abs(self.nroot(-x, 13) + 35) - self.assertLessEqual(err, 0.000000001) def testOne(self): # Test that the root of 1.0 is 1.0. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 23 23:55:46 2016 From: python-checkins at python.org (steven.daprano) Date: Wed, 24 Aug 2016 03:55:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_expected_failure_fr?= =?utf-8?q?om_test_of_=5Fproduct_internal_function=2E?= Message-ID: <20160824035546.38904.25794.FB370244@psf.io> https://hg.python.org/cpython/rev/de23b73e911b changeset: 102885:de23b73e911b user: Steven D'Aprano date: Wed Aug 24 13:54:31 2016 +1000 summary: Remove expected failure from test of _product internal function. files: Lib/test/test_statistics.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_statistics.py b/Lib/test/test_statistics.py --- a/Lib/test/test_statistics.py +++ b/Lib/test/test_statistics.py @@ -1041,11 +1041,11 @@ self.assertEqual(mant, F(10)) self.assertTrue(isinstance(mant, F)) - @unittest.expectedFailure def test_decimal(self): D = Decimal data = [D('24.5'), D('17.6'), D('0.025'), D('1.3')] - assert False + expected = D('14.014000') + self.assertEqual(statistics._product(data), (0, expected)) def test_mixed_decimal_float(self): # Test that mixed Decimal and float raises. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 02:39:18 2016 From: python-checkins at python.org (martin.panter) Date: Wed, 24 Aug 2016 06:39:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2312319=3A_Support_?= =?utf-8?q?for_chunked_encoding_of_HTTP_request_bodies?= Message-ID: <20160824063917.24170.13006.165DE399@psf.io> https://hg.python.org/cpython/rev/0effb34f3276 changeset: 102886:0effb34f3276 user: Martin Panter date: Wed Aug 24 06:33:33 2016 +0000 summary: Issue #12319: Support for chunked encoding of HTTP request bodies When the body object is a file, its size is no longer determined with fstat(), since that can report the wrong result (e.g. reading from a pipe). Instead, determine the size using seek(), or fall back to chunked encoding for unseekable files. Also, change the logic for detecting text files to check for TextIOBase inheritance, rather than inspecting the ?mode? attribute, which may not exist (e.g. BytesIO and StringIO). The Content-Length for text files is no longer determined ahead of time, because the original logic could have been wrong depending on the codec and newline translation settings. Patch by Demian Brecht and Rolf Krahl, with a few tweaks by me. files: Doc/library/http.client.rst | 94 +++++++-- Doc/library/urllib.request.rst | 60 +++-- Doc/whatsnew/3.6.rst | 19 ++ Lib/http/client.py | 199 +++++++++++++++----- Lib/test/test_httplib.py | 151 +++++++++++++++- Lib/test/test_urllib2.py | 107 ++++++++-- Lib/urllib/request.py | 42 ++-- Misc/ACKS | 1 + Misc/NEWS | 8 + 9 files changed, 531 insertions(+), 150 deletions(-) diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -219,39 +219,62 @@ :class:`HTTPConnection` instances have the following methods: -.. method:: HTTPConnection.request(method, url, body=None, headers={}) +.. method:: HTTPConnection.request(method, url, body=None, headers={}, *, \ + encode_chunked=False) This will send a request to the server using the HTTP request method *method* and the selector *url*. If *body* is specified, the specified data is sent after the headers are - finished. It may be a string, a :term:`bytes-like object`, an open - :term:`file object`, or an iterable of :term:`bytes-like object`\s. If - *body* is a string, it is encoded as ISO-8859-1, the default for HTTP. If - it is a bytes-like object the bytes are sent as is. If it is a :term:`file - object`, the contents of the file is sent; this file object should support - at least the ``read()`` method. If the file object has a ``mode`` - attribute, the data returned by the ``read()`` method will be encoded as - ISO-8859-1 unless the ``mode`` attribute contains the substring ``b``, - otherwise the data returned by ``read()`` is sent as is. If *body* is an - iterable, the elements of the iterable are sent as is until the iterable is - exhausted. + finished. It may be a :class:`str`, a :term:`bytes-like object`, an + open :term:`file object`, or an iterable of :class:`bytes`. If *body* + is a string, it is encoded as ISO-8859-1, the default for HTTP. If it + is a bytes-like object, the bytes are sent as is. If it is a :term:`file + object`, the contents of the file is sent; this file object should + support at least the ``read()`` method. If the file object is an + instance of :class:`io.TextIOBase`, the data returned by the ``read()`` + method will be encoded as ISO-8859-1, otherwise the data returned by + ``read()`` is sent as is. If *body* is an iterable, the elements of the + iterable are sent as is until the iterable is exhausted. - The *headers* argument should be a mapping of extra HTTP - headers to send with the request. + The *headers* argument should be a mapping of extra HTTP headers to send + with the request. - If *headers* does not contain a Content-Length item, one is added - automatically if possible. If *body* is ``None``, the Content-Length header - is set to ``0`` for methods that expect a body (``PUT``, ``POST``, and - ``PATCH``). If *body* is a string or bytes object, the Content-Length - header is set to its length. If *body* is a :term:`file object` and it - works to call :func:`~os.fstat` on the result of its ``fileno()`` method, - then the Content-Length header is set to the ``st_size`` reported by the - ``fstat`` call. Otherwise no Content-Length header is added. + If *headers* contains neither Content-Length nor Transfer-Encoding, a + Content-Length header will be added automatically if possible. If + *body* is ``None``, the Content-Length header is set to ``0`` for + methods that expect a body (``PUT``, ``POST``, and ``PATCH``). If + *body* is a string or bytes-like object, the Content-Length header is + set to its length. If *body* is a binary :term:`file object` + supporting :meth:`~io.IOBase.seek`, this will be used to determine + its size. Otherwise, the Content-Length header is not added + automatically. In cases where determining the Content-Length up + front is not possible, the body will be chunk-encoded and the + Transfer-Encoding header will automatically be set. + + The *encode_chunked* argument is only relevant if Transfer-Encoding is + specified in *headers*. If *encode_chunked* is ``False``, the + HTTPConnection object assumes that all encoding is handled by the + calling code. If it is ``True``, the body will be chunk-encoded. + + .. note:: + Chunked transfer encoding has been added to the HTTP protocol + version 1.1. Unless the HTTP server is known to handle HTTP 1.1, + the caller must either specify the Content-Length or must use a + body representation whose length can be determined automatically. .. versionadded:: 3.2 *body* can now be an iterable. + .. versionchanged:: 3.6 + If neither Content-Length nor Transfer-Encoding are set in + *headers* and Content-Length cannot be determined, *body* will now + be automatically chunk-encoded. The *encode_chunked* argument + was added. + The Content-Length for binary file objects is determined with seek. + No attempt is made to determine the Content-Length for text file + objects. + .. method:: HTTPConnection.getresponse() Should be called after a request is sent to get the response from the server. @@ -336,13 +359,32 @@ an argument. -.. method:: HTTPConnection.endheaders(message_body=None) +.. method:: HTTPConnection.endheaders(message_body=None, *, encode_chunked=False) Send a blank line to the server, signalling the end of the headers. The optional *message_body* argument can be used to pass a message body - associated with the request. The message body will be sent in the same - packet as the message headers if it is string, otherwise it is sent in a - separate packet. + associated with the request. + + If *encode_chunked* is ``True``, the result of each iteration of + *message_body* will be chunk-encoded as specified in :rfc:`7230`, + Section 3.3.1. How the data is encoded is dependent on the type of + *message_body*. If *message_body* implements the :ref:`buffer interface + ` the encoding will result in a single chunk. + If *message_body* is a :class:`collections.Iterable`, each iteration + of *message_body* will result in a chunk. If *message_body* is a + :term:`file object`, each call to ``.read()`` will result in a chunk. + The method automatically signals the end of the chunk-encoded data + immediately after *message_body*. + + .. note:: Due to the chunked encoding specification, empty chunks + yielded by an iterator body will be ignored by the chunk-encoder. + This is to avoid premature termination of the read of the request by + the target server due to malformed encoding. + + .. versionadded:: 3.6 + Chunked encoding support. The *encode_chunked* parameter was + added. + .. method:: HTTPConnection.send(data) diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst --- a/Doc/library/urllib.request.rst +++ b/Doc/library/urllib.request.rst @@ -30,18 +30,9 @@ Open the URL *url*, which can be either a string or a :class:`Request` object. - *data* must be a bytes object specifying additional data to be sent to the - server, or ``None`` if no such data is needed. *data* may also be an - iterable object and in that case Content-Length value must be specified in - the headers. Currently HTTP requests are the only ones that use *data*; the - HTTP request will be a POST instead of a GET when the *data* parameter is - provided. - - *data* should be a buffer in the standard - :mimetype:`application/x-www-form-urlencoded` format. The - :func:`urllib.parse.urlencode` function takes a mapping or sequence of - 2-tuples and returns an ASCII text string in this format. It should - be encoded to bytes before being used as the *data* parameter. + *data* must be an object specifying additional data to be sent to the + server, or ``None`` if no such data is needed. See :class:`Request` + for details. urllib.request module uses HTTP/1.1 and includes ``Connection:close`` header in its HTTP requests. @@ -192,14 +183,22 @@ *url* should be a string containing a valid URL. - *data* must be a bytes object specifying additional data to send to the - server, or ``None`` if no such data is needed. Currently HTTP requests are - the only ones that use *data*; the HTTP request will be a POST instead of a - GET when the *data* parameter is provided. *data* should be a buffer in the - standard :mimetype:`application/x-www-form-urlencoded` format. - The :func:`urllib.parse.urlencode` function takes a mapping or sequence of - 2-tuples and returns an ASCII string in this format. It should be - encoded to bytes before being used as the *data* parameter. + *data* must be an object specifying additional data to send to the + server, or ``None`` if no such data is needed. Currently HTTP + requests are the only ones that use *data*. The supported object + types include bytes, file-like objects, and iterables. If no + ``Content-Length`` header has been provided, :class:`HTTPHandler` will + try to determine the length of *data* and set this header accordingly. + If this fails, ``Transfer-Encoding: chunked`` as specified in + :rfc:`7230`, Section 3.3.1 will be used to send the data. See + :meth:`http.client.HTTPConnection.request` for details on the + supported object types and on how the content length is determined. + + For an HTTP POST request method, *data* should be a buffer in the + standard :mimetype:`application/x-www-form-urlencoded` format. The + :func:`urllib.parse.urlencode` function takes a mapping or sequence + of 2-tuples and returns an ASCII string in this format. It should + be encoded to bytes before being used as the *data* parameter. *headers* should be a dictionary, and will be treated as if :meth:`add_header` was called with each key and value as arguments. @@ -211,8 +210,10 @@ :mod:`urllib`'s default user agent string is ``"Python-urllib/2.6"`` (on Python 2.6). - An example of using ``Content-Type`` header with *data* argument would be - sending a dictionary like ``{"Content-Type": "application/x-www-form-urlencoded"}``. + An appropriate ``Content-Type`` header should be included if the *data* + argument is present. If this header has not been provided and *data* + is not None, ``Content-Type: application/x-www-form-urlencoded`` will + be added as a default. The final two arguments are only of interest for correct handling of third-party HTTP cookies: @@ -235,15 +236,28 @@ *method* should be a string that indicates the HTTP request method that will be used (e.g. ``'HEAD'``). If provided, its value is stored in the :attr:`~Request.method` attribute and is used by :meth:`get_method()`. - Subclasses may indicate a default method by setting the + The default is ``'GET'`` if *data* is ``None`` or ``'POST'`` otherwise. + Subclasses may indicate a different default method by setting the :attr:`~Request.method` attribute in the class itself. + .. note:: + The request will not work as expected if the data object is unable + to deliver its content more than once (e.g. a file or an iterable + that can produce the content only once) and the request is retried + for HTTP redirects or authentication. The *data* is sent to the + HTTP server right away after the headers. There is no support for + a 100-continue expectation in the library. + .. versionchanged:: 3.3 :attr:`Request.method` argument is added to the Request class. .. versionchanged:: 3.4 Default :attr:`Request.method` may be indicated at the class level. + .. versionchanged:: 3.6 + Do not raise an error if the ``Content-Length`` has not been + provided and could not be determined. Fall back to use chunked + transfer encoding instead. .. class:: OpenerDirector() diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -324,6 +324,15 @@ :issue:`23848`.) +http.client +----------- + +:meth:`HTTPConnection.request() ` and +:meth:`~http.client.HTTPConnection.endheaders` both now support +chunked encoding request bodies. +(Contributed by Demian Brecht and Rolf Krahl in :issue:`12319`.) + + idlelib and IDLE ---------------- @@ -500,6 +509,16 @@ (Contributed by Amit Saha in :issue:`26323`.) +urllib.request +-------------- + +If a HTTP request has a non-empty body but no Content-Length header +and the content length cannot be determined up front, rather than +throwing an error, :class:`~urllib.request.AbstractHTTPHandler` now +falls back to use chunked transfer encoding. +(Contributed by Demian Brecht and Rolf Krahl in :issue:`12319`.) + + urllib.robotparser ------------------ diff --git a/Lib/http/client.py b/Lib/http/client.py --- a/Lib/http/client.py +++ b/Lib/http/client.py @@ -795,6 +795,58 @@ auto_open = 1 debuglevel = 0 + @staticmethod + def _is_textIO(stream): + """Test whether a file-like object is a text or a binary stream. + """ + return isinstance(stream, io.TextIOBase) + + @staticmethod + def _get_content_length(body, method): + """Get the content-length based on the body. + + If the body is "empty", we set Content-Length: 0 for methods + that expect a body (RFC 7230, Section 3.3.2). If the body is + set for other methods, we set the header provided we can + figure out what the length is. + """ + if not body: + # do an explicit check for not None here to distinguish + # between unset and set but empty + if method.upper() in _METHODS_EXPECTING_BODY or body is not None: + return 0 + else: + return None + + if hasattr(body, 'read'): + # file-like object. + if HTTPConnection._is_textIO(body): + # text streams are unpredictable because it depends on + # character encoding and line ending translation. + return None + else: + # Is it seekable? + try: + curpos = body.tell() + sz = body.seek(0, io.SEEK_END) + except (TypeError, AttributeError, OSError): + return None + else: + body.seek(curpos) + return sz - curpos + + try: + # does it implement the buffer protocol (bytes, bytearray, array)? + mv = memoryview(body) + return mv.nbytes + except TypeError: + pass + + if isinstance(body, str): + return len(body) + + return None + def __init__(self, host, port=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT, source_address=None): self.timeout = timeout @@ -933,18 +985,9 @@ if hasattr(data, "read") : if self.debuglevel > 0: print("sendIng a read()able") - encode = False - try: - mode = data.mode - except AttributeError: - # io.BytesIO and other file-like objects don't have a `mode` - # attribute. - pass - else: - if "b" not in mode: - encode = True - if self.debuglevel > 0: - print("encoding file using iso-8859-1") + encode = self._is_textIO(data) + if encode and self.debuglevel > 0: + print("encoding file using iso-8859-1") while 1: datablock = data.read(blocksize) if not datablock: @@ -970,7 +1013,22 @@ """ self._buffer.append(s) - def _send_output(self, message_body=None): + def _read_readable(self, readable): + blocksize = 8192 + if self.debuglevel > 0: + print("sendIng a read()able") + encode = self._is_textIO(readable) + if encode and self.debuglevel > 0: + print("encoding file using iso-8859-1") + while True: + datablock = readable.read(blocksize) + if not datablock: + break + if encode: + datablock = datablock.encode("iso-8859-1") + yield datablock + + def _send_output(self, message_body=None, encode_chunked=False): """Send the currently buffered request and clear the buffer. Appends an extra \\r\\n to the buffer. @@ -979,10 +1037,50 @@ self._buffer.extend((b"", b"")) msg = b"\r\n".join(self._buffer) del self._buffer[:] + self.send(msg) - self.send(msg) if message_body is not None: - self.send(message_body) + + # create a consistent interface to message_body + if hasattr(message_body, 'read'): + # Let file-like take precedence over byte-like. This + # is needed to allow the current position of mmap'ed + # files to be taken into account. + chunks = self._read_readable(message_body) + else: + try: + # this is solely to check to see if message_body + # implements the buffer API. it /would/ be easier + # to capture if PyObject_CheckBuffer was exposed + # to Python. + memoryview(message_body) + except TypeError: + try: + chunks = iter(message_body) + except TypeError: + raise TypeError("message_body should be a bytes-like " + "object or an iterable, got %r" + % type(message_body)) + else: + # the object implements the buffer interface and + # can be passed directly into socket methods + chunks = (message_body,) + + for chunk in chunks: + if not chunk: + if self.debuglevel > 0: + print('Zero length chunk ignored') + continue + + if encode_chunked and self._http_vsn == 11: + # chunked encoding + chunk = f'{len(chunk):X}\r\n'.encode('ascii') + chunk \ + + b'\r\n' + self.send(chunk) + + if encode_chunked and self._http_vsn == 11: + # end chunked transfer + self.send(b'0\r\n\r\n') def putrequest(self, method, url, skip_host=0, skip_accept_encoding=0): """Send a request to the server. @@ -1135,52 +1233,27 @@ header = header + b': ' + value self._output(header) - def endheaders(self, message_body=None): + def endheaders(self, message_body=None, *, encode_chunked=False): """Indicate that the last header line has been sent to the server. This method sends the request to the server. The optional message_body argument can be used to pass a message body associated with the - request. The message body will be sent in the same packet as the - message headers if it is a string, otherwise it is sent as a separate - packet. + request. """ if self.__state == _CS_REQ_STARTED: self.__state = _CS_REQ_SENT else: raise CannotSendHeader() - self._send_output(message_body) + self._send_output(message_body, encode_chunked=encode_chunked) - def request(self, method, url, body=None, headers={}): + def request(self, method, url, body=None, headers={}, *, + encode_chunked=False): """Send a complete request to the server.""" - self._send_request(method, url, body, headers) + self._send_request(method, url, body, headers, encode_chunked) - def _set_content_length(self, body, method): - # Set the content-length based on the body. If the body is "empty", we - # set Content-Length: 0 for methods that expect a body (RFC 7230, - # Section 3.3.2). If the body is set for other methods, we set the - # header provided we can figure out what the length is. - thelen = None - method_expects_body = method.upper() in _METHODS_EXPECTING_BODY - if body is None and method_expects_body: - thelen = '0' - elif body is not None: - try: - thelen = str(len(body)) - except TypeError: - # If this is a file-like object, try to - # fstat its file descriptor - try: - thelen = str(os.fstat(body.fileno()).st_size) - except (AttributeError, OSError): - # Don't send a length if this failed - if self.debuglevel > 0: print("Cannot stat!!") - - if thelen is not None: - self.putheader('Content-Length', thelen) - - def _send_request(self, method, url, body, headers): + def _send_request(self, method, url, body, headers, encode_chunked): # Honor explicitly requested Host: and Accept-Encoding: headers. - header_names = dict.fromkeys([k.lower() for k in headers]) + header_names = frozenset(k.lower() for k in headers) skips = {} if 'host' in header_names: skips['skip_host'] = 1 @@ -1189,15 +1262,41 @@ self.putrequest(method, url, **skips) + # chunked encoding will happen if HTTP/1.1 is used and either + # the caller passes encode_chunked=True or the following + # conditions hold: + # 1. content-length has not been explicitly set + # 2. the length of the body cannot be determined + # (e.g. it is a generator or unseekable file) + # 3. Transfer-Encoding has NOT been explicitly set by the caller + if 'content-length' not in header_names: - self._set_content_length(body, method) + # only chunk body if not explicitly set for backwards + # compatibility, assuming the client code is already handling the + # chunking + if 'transfer-encoding' not in header_names: + # if content-length cannot be automatically determined, fall + # back to chunked encoding + encode_chunked = False + content_length = self._get_content_length(body, method) + if content_length is None: + if body: + if self.debuglevel > 0: + print('Unable to determine size of %r' % body) + encode_chunked = True + self.putheader('Transfer-Encoding', 'chunked') + else: + self.putheader('Content-Length', str(content_length)) + else: + encode_chunked = False + for hdr, value in headers.items(): self.putheader(hdr, value) if isinstance(body, str): # RFC 2616 Section 3.7.1 says that text default has a # default charset of iso-8859-1. body = _encode(body, 'body') - self.endheaders(body) + self.endheaders(body, encode_chunked=encode_chunked) def getresponse(self): """Get the response from the server. diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -314,6 +314,124 @@ conn.putheader(name, value) +class TransferEncodingTest(TestCase): + expected_body = b"It's just a flesh wound" + + def test_endheaders_chunked(self): + conn = client.HTTPConnection('example.com') + conn.sock = FakeSocket(b'') + conn.putrequest('POST', '/') + conn.endheaders(self._make_body(), encode_chunked=True) + + _, _, body = self._parse_request(conn.sock.data) + body = self._parse_chunked(body) + self.assertEqual(body, self.expected_body) + + def test_explicit_headers(self): + # explicit chunked + conn = client.HTTPConnection('example.com') + conn.sock = FakeSocket(b'') + # this shouldn't actually be automatically chunk-encoded because the + # calling code has explicitly stated that it's taking care of it + conn.request( + 'POST', '/', self._make_body(), {'Transfer-Encoding': 'chunked'}) + + _, headers, body = self._parse_request(conn.sock.data) + self.assertNotIn('content-length', [k.lower() for k in headers.keys()]) + self.assertEqual(headers['Transfer-Encoding'], 'chunked') + self.assertEqual(body, self.expected_body) + + # explicit chunked, string body + conn = client.HTTPConnection('example.com') + conn.sock = FakeSocket(b'') + conn.request( + 'POST', '/', self.expected_body.decode('latin-1'), + {'Transfer-Encoding': 'chunked'}) + + _, headers, body = self._parse_request(conn.sock.data) + self.assertNotIn('content-length', [k.lower() for k in headers.keys()]) + self.assertEqual(headers['Transfer-Encoding'], 'chunked') + self.assertEqual(body, self.expected_body) + + # User-specified TE, but request() does the chunk encoding + conn = client.HTTPConnection('example.com') + conn.sock = FakeSocket(b'') + conn.request('POST', '/', + headers={'Transfer-Encoding': 'gzip, chunked'}, + encode_chunked=True, + body=self._make_body()) + _, headers, body = self._parse_request(conn.sock.data) + self.assertNotIn('content-length', [k.lower() for k in headers]) + self.assertEqual(headers['Transfer-Encoding'], 'gzip, chunked') + self.assertEqual(self._parse_chunked(body), self.expected_body) + + def test_request(self): + for empty_lines in (False, True,): + conn = client.HTTPConnection('example.com') + conn.sock = FakeSocket(b'') + conn.request( + 'POST', '/', self._make_body(empty_lines=empty_lines)) + + _, headers, body = self._parse_request(conn.sock.data) + body = self._parse_chunked(body) + self.assertEqual(body, self.expected_body) + self.assertEqual(headers['Transfer-Encoding'], 'chunked') + + # Content-Length and Transfer-Encoding SHOULD not be sent in the + # same request + self.assertNotIn('content-length', [k.lower() for k in headers]) + + def _make_body(self, empty_lines=False): + lines = self.expected_body.split(b' ') + for idx, line in enumerate(lines): + # for testing handling empty lines + if empty_lines and idx % 2: + yield b'' + if idx < len(lines) - 1: + yield line + b' ' + else: + yield line + + def _parse_request(self, data): + lines = data.split(b'\r\n') + request = lines[0] + headers = {} + n = 1 + while n < len(lines) and len(lines[n]) > 0: + key, val = lines[n].split(b':') + key = key.decode('latin-1').strip() + headers[key] = val.decode('latin-1').strip() + n += 1 + + return request, headers, b'\r\n'.join(lines[n + 1:]) + + def _parse_chunked(self, data): + body = [] + trailers = {} + n = 0 + lines = data.split(b'\r\n') + # parse body + while True: + size, chunk = lines[n:n+2] + size = int(size, 16) + + if size == 0: + n += 1 + break + + self.assertEqual(size, len(chunk)) + body.append(chunk) + + n += 2 + # we /should/ hit the end chunk, but check against the size of + # lines so we're not stuck in an infinite loop should we get + # malformed data + if n > len(lines): + break + + return b''.join(body) + + class BasicTest(TestCase): def test_status_lines(self): # Test HTTP status lines @@ -564,11 +682,11 @@ yield None yield 'data_two' - class UpdatingFile(): + class UpdatingFile(io.TextIOBase): mode = 'r' d = data() def read(self, blocksize=-1): - return self.d.__next__() + return next(self.d) expected = b'data' @@ -1546,6 +1664,26 @@ message = client.parse_headers(f) return message, f + def test_list_body(self): + # Note that no content-length is automatically calculated for + # an iterable. The request will fall back to send chunked + # transfer encoding. + cases = ( + ([b'foo', b'bar'], b'3\r\nfoo\r\n3\r\nbar\r\n0\r\n\r\n'), + ((b'foo', b'bar'), b'3\r\nfoo\r\n3\r\nbar\r\n0\r\n\r\n'), + ) + for body, expected in cases: + with self.subTest(body): + self.conn = client.HTTPConnection('example.com') + self.conn.sock = self.sock = FakeSocket('') + + self.conn.request('PUT', '/url', body) + msg, f = self.get_headers_and_fp() + self.assertNotIn('Content-Type', msg) + self.assertNotIn('Content-Length', msg) + self.assertEqual(msg.get('Transfer-Encoding'), 'chunked') + self.assertEqual(expected, f.read()) + def test_manual_content_length(self): # Set an incorrect content-length so that we can verify that # it will not be over-ridden by the library. @@ -1588,8 +1726,13 @@ message, f = self.get_headers_and_fp() self.assertEqual("text/plain", message.get_content_type()) self.assertIsNone(message.get_charset()) - self.assertEqual("4", message.get("content-length")) - self.assertEqual(b'body', f.read()) + # Note that the length of text files is unpredictable + # because it depends on character encoding and line ending + # translation. No content-length will be set, the body + # will be sent using chunked transfer encoding. + self.assertIsNone(message.get("content-length")) + self.assertEqual("chunked", message.get("transfer-encoding")) + self.assertEqual(b'4\r\nbody\r\n0\r\n\r\n', f.read()) def test_binary_file_body(self): self.addCleanup(support.unlink, support.TESTFN) diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -7,6 +7,8 @@ import socket import array import sys +import tempfile +import subprocess import urllib.request # The proxy bypass method imported below has logic specific to the OSX @@ -335,7 +337,8 @@ else: self._tunnel_headers.clear() - def request(self, method, url, body=None, headers=None): + def request(self, method, url, body=None, headers=None, *, + encode_chunked=False): self.method = method self.selector = url if headers is not None: @@ -343,6 +346,7 @@ self.req_headers.sort() if body: self.data = body + self.encode_chunked = encode_chunked if self.raise_on_endheaders: raise OSError() @@ -908,7 +912,75 @@ self.assertEqual(req.unredirected_hdrs["Host"], "baz") self.assertEqual(req.unredirected_hdrs["Spam"], "foo") - # Check iterable body support + def test_http_body_file(self): + # A regular file - Content Length is calculated unless already set. + + h = urllib.request.AbstractHTTPHandler() + o = h.parent = MockOpener() + + file_obj = tempfile.NamedTemporaryFile(mode='w+b', delete=False) + file_path = file_obj.name + file_obj.write(b"Something\nSomething\nSomething\n") + file_obj.close() + + for headers in {}, {"Content-Length": 30}: + with open(file_path, "rb") as f: + req = Request("http://example.com/", f, headers) + newreq = h.do_request_(req) + self.assertEqual(int(newreq.get_header('Content-length')), 30) + + os.unlink(file_path) + + def test_http_body_fileobj(self): + # A file object - Content Length is calculated unless already set. + # (Note that there are some subtle differences to a regular + # file, that is why we are testing both cases.) + + h = urllib.request.AbstractHTTPHandler() + o = h.parent = MockOpener() + + file_obj = io.BytesIO() + file_obj.write(b"Something\nSomething\nSomething\n") + + for headers in {}, {"Content-Length": 30}: + file_obj.seek(0) + req = Request("http://example.com/", file_obj, headers) + newreq = h.do_request_(req) + self.assertEqual(int(newreq.get_header('Content-length')), 30) + + file_obj.close() + + def test_http_body_pipe(self): + # A file reading from a pipe. + # A pipe cannot be seek'ed. There is no way to determine the + # content length up front. Thus, do_request_() should fall + # back to Transfer-encoding chunked. + + h = urllib.request.AbstractHTTPHandler() + o = h.parent = MockOpener() + + cmd = [sys.executable, "-c", + r"import sys; " + r"sys.stdout.buffer.write(b'Something\nSomething\nSomething\n')"] + for headers in {}, {"Content-Length": 30}: + with subprocess.Popen(cmd, stdout=subprocess.PIPE) as proc: + req = Request("http://example.com/", proc.stdout, headers) + newreq = h.do_request_(req) + if not headers: + self.assertEqual(newreq.get_header('Content-length'), None) + self.assertEqual(newreq.get_header('Transfer-encoding'), + 'chunked') + else: + self.assertEqual(int(newreq.get_header('Content-length')), + 30) + + def test_http_body_iterable(self): + # Generic iterable. There is no way to determine the content + # length up front. Fall back to Transfer-encoding chunked. + + h = urllib.request.AbstractHTTPHandler() + o = h.parent = MockOpener() + def iterable_body(): yield b"one" yield b"two" @@ -916,32 +988,19 @@ for headers in {}, {"Content-Length": 11}: req = Request("http://example.com/", iterable_body(), headers) + newreq = h.do_request_(req) if not headers: - # Having an iterable body without a Content-Length should - # raise an exception - self.assertRaises(ValueError, h.do_request_, req) + self.assertEqual(newreq.get_header('Content-length'), None) + self.assertEqual(newreq.get_header('Transfer-encoding'), + 'chunked') else: - newreq = h.do_request_(req) + self.assertEqual(int(newreq.get_header('Content-length')), 11) - # A file object. - # Test only Content-Length attribute of request. + def test_http_body_array(self): + # array.array Iterable - Content Length is calculated - file_obj = io.BytesIO() - file_obj.write(b"Something\nSomething\nSomething\n") - - for headers in {}, {"Content-Length": 30}: - req = Request("http://example.com/", file_obj, headers) - if not headers: - # Having an iterable body without a Content-Length should - # raise an exception - self.assertRaises(ValueError, h.do_request_, req) - else: - newreq = h.do_request_(req) - self.assertEqual(int(newreq.get_header('Content-length')), 30) - - file_obj.close() - - # array.array Iterable - Content Length is calculated + h = urllib.request.AbstractHTTPHandler() + o = h.parent = MockOpener() iterable_array = array.array("I",[1,2,3,4]) diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -141,17 +141,9 @@ *, cafile=None, capath=None, cadefault=False, context=None): '''Open the URL url, which can be either a string or a Request object. - *data* must be a bytes object specifying additional data to be sent to the - server, or None if no such data is needed. data may also be an iterable - object and in that case Content-Length value must be specified in the - headers. Currently HTTP requests are the only ones that use data; the HTTP - request will be a POST instead of a GET when the data parameter is - provided. - - *data* should be a buffer in the standard application/x-www-form-urlencoded - format. The urllib.parse.urlencode() function takes a mapping or sequence - of 2-tuples and returns an ASCII text string in this format. It should be - encoded to bytes before being used as the data parameter. + *data* must be an object specifying additional data to be sent to + the server, or None if no such data is needed. See Request for + details. urllib.request module uses HTTP/1.1 and includes a "Connection:close" header in its HTTP requests. @@ -1235,6 +1227,11 @@ def set_http_debuglevel(self, level): self._debuglevel = level + def _get_content_length(self, request): + return http.client.HTTPConnection._get_content_length( + request.data, + request.get_method()) + def do_request_(self, request): host = request.host if not host: @@ -1243,24 +1240,22 @@ if request.data is not None: # POST data = request.data if isinstance(data, str): - msg = "POST data should be bytes or an iterable of bytes. " \ - "It cannot be of type str." + msg = "POST data should be bytes, an iterable of bytes, " \ + "or a file object. It cannot be of type str." raise TypeError(msg) if not request.has_header('Content-type'): request.add_unredirected_header( 'Content-type', 'application/x-www-form-urlencoded') - if not request.has_header('Content-length'): - try: - mv = memoryview(data) - except TypeError: - if isinstance(data, collections.Iterable): - raise ValueError("Content-Length should be specified " - "for iterable data of type %r %r" % (type(data), - data)) + if (not request.has_header('Content-length') + and not request.has_header('Transfer-encoding')): + content_length = self._get_content_length(request) + if content_length is not None: + request.add_unredirected_header( + 'Content-length', str(content_length)) else: request.add_unredirected_header( - 'Content-length', '%d' % (len(mv) * mv.itemsize)) + 'Transfer-encoding', 'chunked') sel_host = host if request.has_proxy(): @@ -1316,7 +1311,8 @@ try: try: - h.request(req.get_method(), req.selector, req.data, headers) + h.request(req.get_method(), req.selector, req.data, headers, + encode_chunked=req.has_header('Transfer-encoding')) except OSError as err: # timeout error raise URLError(err) r = h.getresponse() diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -795,6 +795,7 @@ Jerzy Kozera Maksim Kozyarchuk Stefan Krah +Rolf Krahl Bob Kras Sebastian Kreft Holger Krekel diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -160,6 +160,14 @@ then further affects other traceback display operations in the module). Patch by Emanuel Barry. +- Issue #12319: Chunked transfer encoding support added to + http.client.HTTPConnection requests. The + urllib.request.AbstractHTTPHandler class does not enforce a Content-Length + header any more. If a HTTP request has a non-empty body, but no + Content-Length header, and the content length cannot be determined + up front, rather than throwing an error, the library now falls back + to use chunked transfer encoding. + - Issue #27664: Add to concurrent.futures.thread.ThreadPoolExecutor() the ability to specify a thread name prefix. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 03:52:25 2016 From: python-checkins at python.org (martin.panter) Date: Wed, 24 Aug 2016 07:52:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2312319=3A_Move_NEW?= =?utf-8?q?S_under_beta_1_heading?= Message-ID: <20160824075221.24131.4883.2AC7C88D@psf.io> https://hg.python.org/cpython/rev/b004da19b869 changeset: 102887:b004da19b869 user: Martin Panter date: Wed Aug 24 07:51:36 2016 +0000 summary: Issue #12319: Move NEWS under beta 1 heading files: Misc/NEWS | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,14 @@ Library ------- +- Issue #12319: Chunked transfer encoding support added to + http.client.HTTPConnection requests. The + urllib.request.AbstractHTTPHandler class does not enforce a Content-Length + header any more. If a HTTP request has a non-empty body, but no + Content-Length header, and the content length cannot be determined + up front, rather than throwing an error, the library now falls back + to use chunked transfer encoding. + - A new version of typing.py from https://github.com/python/typing: - Collection (only for 3.6) (Issue #27598) - Add FrozenSet to __all__ (upstream #261) @@ -160,14 +168,6 @@ then further affects other traceback display operations in the module). Patch by Emanuel Barry. -- Issue #12319: Chunked transfer encoding support added to - http.client.HTTPConnection requests. The - urllib.request.AbstractHTTPHandler class does not enforce a Content-Length - header any more. If a HTTP request has a non-empty body, but no - Content-Length header, and the content length cannot be determined - up front, rather than throwing an error, the library now falls back - to use chunked transfer encoding. - - Issue #27664: Add to concurrent.futures.thread.ThreadPoolExecutor() the ability to specify a thread name prefix. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 09:00:11 2016 From: python-checkins at python.org (r.david.murray) Date: Wed, 24 Aug 2016 13:00:11 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzI0NjY6IGZpeCB0?= =?utf-8?q?est_failure_on_windows=2E?= Message-ID: <20160824130010.82710.73861.FEC9415D@psf.io> https://hg.python.org/cpython/rev/63799c310b69 changeset: 102888:63799c310b69 branch: 2.7 parent: 102858:75111791110b user: R David Murray date: Wed Aug 24 08:59:47 2016 -0400 summary: #2466: fix test failure on windows. Windows symlink support was introduced in python3, so we need to skip those tests differently on python2. Patch by Xiang Zhang. files: Lib/test/test_posixpath.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_posixpath.py b/Lib/test/test_posixpath.py --- a/Lib/test/test_posixpath.py +++ b/Lib/test/test_posixpath.py @@ -1,5 +1,4 @@ import unittest -from test import symlink_support from test import test_support, test_genericpath from test import test_support as support @@ -106,7 +105,7 @@ f.write("foo") f.close() self.assertIs(posixpath.islink(test_support.TESTFN + "1"), False) - if symlink_support.can_symlink(): + if hasattr(os, 'symlink'): os.symlink(test_support.TESTFN + "1", test_support.TESTFN + "2") self.assertIs(posixpath.islink(test_support.TESTFN + "2"), True) os.remove(test_support.TESTFN + "1") @@ -211,7 +210,8 @@ finally: safe_rmdir(ABSTFN) - @symlink_support.skip_unless_symlink + @unittest.skipUnless(hasattr(os, 'symlink'), + 'Requires functional symlink implementation') def test_ismount_symlinks(self): # Symlinks are never mountpoints. try: -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Wed Aug 24 09:25:06 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 24 Aug 2016 14:25:06 +0100 Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7 2016-08-24 Message-ID: <19a50a2a-2618-4238-bb06-de1358ebdf80@irsmsx153.ger.corp.intel.com> Results for project Python 2.7, build date 2016-08-24 02:47:04 +0000 commit: 75111791110b previous commit: 370bbeba21b3 revision date: 2016-08-23 16:30:28 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.12% -0.68% 4.92% 5.24% :-) pybench 0.25% -0.12% 6.12% 3.94% :-| regex_v8 0.58% 0.23% -1.97% 10.35% :-) nbody 0.10% -0.08% 7.73% 2.45% :-) json_dump_v2 0.29% 0.48% 3.57% 10.06% :-| normal_startup 0.85% -0.38% -0.64% 3.01% :-) ssbench 0.14% -0.57% 2.40% 1.92% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-08-24/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Wed Aug 24 09:25:41 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 24 Aug 2016 14:25:41 +0100 Subject: [Python-checkins] BAD Benchmark Results for Python Default 2016-08-24 Message-ID: <1415c310-99f9-4b2c-82c3-9e607c7efa59@irsmsx153.ger.corp.intel.com> Results for project Python default, build date 2016-08-24 02:00:20 +0000 commit: ed5f5d490490 previous commit: 620b2e554be4 revision date: 2016-08-24 01:12:40 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.24% 0.48% 10.21% 15.83% :-) pybench 0.77% 0.18% 5.98% 6.88% :-( regex_v8 2.62% -1.37% -4.72% 7.26% :-| nbody 0.10% -0.25% 0.64% 8.72% :-( json_dump_v2 0.30% -1.63% -3.38% 13.81% :-| normal_startup 0.54% -0.50% -0.05% 6.49% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-default-2016-08-24/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Wed Aug 24 12:30:00 2016 From: python-checkins at python.org (zach.ware) Date: Wed, 24 Aug 2016 16:30:00 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEyNjYw?= =?utf-8?q?=3A_Backport_84bbb8d2d237?= Message-ID: <20160824162959.113118.50097.BAC2AB7C@psf.io> https://hg.python.org/cpython/rev/ef13efb89afe changeset: 102889:ef13efb89afe branch: 2.7 user: Zachary Ware date: Wed Aug 24 11:14:34 2016 -0500 summary: Issue #12660: Backport 84bbb8d2d237 files: Lib/test/test_gdb.py | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -7,6 +7,7 @@ import re import subprocess import sys +import sysconfig import unittest import sysconfig @@ -77,6 +78,9 @@ ).communicate() return out, err +if not sysconfig.is_python_build(): + raise unittest.SkipTest("test_gdb only works on source builds at the moment.") + # Verify that "gdb" was built with the embedded python support enabled: gdbpy_version, _ = run_gdb("--eval-command=python import sys; print(sys.version_info)") if not gdbpy_version: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 12:49:24 2016 From: python-checkins at python.org (vinay.sajip) Date: Wed, 24 Aug 2016 16:49:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2320124=3A_clarifi?= =?utf-8?q?ed_usage_of_the_atTime_parameter_in?= Message-ID: <20160824164923.63726.19536.2FD5F16B@psf.io> https://hg.python.org/cpython/rev/8d13c1f33d21 changeset: 102890:8d13c1f33d21 parent: 102887:b004da19b869 user: Vinay Sajip date: Wed Aug 24 17:49:15 2016 +0100 summary: Closes #20124: clarified usage of the atTime parameter in TimedRotatingFileHandler documentation. files: Doc/library/logging.handlers.rst | 52 +++++++++++++------ 1 files changed, 35 insertions(+), 17 deletions(-) diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -327,21 +327,24 @@ You can use the *when* to specify the type of *interval*. The list of possible values is below. Note that they are not case sensitive. - +----------------+-----------------------+ - | Value | Type of interval | - +================+=======================+ - | ``'S'`` | Seconds | - +----------------+-----------------------+ - | ``'M'`` | Minutes | - +----------------+-----------------------+ - | ``'H'`` | Hours | - +----------------+-----------------------+ - | ``'D'`` | Days | - +----------------+-----------------------+ - | ``'W0'-'W6'`` | Weekday (0=Monday) | - +----------------+-----------------------+ - | ``'midnight'`` | Roll over at midnight | - +----------------+-----------------------+ + +----------------+----------------------------+-------------------------+ + | Value | Type of interval | If/how *atTime* is used | + +================+============================+=========================+ + | ``'S'`` | Seconds | Ignored | + +----------------+----------------------------+-------------------------+ + | ``'M'`` | Minutes | Ignored | + +----------------+----------------------------+-------------------------+ + | ``'H'`` | Hours | Ignored | + +----------------+----------------------------+-------------------------+ + | ``'D'`` | Days | Ignored | + +----------------+----------------------------+-------------------------+ + | ``'W0'-'W6'`` | Weekday (0=Monday) | Used to compute initial | + | | | rollover time | + +----------------+----------------------------+-------------------------+ + | ``'midnight'`` | Roll over at midnight, if | Used to compute initial | + | | *atTime* not specified, | rollover time | + | | else at time *atTime* | | + +----------------+----------------------------+-------------------------+ When using weekday-based rotation, specify 'W0' for Monday, 'W1' for Tuesday, and so on up to 'W6' for Sunday. In this case, the value passed for @@ -369,7 +372,23 @@ If *atTime* is not ``None``, it must be a ``datetime.time`` instance which specifies the time of day when rollover occurs, for the cases where rollover - is set to happen "at midnight" or "on a particular weekday". + is set to happen "at midnight" or "on a particular weekday". Note that in + these cases, the *atTime* value is effectively used to compute the *initial* + rollover, and subsequent rollovers would be calculated via the normal + interval calculation. + + .. note:: Calculation of the initial rollover time is done when the handler + is initialised. Calculation of subsequent rollover times is done only + when rollover occurs, and rollover occurs only when emitting output. If + this is not kept in mind, it might lead to some confusion. For example, + if an interval of "every minute" is set, that does not mean you will + always see log files with times (in the filename) separated by a minute; + if, during application execution, logging output is generated more + frequently than once a minute, *then* you can expect to see log files + with times separated by a minute. If, on the other hand, logging messages + are only output once every five minutes (say), then there will be gaps in + the file times corresponding to the minutes where no output (and hence no + rollover) occurred. .. versionchanged:: 3.4 *atTime* parameter was added. @@ -382,7 +401,6 @@ Does a rollover, as described above. - .. method:: emit(record) Outputs the record to the file, catering for rollover as described above. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 17:44:30 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 24 Aug 2016 21:44:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=236057=3A_Merge_from_3=2E5?= Message-ID: <20160824214430.64129.45129.1A7B6EA9@psf.io> https://hg.python.org/cpython/rev/6f8042107d76 changeset: 102892:6f8042107d76 parent: 102890:8d13c1f33d21 parent: 102891:1dbd1a9a6a9c user: Berker Peksag date: Thu Aug 25 00:45:36 2016 +0300 summary: Issue #6057: Merge from 3.5 files: Doc/library/sqlite3.rst | 34 +++++++++++++++++++++++++++- Misc/ACKS | 1 + 2 files changed, 33 insertions(+), 2 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -542,7 +542,7 @@ .. literalinclude:: ../includes/sqlite3/execute_1.py :meth:`execute` will only execute a single SQL statement. If you try to execute - more than one statement with it, it will raise an ``sqlite3.Warning``. Use + more than one statement with it, it will raise an :exc:`.Warning`. Use :meth:`executescript` if you want to execute multiple SQL statements with one call. @@ -605,7 +605,7 @@ Close the cursor now (rather than whenever ``__del__`` is called). - The cursor will be unusable from this point forward; a ``ProgrammingError`` + The cursor will be unusable from this point forward; a :exc:`ProgrammingError` exception will be raised if any operation is attempted with the cursor. .. attribute:: rowcount @@ -726,6 +726,36 @@ 35.14 +.. _sqlite3-exceptions: + +Exceptions +---------- + +.. exception:: Warning + + A subclass of :exc:`Exception`. + +.. exception:: Error + + The base class of the other exceptions in this module. It is a subclass + of :exc:`Exception`. + +.. exception:: DatabaseError + + Exception raised for errors that are related to the database. + +.. exception:: IntegrityError + + Exception raised when the relational integrity of the database is affected, + e.g. a foreign key check fails. It is a subclass of :exc:`DatabaseError`. + +.. exception:: ProgrammingError + + Exception raised for programming errors, e.g. table not found or already + exists, syntax error in the SQL statement, wrong number of parameters + specified, etc. It is a subclass of :exc:`DatabaseError`. + + .. _sqlite3-types: SQLite and Python types diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1377,6 +1377,7 @@ Alex Shkop Joel Shprentz Yue Shuaijie +Jaysinh Shukla Terrel Shumway Eric Siegerman Paul Sijben -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 17:44:30 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 24 Aug 2016 21:44:30 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzYwNTc6?= =?utf-8?q?_Document_exceptions_in_sqlite3_module?= Message-ID: <20160824214429.113453.1094.9A5E1889@psf.io> https://hg.python.org/cpython/rev/1dbd1a9a6a9c changeset: 102891:1dbd1a9a6a9c branch: 3.5 parent: 102879:7ac8e8568da2 user: Berker Peksag date: Thu Aug 25 00:45:07 2016 +0300 summary: Issue #6057: Document exceptions in sqlite3 module Patch by Jaysinh Shukla and St?phane Wirtel. files: Doc/library/sqlite3.rst | 34 +++++++++++++++++++++++++++- Misc/ACKS | 1 + 2 files changed, 33 insertions(+), 2 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -542,7 +542,7 @@ .. literalinclude:: ../includes/sqlite3/execute_1.py :meth:`execute` will only execute a single SQL statement. If you try to execute - more than one statement with it, it will raise an ``sqlite3.Warning``. Use + more than one statement with it, it will raise an :exc:`.Warning`. Use :meth:`executescript` if you want to execute multiple SQL statements with one call. @@ -605,7 +605,7 @@ Close the cursor now (rather than whenever ``__del__`` is called). - The cursor will be unusable from this point forward; a ``ProgrammingError`` + The cursor will be unusable from this point forward; a :exc:`ProgrammingError` exception will be raised if any operation is attempted with the cursor. .. attribute:: rowcount @@ -719,6 +719,36 @@ 35.14 +.. _sqlite3-exceptions: + +Exceptions +---------- + +.. exception:: Warning + + A subclass of :exc:`Exception`. + +.. exception:: Error + + The base class of the other exceptions in this module. It is a subclass + of :exc:`Exception`. + +.. exception:: DatabaseError + + Exception raised for errors that are related to the database. + +.. exception:: IntegrityError + + Exception raised when the relational integrity of the database is affected, + e.g. a foreign key check fails. It is a subclass of :exc:`DatabaseError`. + +.. exception:: ProgrammingError + + Exception raised for programming errors, e.g. table not found or already + exists, syntax error in the SQL statement, wrong number of parameters + specified, etc. It is a subclass of :exc:`DatabaseError`. + + .. _sqlite3-types: SQLite and Python types diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1360,6 +1360,7 @@ Alex Shkop Joel Shprentz Yue Shuaijie +Jaysinh Shukla Terrel Shumway Eric Siegerman Paul Sijben -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 17:49:52 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 24 Aug 2016 21:49:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E5?= Message-ID: <20160824214952.113433.61522.274EC4EB@psf.io> https://hg.python.org/cpython/rev/aa73c28c7ba1 changeset: 102894:aa73c28c7ba1 parent: 102892:6f8042107d76 parent: 102893:78c039440d6b user: Berker Peksag date: Thu Aug 25 00:50:59 2016 +0300 summary: Merge from 3.5 files: Doc/library/sqlite3.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -542,7 +542,7 @@ .. literalinclude:: ../includes/sqlite3/execute_1.py :meth:`execute` will only execute a single SQL statement. If you try to execute - more than one statement with it, it will raise an :exc:`.Warning`. Use + more than one statement with it, it will raise a :exc:`.Warning`. Use :meth:`executescript` if you want to execute multiple SQL statements with one call. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 17:49:53 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 24 Aug 2016 21:49:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_typo_in_Cu?= =?utf-8?b?cnNvci5leGVjdXRlKCku?= Message-ID: <20160824214951.17866.527.71ACB711@psf.io> https://hg.python.org/cpython/rev/78c039440d6b changeset: 102893:78c039440d6b branch: 3.5 parent: 102891:1dbd1a9a6a9c user: Berker Peksag date: Thu Aug 25 00:50:24 2016 +0300 summary: Fix typo in Cursor.execute(). files: Doc/library/sqlite3.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -542,7 +542,7 @@ .. literalinclude:: ../includes/sqlite3/execute_1.py :meth:`execute` will only execute a single SQL statement. If you try to execute - more than one statement with it, it will raise an :exc:`.Warning`. Use + more than one statement with it, it will raise a :exc:`.Warning`. Use :meth:`executescript` if you want to execute multiple SQL statements with one call. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 17:53:35 2016 From: python-checkins at python.org (terry.reedy) Date: Wed, 24 Aug 2016 21:53:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_misspellin?= =?utf-8?q?g?= Message-ID: <20160824215334.68217.23073.993FE943@psf.io> https://hg.python.org/cpython/rev/37335860e0f7 changeset: 102895:37335860e0f7 branch: 3.5 parent: 102893:78c039440d6b user: Terry Jan Reedy date: Wed Aug 24 17:52:57 2016 -0400 summary: Fix misspelling files: Doc/library/pdb.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -158,7 +158,7 @@ By default, Pdb sets a handler for the SIGINT signal (which is sent when the user presses :kbd:`Ctrl-C` on the console) when you give a ``continue`` command. This allows you to break into the debugger again by pressing :kbd:`Ctrl-C`. If you - want Pdb not to touch the SIGINT handler, set *nosigint* tot true. + want Pdb not to touch the SIGINT handler, set *nosigint* to true. Example call to enable tracing with *skip*:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 17:53:35 2016 From: python-checkins at python.org (terry.reedy) Date: Wed, 24 Aug 2016 21:53:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20160824215335.64134.62013.AF05BE80@psf.io> https://hg.python.org/cpython/rev/cfc923275368 changeset: 102896:cfc923275368 parent: 102894:aa73c28c7ba1 parent: 102895:37335860e0f7 user: Terry Jan Reedy date: Wed Aug 24 17:53:16 2016 -0400 summary: Merge with 3.5 files: Doc/library/pdb.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -158,7 +158,7 @@ By default, Pdb sets a handler for the SIGINT signal (which is sent when the user presses :kbd:`Ctrl-C` on the console) when you give a ``continue`` command. This allows you to break into the debugger again by pressing :kbd:`Ctrl-C`. If you - want Pdb not to touch the SIGINT handler, set *nosigint* tot true. + want Pdb not to touch the SIGINT handler, set *nosigint* to true. Example call to enable tracing with *skip*:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 18:07:29 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 24 Aug 2016 22:07:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Use_Py=5Fssize=5Ft_type_fo?= =?utf-8?q?r_number_of_arguments?= Message-ID: <20160824220728.39133.80408.FE7F1962@psf.io> https://hg.python.org/cpython/rev/9479da828358 changeset: 102897:9479da828358 user: Victor Stinner date: Thu Aug 25 00:04:09 2016 +0200 summary: Use Py_ssize_t type for number of arguments Issue #27848: use Py_ssize_t rather than C int for the number of function positional and keyword arguments. files: Include/abstract.h | 2 +- Include/funcobject.h | 3 +- Include/methodobject.h | 3 +- Objects/abstract.c | 2 +- Objects/methodobject.c | 9 +- Python/bltinmodule.c | 2 +- Python/ceval.c | 197 ++++++++++++++++------------ 7 files changed, 128 insertions(+), 90 deletions(-) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -280,7 +280,7 @@ Return the result on success. Raise an exception on return NULL on error. */ PyAPI_FUNC(PyObject *) _PyObject_FastCallDict(PyObject *func, - PyObject **args, int nargs, + PyObject **args, Py_ssize_t nargs, PyObject *kwargs); #define _PyObject_FastCall(func, args, nargs) \ diff --git a/Include/funcobject.h b/Include/funcobject.h --- a/Include/funcobject.h +++ b/Include/funcobject.h @@ -61,7 +61,8 @@ #ifndef Py_LIMITED_API PyAPI_FUNC(PyObject *) _PyFunction_FastCallDict( PyObject *func, - PyObject **args, int nargs, + PyObject **args, + Py_ssize_t nargs, PyObject *kwargs); #endif diff --git a/Include/methodobject.h b/Include/methodobject.h --- a/Include/methodobject.h +++ b/Include/methodobject.h @@ -39,7 +39,8 @@ #ifndef Py_LIMITED_API PyAPI_FUNC(PyObject *) _PyCFunction_FastCallDict(PyObject *func, - PyObject **args, int nargs, + PyObject **args, + Py_ssize_t nargs, PyObject *kwargs); #endif diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2255,7 +2255,7 @@ } PyObject * -_PyObject_FastCallDict(PyObject *func, PyObject **args, int nargs, +_PyObject_FastCallDict(PyObject *func, PyObject **args, Py_ssize_t nargs, PyObject *kwargs) { ternaryfunc call; diff --git a/Objects/methodobject.c b/Objects/methodobject.c --- a/Objects/methodobject.c +++ b/Objects/methodobject.c @@ -146,15 +146,20 @@ } PyObject * -_PyCFunction_FastCallDict(PyObject *func_obj, PyObject **args, int nargs, +_PyCFunction_FastCallDict(PyObject *func_obj, PyObject **args, Py_ssize_t nargs, PyObject *kwargs) { - PyCFunctionObject* func = (PyCFunctionObject*)func_obj; + PyCFunctionObject *func = (PyCFunctionObject*)func_obj; PyCFunction meth = PyCFunction_GET_FUNCTION(func); PyObject *self = PyCFunction_GET_SELF(func); PyObject *result; int flags; + assert(func != NULL); + assert(nargs >= 0); + assert(nargs == 0 || args != NULL); + assert(kwargs == NULL || PyDict_Check(kwargs)); + /* _PyCFunction_FastCallDict() must not be called with an exception set, because it may clear it (directly or indirectly) and so the caller loses its exception */ diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -2095,7 +2095,7 @@ PyObject *callable; static char *kwlist[] = {"iterable", "key", "reverse", 0}; int reverse; - int nargs; + Py_ssize_t nargs; /* args 1-3 should match listsort in Objects/listobject.c */ if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|Oi:sorted", diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -113,13 +113,13 @@ #else static PyObject * call_function(PyObject ***, int); #endif -static PyObject * fast_function(PyObject *, PyObject **, int, int, int); -static PyObject * do_call(PyObject *, PyObject ***, int, int); -static PyObject * ext_do_call(PyObject *, PyObject ***, int, int, int); -static PyObject * update_keyword_args(PyObject *, int, PyObject ***, +static PyObject * fast_function(PyObject *, PyObject **, Py_ssize_t, Py_ssize_t); +static PyObject * do_call(PyObject *, PyObject ***, Py_ssize_t, Py_ssize_t); +static PyObject * ext_do_call(PyObject *, PyObject ***, int, Py_ssize_t, Py_ssize_t); +static PyObject * update_keyword_args(PyObject *, Py_ssize_t, PyObject ***, PyObject *); -static PyObject * update_star_args(int, int, PyObject *, PyObject ***); -static PyObject * load_args(PyObject ***, int); +static PyObject * update_star_args(Py_ssize_t, Py_ssize_t, PyObject *, PyObject ***); +static PyObject * load_args(PyObject ***, Py_ssize_t); #define CALL_FLAG_VAR 1 #define CALL_FLAG_KW 2 @@ -2558,7 +2558,7 @@ TARGET(BUILD_TUPLE_UNPACK) TARGET(BUILD_LIST_UNPACK) { int convert_to_tuple = opcode == BUILD_TUPLE_UNPACK; - int i; + Py_ssize_t i; PyObject *sum = PyList_New(0); PyObject *return_value; if (sum == NULL) @@ -2611,7 +2611,7 @@ } TARGET(BUILD_SET_UNPACK) { - int i; + Py_ssize_t i; PyObject *sum = PySet_New(NULL); if (sum == NULL) goto error; @@ -2630,7 +2630,7 @@ } TARGET(BUILD_MAP) { - int i; + Py_ssize_t i; PyObject *map = _PyDict_NewPresized((Py_ssize_t)oparg); if (map == NULL) goto error; @@ -2654,7 +2654,7 @@ } TARGET(BUILD_CONST_KEY_MAP) { - int i; + Py_ssize_t i; PyObject *map; PyObject *keys = TOP(); if (!PyTuple_CheckExact(keys) || @@ -2691,7 +2691,7 @@ int with_call = opcode == BUILD_MAP_UNPACK_WITH_CALL; int num_maps; int function_location; - int i; + Py_ssize_t i; PyObject *sum = PyDict_New(); if (sum == NULL) goto error; @@ -3265,16 +3265,20 @@ TARGET(CALL_FUNCTION_VAR) TARGET(CALL_FUNCTION_KW) TARGET(CALL_FUNCTION_VAR_KW) { - int na = oparg & 0xff; - int nk = (oparg>>8) & 0xff; + Py_ssize_t nargs = oparg & 0xff; + Py_ssize_t nkwargs = (oparg>>8) & 0xff; int flags = (opcode - CALL_FUNCTION) & 3; - int n = na + 2 * nk; + Py_ssize_t n; PyObject **pfunc, *func, **sp, *res; PCALL(PCALL_ALL); - if (flags & CALL_FLAG_VAR) + + n = nargs + 2 * nkwargs; + if (flags & CALL_FLAG_VAR) { n++; - if (flags & CALL_FLAG_KW) + } + if (flags & CALL_FLAG_KW) { n++; + } pfunc = stack_pointer - n - 1; func = *pfunc; @@ -3285,13 +3289,14 @@ func = PyMethod_GET_FUNCTION(func); Py_INCREF(func); Py_SETREF(*pfunc, self); - na++; - /* n++; */ - } else + nargs++; + } + else { Py_INCREF(func); + } sp = stack_pointer; READ_TIMESTAMP(intr0); - res = ext_do_call(func, &sp, flags, na, nk); + res = ext_do_call(func, &sp, flags, nargs, nkwargs); READ_TIMESTAMP(intr1); stack_pointer = sp; Py_DECREF(func); @@ -3579,9 +3584,11 @@ state came into existence in this frame. (An uncaught exception would have why == WHY_EXCEPTION, and we wouldn't be here). */ int i; - for (i = 0; i < f->f_iblock; i++) - if (f->f_blockstack[i].b_type == EXCEPT_HANDLER) + for (i = 0; i < f->f_iblock; i++) { + if (f->f_blockstack[i].b_type == EXCEPT_HANDLER) { break; + } + } if (i == f->f_iblock) /* We did not create this exception. */ restore_and_clear_exc_state(tstate, f); @@ -3692,12 +3699,12 @@ } static void -missing_arguments(PyCodeObject *co, int missing, int defcount, +missing_arguments(PyCodeObject *co, Py_ssize_t missing, Py_ssize_t defcount, PyObject **fastlocals) { - int i, j = 0; - int start, end; - int positional = defcount != -1; + Py_ssize_t i, j = 0; + Py_ssize_t start, end; + int positional = (defcount != -1); const char *kind = positional ? "positional" : "keyword-only"; PyObject *missing_names; @@ -3730,33 +3737,39 @@ } static void -too_many_positional(PyCodeObject *co, int given, int defcount, PyObject **fastlocals) +too_many_positional(PyCodeObject *co, Py_ssize_t given, Py_ssize_t defcount, + PyObject **fastlocals) { int plural; - int kwonly_given = 0; - int i; + Py_ssize_t kwonly_given = 0; + Py_ssize_t i; PyObject *sig, *kwonly_sig; + Py_ssize_t co_argcount = co->co_argcount; assert((co->co_flags & CO_VARARGS) == 0); /* Count missing keyword-only args. */ - for (i = co->co_argcount; i < co->co_argcount + co->co_kwonlyargcount; i++) - if (GETLOCAL(i) != NULL) + for (i = co_argcount; i < co_argcount + co->co_kwonlyargcount; i++) { + if (GETLOCAL(i) != NULL) { kwonly_given++; + } + } if (defcount) { - int atleast = co->co_argcount - defcount; + Py_ssize_t atleast = co_argcount - defcount; plural = 1; - sig = PyUnicode_FromFormat("from %d to %d", atleast, co->co_argcount); + sig = PyUnicode_FromFormat("from %zd to %zd", atleast, co_argcount); } else { - plural = co->co_argcount != 1; - sig = PyUnicode_FromFormat("%d", co->co_argcount); + plural = (co_argcount != 1); + sig = PyUnicode_FromFormat("%zd", co_argcount); } if (sig == NULL) return; if (kwonly_given) { - const char *format = " positional argument%s (and %d keyword-only argument%s)"; - kwonly_sig = PyUnicode_FromFormat(format, given != 1 ? "s" : "", kwonly_given, - kwonly_given != 1 ? "s" : ""); + const char *format = " positional argument%s (and %zd keyword-only argument%s)"; + kwonly_sig = PyUnicode_FromFormat(format, + given != 1 ? "s" : "", + kwonly_given, + kwonly_given != 1 ? "s" : ""); if (kwonly_sig == NULL) { Py_DECREF(sig); return; @@ -3768,7 +3781,7 @@ assert(kwonly_sig != NULL); } PyErr_Format(PyExc_TypeError, - "%U() takes %U positional argument%s but %d%U %s given", + "%U() takes %U positional argument%s but %zd%U %s given", co->co_name, sig, plural ? "s" : "", @@ -3786,8 +3799,10 @@ static PyObject * _PyEval_EvalCodeWithName(PyObject *_co, PyObject *globals, PyObject *locals, - PyObject **args, int argcount, PyObject **kws, int kwcount, - PyObject **defs, int defcount, PyObject *kwdefs, PyObject *closure, + PyObject **args, Py_ssize_t argcount, + PyObject **kws, Py_ssize_t kwcount, + PyObject **defs, Py_ssize_t defcount, + PyObject *kwdefs, PyObject *closure, PyObject *name, PyObject *qualname) { PyCodeObject* co = (PyCodeObject*)_co; @@ -4633,16 +4648,16 @@ } static void -err_args(PyObject *func, int flags, int nargs) +err_args(PyObject *func, int flags, Py_ssize_t nargs) { if (flags & METH_NOARGS) PyErr_Format(PyExc_TypeError, - "%.200s() takes no arguments (%d given)", + "%.200s() takes no arguments (%zd given)", ((PyCFunctionObject *)func)->m_ml->ml_name, nargs); else PyErr_Format(PyExc_TypeError, - "%.200s() takes exactly one argument (%d given)", + "%.200s() takes exactly one argument (%zd given)", ((PyCFunctionObject *)func)->m_ml->ml_name, nargs); } @@ -4685,9 +4700,9 @@ #endif ) { - int na = oparg & 0xff; - int nk = (oparg>>8) & 0xff; - int n = na + 2 * nk; + Py_ssize_t nargs = oparg & 0xff; + Py_ssize_t nkwargs = (oparg>>8) & 0xff; + int n = nargs + 2 * nkwargs; PyObject **pfunc = (*pp_stack) - n - 1; PyObject *func = *pfunc; PyObject *x, *w; @@ -4695,7 +4710,7 @@ /* Always dispatch PyCFunction first, because these are presumed to be the most frequent callable object. */ - if (PyCFunction_Check(func) && nk == 0) { + if (PyCFunction_Check(func) && nkwargs == 0) { int flags = PyCFunction_GET_FLAGS(func); PyThreadState *tstate = PyThreadState_GET(); @@ -4703,12 +4718,12 @@ if (flags & (METH_NOARGS | METH_O)) { PyCFunction meth = PyCFunction_GET_FUNCTION(func); PyObject *self = PyCFunction_GET_SELF(func); - if (flags & METH_NOARGS && na == 0) { + if (flags & METH_NOARGS && nargs == 0) { C_TRACE(x, (*meth)(self,NULL)); x = _Py_CheckFunctionResult(func, x, NULL); } - else if (flags & METH_O && na == 1) { + else if (flags & METH_O && nargs == 1) { PyObject *arg = EXT_POP(*pp_stack); C_TRACE(x, (*meth)(self,arg)); Py_DECREF(arg); @@ -4716,13 +4731,13 @@ x = _Py_CheckFunctionResult(func, x, NULL); } else { - err_args(func, flags, na); + err_args(func, flags, nargs); x = NULL; } } else { PyObject *callargs; - callargs = load_args(pp_stack, na); + callargs = load_args(pp_stack, nargs); if (callargs != NULL) { READ_TIMESTAMP(*pintr0); C_TRACE(x, PyCFunction_Call(func,callargs,NULL)); @@ -4744,16 +4759,18 @@ func = PyMethod_GET_FUNCTION(func); Py_INCREF(func); Py_SETREF(*pfunc, self); - na++; + nargs++; n++; - } else + } + else { Py_INCREF(func); + } READ_TIMESTAMP(*pintr0); if (PyFunction_Check(func)) { - x = fast_function(func, (*pp_stack) - n, n, na, nk); + x = fast_function(func, (*pp_stack) - n, nargs, nkwargs); } else { - x = do_call(func, pp_stack, na, nk); + x = do_call(func, pp_stack, nargs, nkwargs); } READ_TIMESTAMP(*pintr1); Py_DECREF(func); @@ -4785,7 +4802,7 @@ */ static PyObject* -_PyFunction_FastCallNoKw(PyCodeObject *co, PyObject **args, Py_ssize_t na, +_PyFunction_FastCallNoKw(PyCodeObject *co, PyObject **args, Py_ssize_t nargs, PyObject *globals) { PyFrameObject *f; @@ -4808,7 +4825,7 @@ fastlocals = f->f_localsplus; - for (i = 0; i < na; i++) { + for (i = 0; i < nargs; i++) { Py_INCREF(*args); fastlocals[i] = *args++; } @@ -4824,7 +4841,7 @@ /* Similar to _PyFunction_FastCall() but keywords are passed a (key, value) pairs in stack */ static PyObject * -fast_function(PyObject *func, PyObject **stack, int n, int nargs, int nk) +fast_function(PyObject *func, PyObject **stack, Py_ssize_t nargs, Py_ssize_t nkwargs) { PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func); PyObject *globals = PyFunction_GET_GLOBALS(func); @@ -4836,7 +4853,7 @@ PCALL(PCALL_FUNCTION); PCALL(PCALL_FAST_FUNCTION); - if (co->co_kwonlyargcount == 0 && nk == 0 && + if (co->co_kwonlyargcount == 0 && nkwargs == 0 && co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) { if (argdefs == NULL && co->co_argcount == nargs) { @@ -4867,13 +4884,13 @@ } return _PyEval_EvalCodeWithName((PyObject*)co, globals, (PyObject *)NULL, stack, nargs, - stack + nargs, nk, + stack + nargs, nkwargs, d, nd, kwdefs, closure, name, qualname); } PyObject * -_PyFunction_FastCallDict(PyObject *func, PyObject **args, int nargs, +_PyFunction_FastCallDict(PyObject *func, PyObject **args, Py_ssize_t nargs, PyObject *kwargs) { PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func); @@ -4882,13 +4899,15 @@ PyObject *kwdefs, *closure, *name, *qualname; PyObject *kwtuple, **k; PyObject **d; - int nd; - Py_ssize_t nk; + Py_ssize_t nd, nk; PyObject *result; PCALL(PCALL_FUNCTION); PCALL(PCALL_FAST_FUNCTION); + assert(func != NULL); + assert(nargs >= 0); + assert(nargs == 0 || args != NULL); assert(kwargs == NULL || PyDict_Check(kwargs)); if (co->co_kwonlyargcount == 0 && @@ -4949,7 +4968,7 @@ result = _PyEval_EvalCodeWithName((PyObject*)co, globals, (PyObject *)NULL, args, nargs, - k, (int)nk, + k, nk, d, nd, kwdefs, closure, name, qualname); Py_XDECREF(kwtuple); @@ -4957,7 +4976,7 @@ } static PyObject * -update_keyword_args(PyObject *orig_kwdict, int nk, PyObject ***pp_stack, +update_keyword_args(PyObject *orig_kwdict, Py_ssize_t nk, PyObject ***pp_stack, PyObject *func) { PyObject *kwdict = NULL; @@ -4997,7 +5016,7 @@ } static PyObject * -update_star_args(int nstack, int nstar, PyObject *stararg, +update_star_args(Py_ssize_t nstack, Py_ssize_t nstar, PyObject *stararg, PyObject ***pp_stack) { PyObject *callargs, *w; @@ -5021,14 +5040,16 @@ if (callargs == NULL) { return NULL; } + if (nstar) { - int i; + Py_ssize_t i; for (i = 0; i < nstar; i++) { - PyObject *a = PyTuple_GET_ITEM(stararg, i); - Py_INCREF(a); - PyTuple_SET_ITEM(callargs, nstack + i, a); + PyObject *arg = PyTuple_GET_ITEM(stararg, i); + Py_INCREF(arg); + PyTuple_SET_ITEM(callargs, nstack + i, arg); } } + while (--nstack >= 0) { w = EXT_POP(*pp_stack); PyTuple_SET_ITEM(callargs, nstack, w); @@ -5037,7 +5058,7 @@ } static PyObject * -load_args(PyObject ***pp_stack, int na) +load_args(PyObject ***pp_stack, Py_ssize_t na) { PyObject *args = PyTuple_New(na); PyObject *w; @@ -5052,18 +5073,18 @@ } static PyObject * -do_call(PyObject *func, PyObject ***pp_stack, int na, int nk) +do_call(PyObject *func, PyObject ***pp_stack, Py_ssize_t nargs, Py_ssize_t nkwargs) { PyObject *callargs = NULL; PyObject *kwdict = NULL; PyObject *result = NULL; - if (nk > 0) { - kwdict = update_keyword_args(NULL, nk, pp_stack, func); + if (nkwargs > 0) { + kwdict = update_keyword_args(NULL, nkwargs, pp_stack, func); if (kwdict == NULL) goto call_fail; } - callargs = load_args(pp_stack, na); + callargs = load_args(pp_stack, nargs); if (callargs == NULL) goto call_fail; #ifdef CALL_PROFILE @@ -5095,9 +5116,10 @@ } static PyObject * -ext_do_call(PyObject *func, PyObject ***pp_stack, int flags, int na, int nk) +ext_do_call(PyObject *func, PyObject ***pp_stack, int flags, + Py_ssize_t nargs, Py_ssize_t nkwargs) { - int nstar = 0; + Py_ssize_t nstar; PyObject *callargs = NULL; PyObject *stararg = NULL; PyObject *kwdict = NULL; @@ -5132,8 +5154,9 @@ kwdict = d; } } - if (nk > 0) { - kwdict = update_keyword_args(kwdict, nk, pp_stack, func); + + if (nkwargs > 0) { + kwdict = update_keyword_args(kwdict, nkwargs, pp_stack, func); if (kwdict == NULL) goto ext_call_fail; } @@ -5161,9 +5184,15 @@ } nstar = PyTuple_GET_SIZE(stararg); } - callargs = update_star_args(na, nstar, stararg, pp_stack); - if (callargs == NULL) + else { + nstar = 0; + } + + callargs = update_star_args(nargs, nstar, stararg, pp_stack); + if (callargs == NULL) { goto ext_call_fail; + } + #ifdef CALL_PROFILE /* At this point, we have to look at the type of func to update the call stats properly. Do it here so as to avoid @@ -5184,8 +5213,10 @@ PyThreadState *tstate = PyThreadState_GET(); C_TRACE(result, PyCFunction_Call(func, callargs, kwdict)); } - else + else { result = PyObject_Call(func, callargs, kwdict); + } + ext_call_fail: Py_XDECREF(callargs); Py_XDECREF(kwdict); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 18:12:25 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 24 Aug 2016 22:12:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_typo_in_test_name?= Message-ID: <20160824221225.82537.31789.F9D64BDA@psf.io> https://hg.python.org/cpython/rev/2592b3c3f10d changeset: 102898:2592b3c3f10d user: Berker Peksag date: Thu Aug 25 01:13:34 2016 +0300 summary: Fix typo in test name Noticed by Xiang Zhang. files: Lib/test/test_httpservers.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_httpservers.py b/Lib/test/test_httpservers.py --- a/Lib/test/test_httpservers.py +++ b/Lib/test/test_httpservers.py @@ -190,7 +190,7 @@ res = self.con.getresponse() self.assertEqual(res.status, HTTPStatus.NOT_IMPLEMENTED) - def test_head_keep_alive(self): + def test_header_keep_alive(self): self.con._http_vsn_str = 'HTTP/1.1' self.con.putrequest('GET', '/') self.con.putheader('Connection', 'keep-alive') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 18:30:18 2016 From: python-checkins at python.org (alexander.belopolsky) Date: Wed, 24 Aug 2016 22:30:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2327595=3A_Documen?= =?utf-8?q?t_PEP_495_=28Local_Time_Disambiguation=29_features=2E?= Message-ID: <20160824223018.20243.43800.07235F56@psf.io> https://hg.python.org/cpython/rev/dcd6d6be81a7 changeset: 102899:dcd6d6be81a7 user: Alexander Belopolsky date: Wed Aug 24 18:30:16 2016 -0400 summary: Closes #27595: Document PEP 495 (Local Time Disambiguation) features. files: Doc/includes/tzinfo-examples.py | 130 ++++++++------- Doc/library/datetime.rst | 159 ++++++++++++++----- 2 files changed, 183 insertions(+), 106 deletions(-) diff --git a/Doc/includes/tzinfo-examples.py b/Doc/includes/tzinfo-examples.py --- a/Doc/includes/tzinfo-examples.py +++ b/Doc/includes/tzinfo-examples.py @@ -1,46 +1,13 @@ -from datetime import tzinfo, timedelta, datetime +from datetime import tzinfo, timedelta, datetime, timezone ZERO = timedelta(0) HOUR = timedelta(hours=1) - -# A UTC class. - -class UTC(tzinfo): - """UTC""" - - def utcoffset(self, dt): - return ZERO - - def tzname(self, dt): - return "UTC" - - def dst(self, dt): - return ZERO - -utc = UTC() - -# A class building tzinfo objects for fixed-offset time zones. -# Note that FixedOffset(0, "UTC") is a different way to build a -# UTC tzinfo object. - -class FixedOffset(tzinfo): - """Fixed offset in minutes east from UTC.""" - - def __init__(self, offset, name): - self.__offset = timedelta(minutes=offset) - self.__name = name - - def utcoffset(self, dt): - return self.__offset - - def tzname(self, dt): - return self.__name - - def dst(self, dt): - return ZERO +SECOND = timedelta(seconds=1) # A class capturing the platform's idea of local time. - +# (May result in wrong values on historical times in +# timezones where UTC offset and/or the DST rules had +# changed in the past.) import time as _time STDOFFSET = timedelta(seconds = -_time.timezone) @@ -53,6 +20,16 @@ class LocalTimezone(tzinfo): + def fromutc(self, dt): + assert dt.tzinfo is self + stamp = (dt - datetime(1970, 1, 1, tzinfo=self)) // SECOND + args = _time.localtime(stamp)[:6] + dst_diff = DSTDIFF // SECOND + # Detect fold + fold = (args == _time.localtime(stamp - dst_diff)) + return datetime(*args, microsecond=dt.microsecond, + tzinfo=self, fold=fold) + def utcoffset(self, dt): if self._isdst(dt): return DSTOFFSET @@ -99,20 +76,37 @@ # In the US, since 2007, DST starts at 2am (standard time) on the second # Sunday in March, which is the first Sunday on or after Mar 8. DSTSTART_2007 = datetime(1, 3, 8, 2) -# and ends at 2am (DST time; 1am standard time) on the first Sunday of Nov. -DSTEND_2007 = datetime(1, 11, 1, 1) +# and ends at 2am (DST time) on the first Sunday of Nov. +DSTEND_2007 = datetime(1, 11, 1, 2) # From 1987 to 2006, DST used to start at 2am (standard time) on the first -# Sunday in April and to end at 2am (DST time; 1am standard time) on the last +# Sunday in April and to end at 2am (DST time) on the last # Sunday of October, which is the first Sunday on or after Oct 25. DSTSTART_1987_2006 = datetime(1, 4, 1, 2) -DSTEND_1987_2006 = datetime(1, 10, 25, 1) +DSTEND_1987_2006 = datetime(1, 10, 25, 2) # From 1967 to 1986, DST used to start at 2am (standard time) on the last -# Sunday in April (the one on or after April 24) and to end at 2am (DST time; -# 1am standard time) on the last Sunday of October, which is the first Sunday +# Sunday in April (the one on or after April 24) and to end at 2am (DST time) +# on the last Sunday of October, which is the first Sunday # on or after Oct 25. DSTSTART_1967_1986 = datetime(1, 4, 24, 2) DSTEND_1967_1986 = DSTEND_1987_2006 +def us_dst_range(year): + # Find start and end times for US DST. For years before 1967, return + # start = end for no DST. + if 2006 < year: + dststart, dstend = DSTSTART_2007, DSTEND_2007 + elif 1986 < year < 2007: + dststart, dstend = DSTSTART_1987_2006, DSTEND_1987_2006 + elif 1966 < year < 1987: + dststart, dstend = DSTSTART_1967_1986, DSTEND_1967_1986 + else: + return (datetime(year, 1, 1), ) * 2 + + start = first_sunday_on_or_after(dststart.replace(year=year)) + end = first_sunday_on_or_after(dstend.replace(year=year)) + return start, end + + class USTimeZone(tzinfo): def __init__(self, hours, reprname, stdname, dstname): @@ -141,27 +135,39 @@ # implementation) passes a datetime with dt.tzinfo is self. return ZERO assert dt.tzinfo is self - - # Find start and end times for US DST. For years before 1967, return - # ZERO for no DST. - if 2006 < dt.year: - dststart, dstend = DSTSTART_2007, DSTEND_2007 - elif 1986 < dt.year < 2007: - dststart, dstend = DSTSTART_1987_2006, DSTEND_1987_2006 - elif 1966 < dt.year < 1987: - dststart, dstend = DSTSTART_1967_1986, DSTEND_1967_1986 - else: - return ZERO - - start = first_sunday_on_or_after(dststart.replace(year=dt.year)) - end = first_sunday_on_or_after(dstend.replace(year=dt.year)) - + start, end = us_dst_range(dt.year) # Can't compare naive to aware objects, so strip the timezone from # dt first. - if start <= dt.replace(tzinfo=None) < end: + dt = dt.replace(tzinfo=None) + if start + HOUR <= dt < end - HOUR: + # DST is in effect. return HOUR - else: - return ZERO + if end - HOUR <= dt < end: + # Fold (an ambiguous hour): use dt.fold to disambiguate. + return ZERO if dt.fold else HOUR + if start <= dt < start + HOUR: + # Gap (a non-existent hour): reverse the fold rule. + return HOUR if dt.fold else ZERO + # DST is off. + return ZERO + + def fromutc(self, dt): + assert dt.tzinfo is self + start, end = us_dst_range(dt.year) + start = start.replace(tzinfo=self) + end = end.replace(tzinfo=self) + std_time = dt + self.stdoffset + dst_time = std_time + HOUR + if end <= dst_time < end + HOUR: + # Repeated hour + return std_time.replace(fold=1) + if std_time < start or dst_time >= end: + # Standard time + return std_time + if start <= std_time < end - HOUR: + # Daylight saving time + return dst_time + Eastern = USTimeZone(-5, "Eastern", "EST", "EDT") Central = USTimeZone(-6, "Central", "CST", "CDT") diff --git a/Doc/library/datetime.rst b/Doc/library/datetime.rst --- a/Doc/library/datetime.rst +++ b/Doc/library/datetime.rst @@ -522,7 +522,7 @@ Instance methods: -.. method:: date.replace(year, month, day) +.. method:: date.replace(year=self.year, month=self.month, day=self.day) Return a date with the same value, except for those parameters given new values by whichever keyword arguments are specified. For example, if ``d == @@ -683,22 +683,26 @@ Constructor: -.. class:: datetime(year, month, day, hour=0, minute=0, second=0, microsecond=0, tzinfo=None) +.. class:: datetime(year, month, day, hour=0, minute=0, second=0, microsecond=0, tzinfo=None, *, fold=0) The year, month and day arguments are required. *tzinfo* may be ``None``, or an instance of a :class:`tzinfo` subclass. The remaining arguments may be integers, in the following ranges: - * ``MINYEAR <= year <= MAXYEAR`` - * ``1 <= month <= 12`` - * ``1 <= day <= number of days in the given month and year`` - * ``0 <= hour < 24`` - * ``0 <= minute < 60`` - * ``0 <= second < 60`` - * ``0 <= microsecond < 1000000`` + * ``MINYEAR <= year <= MAXYEAR``, + * ``1 <= month <= 12``, + * ``1 <= day <= number of days in the given month and year``, + * ``0 <= hour < 24``, + * ``0 <= minute < 60``, + * ``0 <= second < 60``, + * ``0 <= microsecond < 1000000``, + * ``fold in [0, 1]``. If an argument outside those ranges is given, :exc:`ValueError` is raised. + .. versionadded:: 3.6 + Added the ``fold`` argument. + Other constructors, all class methods: .. classmethod:: datetime.today() @@ -758,6 +762,8 @@ instead of :exc:`ValueError` on :c:func:`localtime` or :c:func:`gmtime` failure. + .. versionchanged:: 3.6 + :meth:`fromtimestamp` may return instances with :attr:`.fold` set to 1. .. classmethod:: datetime.utcfromtimestamp(timestamp) @@ -794,7 +800,7 @@ microsecond of the result are all 0, and :attr:`.tzinfo` is ``None``. -.. classmethod:: datetime.combine(date, time[, tzinfo]) +.. classmethod:: datetime.combine(date, time, tzinfo=self.tzinfo) Return a new :class:`.datetime` object whose date components are equal to the given :class:`date` object's, and whose time components @@ -886,6 +892,16 @@ or ``None`` if none was passed. +.. attribute:: datetime.fold + + In ``[0, 1]``. Used to disambiguate wall times during a repeated interval. (A + repeated interval occurs when clocks are rolled back at the end of daylight saving + time or when the UTC offset for the current zone is decreased for political reasons.) + The value 0 (1) represents the earlier (later) of the two moments with the same wall + time representation. + + .. versionadded:: 3.6 + Supported operations: +---------------------------------------+--------------------------------+ @@ -974,23 +990,34 @@ .. method:: datetime.time() - Return :class:`.time` object with same hour, minute, second and microsecond. + Return :class:`.time` object with same hour, minute, second, microsecond and fold. :attr:`.tzinfo` is ``None``. See also method :meth:`timetz`. + .. versionchanged:: 3.6 + The fold value is copied to the returned :class:`.time` object. + .. method:: datetime.timetz() - Return :class:`.time` object with same hour, minute, second, microsecond, and + Return :class:`.time` object with same hour, minute, second, microsecond, fold, and tzinfo attributes. See also method :meth:`time`. - -.. method:: datetime.replace([year[, month[, day[, hour[, minute[, second[, microsecond[, tzinfo]]]]]]]]) + .. versionchanged:: 3.6 + The fold value is copied to the returned :class:`.time` object. + + +.. method:: datetime.replace(year=self.year, month=self.month, day=self.day, \ + hour=self.hour, minute=self.minute, second=self.second, microsecond=self.microsecond, \ + tzinfo=self.tzinfo, * fold=0) Return a datetime with the same attributes, except for those attributes given new values by whichever keyword arguments are specified. Note that ``tzinfo=None`` can be specified to create a naive datetime from an aware datetime with no conversion of date and time data. + .. versionadded:: 3.6 + Added the ``fold`` argument. + .. method:: datetime.astimezone(tz=None) @@ -999,23 +1026,20 @@ *self*, but in *tz*'s local time. If provided, *tz* must be an instance of a :class:`tzinfo` subclass, and its - :meth:`utcoffset` and :meth:`dst` methods must not return ``None``. *self* must - be aware (``self.tzinfo`` must not be ``None``, and ``self.utcoffset()`` must - not return ``None``). + :meth:`utcoffset` and :meth:`dst` methods must not return ``None``. If *self* + is naive (``self.tzinfo is None``), it is presumed to represent time in the + system timezone. If called without arguments (or with ``tz=None``) the system local - timezone is assumed. The ``.tzinfo`` attribute of the converted + timezone is assumed for the target timezone. The ``.tzinfo`` attribute of the converted datetime instance will be set to an instance of :class:`timezone` with the zone name and offset obtained from the OS. If ``self.tzinfo`` is *tz*, ``self.astimezone(tz)`` is equal to *self*: no adjustment of date or time data is performed. Else the result is local - time in time zone *tz*, representing the same UTC time as *self*: after - ``astz = dt.astimezone(tz)``, ``astz - astz.utcoffset()`` will usually have - the same date and time data as ``dt - dt.utcoffset()``. The discussion - of class :class:`tzinfo` explains the cases at Daylight Saving Time transition - boundaries where this cannot be achieved (an issue only if *tz* models both - standard and daylight time). + time in the timezone *tz*, representing the same UTC time as *self*: after + ``astz = dt.astimezone(tz)``, ``astz - astz.utcoffset()`` will have + the same date and time data as ``dt - dt.utcoffset()``. If you merely want to attach a time zone object *tz* to a datetime *dt* without adjustment of date and time data, use ``dt.replace(tzinfo=tz)``. If you @@ -1037,6 +1061,10 @@ .. versionchanged:: 3.3 *tz* now can be omitted. + .. versionchanged:: 3.6 + The :meth:`astimezone` method can now be called on naive instances that + are presumed to represent system local time. + .. method:: datetime.utcoffset() @@ -1113,6 +1141,10 @@ .. versionadded:: 3.3 + .. versionchanged:: 3.6 + The :meth:`timestamp` method uses the :attr:`.fold` attribute to + disambiguate the times during a repeated interval. + .. note:: There is no method to obtain the POSIX timestamp directly from a @@ -1342,16 +1374,17 @@ A time object represents a (local) time of day, independent of any particular day, and subject to adjustment via a :class:`tzinfo` object. -.. class:: time(hour=0, minute=0, second=0, microsecond=0, tzinfo=None) +.. class:: time(hour=0, minute=0, second=0, microsecond=0, tzinfo=None, *, fold=0) All arguments are optional. *tzinfo* may be ``None``, or an instance of a :class:`tzinfo` subclass. The remaining arguments may be integers, in the following ranges: - * ``0 <= hour < 24`` - * ``0 <= minute < 60`` - * ``0 <= second < 60`` - * ``0 <= microsecond < 1000000``. + * ``0 <= hour < 24``, + * ``0 <= minute < 60``, + * ``0 <= second < 60``, + * ``0 <= microsecond < 1000000``, + * ``fold in [0, 1]``. If an argument outside those ranges is given, :exc:`ValueError` is raised. All default to ``0`` except *tzinfo*, which defaults to :const:`None`. @@ -1404,6 +1437,17 @@ ``None`` if none was passed. +.. attribute:: time.fold + + In ``[0, 1]``. Used to disambiguate wall times during a repeated interval. (A + repeated interval occurs when clocks are rolled back at the end of daylight saving + time or when the UTC offset for the current zone is decreased for political reasons.) + The value 0 (1) represents the earlier (later) of the two moments with the same wall + time representation. + + .. versionadded:: 3.6 + + Supported operations: * comparison of :class:`.time` to :class:`.time`, where *a* is considered less @@ -1439,13 +1483,17 @@ Instance methods: -.. method:: time.replace([hour[, minute[, second[, microsecond[, tzinfo]]]]]) +.. method:: time.replace(hour=self.hour, minute=self.minute, second=self.second, \ + microsecond=self.microsecond, tzinfo=self.tzinfo, * fold=0) Return a :class:`.time` with the same value, except for those attributes given new values by whichever keyword arguments are specified. Note that ``tzinfo=None`` can be specified to create a naive :class:`.time` from an aware :class:`.time`, without conversion of the time data. + .. versionadded:: 3.6 + Added the ``fold`` argument. + .. method:: time.isoformat(timespec='auto') @@ -1754,9 +1802,19 @@ When DST starts (the "start" line), the local wall clock leaps from 1:59 to 3:00. A wall time of the form 2:MM doesn't really make sense on that day, so ``astimezone(Eastern)`` won't deliver a result with ``hour == 2`` on the day DST -begins. In order for :meth:`astimezone` to make this guarantee, the -:meth:`tzinfo.dst` method must consider times in the "missing hour" (2:MM for -Eastern) to be in daylight time. +begins. For example, at the Spring forward transition of 2016, we get + + >>> u0 = datetime(2016, 3, 13, 5, tzinfo=timezone.utc) + >>> for i in range(4): + ... u = u0 + i*HOUR + ... t = u.astimezone(Eastern) + ... print(u.time(), 'UTC =', t.time(), t.tzname()) + ... + 05:00:00 UTC = 00:00:00 EST + 06:00:00 UTC = 01:00:00 EST + 07:00:00 UTC = 03:00:00 EDT + 08:00:00 UTC = 04:00:00 EDT + When DST ends (the "end" line), there's a potentially worse problem: there's an hour that can't be spelled unambiguously in local wall time: the last hour of @@ -1765,28 +1823,41 @@ to 1:00 (standard time) again. Local times of the form 1:MM are ambiguous. :meth:`astimezone` mimics the local clock's behavior by mapping two adjacent UTC hours into the same local hour then. In the Eastern example, UTC times of the -form 5:MM and 6:MM both map to 1:MM when converted to Eastern. In order for -:meth:`astimezone` to make this guarantee, the :meth:`tzinfo.dst` method must -consider times in the "repeated hour" to be in standard time. This is easily -arranged, as in the example, by expressing DST switch times in the time zone's -standard local time. - -Applications that can't bear such ambiguities should avoid using hybrid +form 5:MM and 6:MM both map to 1:MM when converted to Eastern, but earlier times +have the :attr:`~datetime.fold` attribute set to 0 and the later times have it set to 1. +For example, at the Fall back transition of 2016, we get + + >>> u0 = datetime(2016, 11, 6, 4, tzinfo=timezone.utc) + >>> for i in range(4): + ... u = u0 + i*HOUR + ... t = u.astimezone(Eastern) + ... print(u.time(), 'UTC =', t.time(), t.tzname(), t.fold) + ... + 04:00:00 UTC = 00:00:00 EDT 0 + 05:00:00 UTC = 01:00:00 EDT 0 + 06:00:00 UTC = 01:00:00 EST 1 + 07:00:00 UTC = 02:00:00 EST 0 + +Note that the :class:`datetime` instances that differ only by the value of the +:attr:`~datetime.fold` attribute are considered equal in comparisons. + +Applications that can't bear wall-time ambiguities should explicitly check the +value of the :attr:`~datetime.fold` atribute or avoid using hybrid :class:`tzinfo` subclasses; there are no ambiguities when using :class:`timezone`, or any other fixed-offset :class:`tzinfo` subclass (such as a class representing only EST (fixed offset -5 hours), or only EDT (fixed offset -4 hours)). .. seealso:: - `pytz `_ + `datetuil.tz `_ The standard library has :class:`timezone` class for handling arbitrary fixed offsets from UTC and :attr:`timezone.utc` as UTC timezone instance. - *pytz* library brings the *IANA timezone database* (also known as the + *datetuil.tz* library brings the *IANA timezone database* (also known as the Olson database) to Python and its usage is recommended. `IANA timezone database `_ - The Time Zone Database (often called tz or zoneinfo) contains code and + The Time Zone Database (often called tz, tzdata or zoneinfo) contains code and data that represent the history of local time for many representative locations around the globe. It is updated periodically to reflect changes made by political bodies to time zone boundaries, UTC offsets, and @@ -1806,7 +1877,7 @@ made to civil time. -.. class:: timezone(offset[, name]) +.. class:: timezone(offset, name=None) The *offset* argument must be specified as a :class:`timedelta` object representing the difference between the local time and UTC. It must -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 18:41:15 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 24 Aug 2016 22:41:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=5FPyObject=5FFastCallDict?= =?utf-8?b?KCk6IGF2b2lkIF9QeV9DaGVja0Z1bmN0aW9uUmVzdWx0KCk=?= Message-ID: <20160824224115.20547.68974.65709897@psf.io> https://hg.python.org/cpython/rev/f4dcb5ff3ecd changeset: 102901:f4dcb5ff3ecd user: Victor Stinner date: Thu Aug 25 00:39:34 2016 +0200 summary: _PyObject_FastCallDict(): avoid _Py_CheckFunctionResult() _PyObject_FastCallDict() only requires _Py_CheckFunctionResult() for the slow-path. Other cases already check for the result. files: Objects/abstract.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2299,10 +2299,10 @@ result = (*call)(func, tuple, kwargs); Py_DECREF(tuple); + + result = _Py_CheckFunctionResult(func, result, NULL); } - result = _Py_CheckFunctionResult(func, result, NULL); - exit: Py_LeaveRecursiveCall(); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 18:41:15 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 24 Aug 2016 22:41:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_=5FPyObject=5FFastCall?= =?utf-8?q?Keywords=28=29?= Message-ID: <20160824224115.20475.13804.44C5E83F@psf.io> https://hg.python.org/cpython/rev/5ec1095612d6 changeset: 102900:5ec1095612d6 user: Victor Stinner date: Thu Aug 25 00:29:32 2016 +0200 summary: Add _PyObject_FastCallKeywords() Issue #27830: Similar to _PyObject_FastCallDict(), but keyword arguments are also passed in the same C array than positional arguments, rather than being passed as a Python dict. files: Include/abstract.h | 17 ++++++ Include/funcobject.h | 6 ++ Objects/abstract.c | 79 ++++++++++++++++++++++++++++++++ Python/ceval.c | 23 +++++--- 4 files changed, 116 insertions(+), 9 deletions(-) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -292,6 +292,23 @@ #define _PyObject_CallArg1(func, arg) \ _PyObject_FastCall((func), &(arg), 1) + /* Call the callable object func with the "fast call" calling convention: + args is a C array for positional arguments followed by (key, value) + pairs for keyword arguments. + + nargs is the number of positional parameters at the beginning of stack. + nkwargs is the number of (key, value) pairs at the end of stack. + + If nargs and nkwargs are equal to zero, stack can be NULL. + + Return the result on success. Raise an exception and return NULL on + error. */ + PyAPI_FUNC(PyObject *) _PyObject_FastCallKeywords( + PyObject *func, + PyObject **stack, + Py_ssize_t nargs, + Py_ssize_t nkwargs); + PyAPI_FUNC(PyObject *) _Py_CheckFunctionResult(PyObject *func, PyObject *result, const char *where); diff --git a/Include/funcobject.h b/Include/funcobject.h --- a/Include/funcobject.h +++ b/Include/funcobject.h @@ -64,6 +64,12 @@ PyObject **args, Py_ssize_t nargs, PyObject *kwargs); + +PyAPI_FUNC(PyObject *) _PyFunction_FastCallKeywords( + PyObject *func, + PyObject **stack, + Py_ssize_t nargs, + Py_ssize_t nkwargs); #endif /* Macros for direct access to these values. Type checks are *not* diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2309,6 +2309,85 @@ return result; } +static PyObject * +_PyStack_AsDict(PyObject **stack, Py_ssize_t nkwargs, PyObject *func) +{ + PyObject *kwdict; + + kwdict = PyDict_New(); + if (kwdict == NULL) { + return NULL; + } + + while (--nkwargs >= 0) { + int err; + PyObject *key = *stack++; + PyObject *value = *stack++; + if (PyDict_GetItem(kwdict, key) != NULL) { + PyErr_Format(PyExc_TypeError, + "%.200s%s got multiple values " + "for keyword argument '%U'", + PyEval_GetFuncName(func), + PyEval_GetFuncDesc(func), + key); + Py_DECREF(kwdict); + return NULL; + } + + err = PyDict_SetItem(kwdict, key, value); + if (err) { + Py_DECREF(kwdict); + return NULL; + } + } + return kwdict; +} + +PyObject * +_PyObject_FastCallKeywords(PyObject *func, PyObject **stack, Py_ssize_t nargs, + Py_ssize_t nkwargs) +{ + PyObject *args, *kwdict, *result; + + /* _PyObject_FastCallKeywords() must not be called with an exception set, + because it may clear it (directly or indirectly) and so the + caller loses its exception */ + assert(!PyErr_Occurred()); + + assert(func != NULL); + assert(nargs >= 0); + assert(nkwargs >= 0); + assert((nargs == 0 && nkwargs == 0) || stack != NULL); + + if (PyFunction_Check(func)) { + /* Fast-path: avoid temporary tuple or dict */ + return _PyFunction_FastCallKeywords(func, stack, nargs, nkwargs); + } + + if (PyCFunction_Check(func) && nkwargs == 0) { + return _PyCFunction_FastCallDict(func, args, nargs, NULL); + } + + /* Slow-path: build temporary tuple and/or dict */ + args = _PyStack_AsTuple(stack, nargs); + + if (nkwargs > 0) { + kwdict = _PyStack_AsDict(stack + nargs, nkwargs, func); + if (kwdict == NULL) { + Py_DECREF(args); + return NULL; + } + } + else { + kwdict = NULL; + } + + result = PyObject_Call(func, args, kwdict); + Py_DECREF(args); + Py_XDECREF(kwdict); + return result; +} + static PyObject* call_function_tail(PyObject *callable, PyObject *args) { diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -113,7 +113,6 @@ #else static PyObject * call_function(PyObject ***, int); #endif -static PyObject * fast_function(PyObject *, PyObject **, Py_ssize_t, Py_ssize_t); static PyObject * do_call(PyObject *, PyObject ***, Py_ssize_t, Py_ssize_t); static PyObject * ext_do_call(PyObject *, PyObject ***, int, Py_ssize_t, Py_ssize_t); static PyObject * update_keyword_args(PyObject *, Py_ssize_t, PyObject ***, @@ -4767,7 +4766,7 @@ } READ_TIMESTAMP(*pintr0); if (PyFunction_Check(func)) { - x = fast_function(func, (*pp_stack) - n, nargs, nkwargs); + x = _PyFunction_FastCallKeywords(func, (*pp_stack) - n, nargs, nkwargs); } else { x = do_call(func, pp_stack, nargs, nkwargs); @@ -4780,7 +4779,7 @@ /* Clear the stack of the function object. Also removes the arguments in case they weren't consumed already - (fast_function() and err_args() leave them on the stack). + (_PyFunction_FastCallKeywords() and err_args() leave them on the stack). */ while ((*pp_stack) > pfunc) { w = EXT_POP(*pp_stack); @@ -4792,7 +4791,7 @@ return x; } -/* The fast_function() function optimize calls for which no argument +/* The _PyFunction_FastCallKeywords() function optimize calls for which no argument tuple is necessary; the objects are passed directly from the stack. For the simplest case -- a function that takes only positional arguments and is called with only positional arguments -- it @@ -4840,8 +4839,9 @@ /* Similar to _PyFunction_FastCall() but keywords are passed a (key, value) pairs in stack */ -static PyObject * -fast_function(PyObject *func, PyObject **stack, Py_ssize_t nargs, Py_ssize_t nkwargs) +PyObject * +_PyFunction_FastCallKeywords(PyObject *func, PyObject **stack, + Py_ssize_t nargs, Py_ssize_t nkwargs) { PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func); PyObject *globals = PyFunction_GET_GLOBALS(func); @@ -4850,6 +4850,11 @@ PyObject **d; int nd; + assert(func != NULL); + assert(nargs >= 0); + assert(nkwargs >= 0); + assert((nargs == 0 && nkwargs == 0) || stack != NULL); + PCALL(PCALL_FUNCTION); PCALL(PCALL_FAST_FUNCTION); @@ -4902,14 +4907,14 @@ Py_ssize_t nd, nk; PyObject *result; - PCALL(PCALL_FUNCTION); - PCALL(PCALL_FAST_FUNCTION); - assert(func != NULL); assert(nargs >= 0); assert(nargs == 0 || args != NULL); assert(kwargs == NULL || PyDict_Check(kwargs)); + PCALL(PCALL_FUNCTION); + PCALL(PCALL_FAST_FUNCTION); + if (co->co_kwonlyargcount == 0 && (kwargs == NULL || PyDict_Size(kwargs) == 0) && co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 19:14:24 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 24 Aug 2016 23:14:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327830=3A_Fix_=5FP?= =?utf-8?q?yObject=5FFastCallKeywords=28=29?= Message-ID: <20160824231424.63704.53063.7A3236E0@psf.io> https://hg.python.org/cpython/rev/0ff25676505d changeset: 102903:0ff25676505d user: Victor Stinner date: Thu Aug 25 01:00:31 2016 +0200 summary: Issue #27830: Fix _PyObject_FastCallKeywords() Pass stack, not unrelated and uninitialized args! files: Objects/abstract.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2365,7 +2365,7 @@ } if (PyCFunction_Check(func) && nkwargs == 0) { - return _PyCFunction_FastCallDict(func, args, nargs, NULL); + return _PyCFunction_FastCallDict(func, stack, nargs, NULL); } /* Slow-path: build temporary tuple and/or dict */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 19:14:25 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 24 Aug 2016 23:14:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_method=5Fcall=28=29_and_sl?= =?utf-8?q?ot=5Ftp=5Fnew=28=29_now_uses_fast_call?= Message-ID: <20160824231424.24301.12399.61FBD4D2@psf.io> https://hg.python.org/cpython/rev/316e5de8a96b changeset: 102904:316e5de8a96b user: Victor Stinner date: Thu Aug 25 01:04:14 2016 +0200 summary: method_call() and slot_tp_new() now uses fast call Issue #27841: Add _PyObject_Call_Prepend() helper function to prepend an argument to existing arguments to call a function. This helper uses fast calls. Modify method_call() and slot_tp_new() to use _PyObject_Call_Prepend(). files: Include/abstract.h | 4 +++ Objects/abstract.c | 39 +++++++++++++++++++++++++++++++ Objects/classobject.c | 29 +++++----------------- Objects/typeobject.c | 25 ++++--------------- 4 files changed, 56 insertions(+), 41 deletions(-) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -309,6 +309,10 @@ Py_ssize_t nargs, Py_ssize_t nkwargs); + PyAPI_FUNC(PyObject *) _PyObject_Call_Prepend(PyObject *func, + PyObject *obj, PyObject *args, + PyObject *kwargs); + PyAPI_FUNC(PyObject *) _Py_CheckFunctionResult(PyObject *func, PyObject *result, const char *where); diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2388,6 +2388,45 @@ return result; } +/* Positional arguments are obj followed args. */ +PyObject * +_PyObject_Call_Prepend(PyObject *func, + PyObject *obj, PyObject *args, PyObject *kwargs) +{ + PyObject *small_stack[8]; + PyObject **stack; + Py_ssize_t argcount; + PyObject *result; + + assert(PyTuple_Check(args)); + + argcount = PyTuple_GET_SIZE(args); + if (argcount + 1 <= (Py_ssize_t)Py_ARRAY_LENGTH(small_stack)) { + stack = small_stack; + } + else { + stack = PyMem_Malloc((argcount + 1) * sizeof(PyObject *)); + if (stack == NULL) { + PyErr_NoMemory(); + return NULL; + } + } + + /* use borrowed references */ + stack[0] = obj; + Py_MEMCPY(&stack[1], + &PyTuple_GET_ITEM(args, 0), + argcount * sizeof(PyObject *)); + + result = _PyObject_FastCallDict(func, + stack, argcount + 1, + kwargs); + if (stack != small_stack) { + PyMem_Free(stack); + } + return result; +} + static PyObject* call_function_tail(PyObject *callable, PyObject *args) { diff --git a/Objects/classobject.c b/Objects/classobject.c --- a/Objects/classobject.c +++ b/Objects/classobject.c @@ -302,34 +302,19 @@ } static PyObject * -method_call(PyObject *func, PyObject *arg, PyObject *kw) +method_call(PyObject *method, PyObject *args, PyObject *kwargs) { - PyObject *self = PyMethod_GET_SELF(func); - PyObject *result; + PyObject *self, *func; - func = PyMethod_GET_FUNCTION(func); + self = PyMethod_GET_SELF(method); if (self == NULL) { PyErr_BadInternalCall(); return NULL; } - else { - Py_ssize_t argcount = PyTuple_Size(arg); - PyObject *newarg = PyTuple_New(argcount + 1); - int i; - if (newarg == NULL) - return NULL; - Py_INCREF(self); - PyTuple_SET_ITEM(newarg, 0, self); - for (i = 0; i < argcount; i++) { - PyObject *v = PyTuple_GET_ITEM(arg, i); - Py_XINCREF(v); - PyTuple_SET_ITEM(newarg, i+1, v); - } - arg = newarg; - } - result = PyObject_Call((PyObject *)func, arg, kw); - Py_DECREF(arg); - return result; + + func = PyMethod_GET_FUNCTION(method); + + return _PyObject_Call_Prepend(func, self, args, kwargs); } static PyObject * diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -6356,29 +6356,16 @@ static PyObject * slot_tp_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { - PyObject *func; - PyObject *newargs, *x; - Py_ssize_t i, n; + PyObject *func, *result; func = _PyObject_GetAttrId((PyObject *)type, &PyId___new__); - if (func == NULL) + if (func == NULL) { return NULL; - assert(PyTuple_Check(args)); - n = PyTuple_GET_SIZE(args); - newargs = PyTuple_New(n+1); - if (newargs == NULL) - return NULL; - Py_INCREF(type); - PyTuple_SET_ITEM(newargs, 0, (PyObject *)type); - for (i = 0; i < n; i++) { - x = PyTuple_GET_ITEM(args, i); - Py_INCREF(x); - PyTuple_SET_ITEM(newargs, i+1, x); - } - x = PyObject_Call(func, newargs, kwds); - Py_DECREF(newargs); + } + + result = _PyObject_Call_Prepend(func, (PyObject *)type, args, kwds); Py_DECREF(func); - return x; + return result; } static void -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 19:14:24 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 24 Aug 2016 23:14:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=5Fpickle=3A_remove_outdat?= =?utf-8?q?ed_comment?= Message-ID: <20160824231424.82561.89460.254A8B48@psf.io> https://hg.python.org/cpython/rev/f03af015cf2e changeset: 102902:f03af015cf2e user: Victor Stinner date: Thu Aug 25 00:58:58 2016 +0200 summary: _pickle: remove outdated comment _Pickle_FastCall() is now fast again! The optimization was introduced in Python 3.2, removed in Python 3.4 and reintroduced in Python 3.6 (thanks to the new generic fastcall functions). files: Modules/_pickle.c | 11 ----------- 1 files changed, 0 insertions(+), 11 deletions(-) diff --git a/Modules/_pickle.c b/Modules/_pickle.c --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -346,17 +346,6 @@ { PyObject *result; - /* Note: this function used to reuse the argument tuple. This used to give - a slight performance boost with older pickle implementations where many - unbuffered reads occurred (thus needing many function calls). - - However, this optimization was removed because it was too complicated - to get right. It abused the C API for tuples to mutate them which led - to subtle reference counting and concurrency bugs. Furthermore, the - introduction of protocol 4 and the prefetching optimization via peek() - significantly reduced the number of function calls we do. Thus, the - benefits became marginal at best. */ - result = _PyObject_CallArg1(func, obj); Py_DECREF(obj); return result; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 24 22:08:46 2016 From: python-checkins at python.org (terry.reedy) Date: Thu, 25 Aug 2016 02:08:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327821=3A_Fix_bug_?= =?utf-8?q?in_idlelib=2Ecomfig_function_and_add_new_tests=2E?= Message-ID: <20160825020846.27029.19528.664A5815@psf.io> https://hg.python.org/cpython/rev/f198457d1475 changeset: 102905:f198457d1475 user: Terry Jan Reedy date: Wed Aug 24 22:08:01 2016 -0400 summary: Issue #27821: Fix bug in idlelib.comfig function and add new tests. files: Lib/idlelib/config.py | 2 +- Lib/idlelib/idle_test/test_config.py | 106 +++++++++++--- 2 files changed, 85 insertions(+), 23 deletions(-) diff --git a/Lib/idlelib/config.py b/Lib/idlelib/config.py --- a/Lib/idlelib/config.py +++ b/Lib/idlelib/config.py @@ -394,7 +394,7 @@ 'name2' may still be set, but it is ignored. """ cfgname = 'highlight' if section == 'Theme' else 'keys' - default = self.GetOption('main', 'Theme', 'default', + default = self.GetOption('main', section, 'default', type='bool', default=True) name = '' if default: diff --git a/Lib/idlelib/idle_test/test_config.py b/Lib/idlelib/idle_test/test_config.py --- a/Lib/idlelib/idle_test/test_config.py +++ b/Lib/idlelib/idle_test/test_config.py @@ -28,53 +28,115 @@ class CurrentColorKeysTest(unittest.TestCase): - """Test correct scenarios for colorkeys and wrap functions. + """ Test colorkeys function with user config [Theme] and [Keys] patterns. - The 5 correct patterns are possible results of config dialog. + colorkeys = config.IdleConf.current_colors_and_keys + Test all patterns written by IDLE and some errors + Item 'default' should really be 'builtin' (versus 'custom). """ colorkeys = idleConf.current_colors_and_keys + default_theme = 'IDLE Classic' + default_keys = idleConf.default_keys() - def test_old_default(self): - # name2 must be blank + def test_old_builtin_theme(self): + # On initial installation, user main is blank. + self.assertEqual(self.colorkeys('Theme'), self.default_theme) + # For old default, name2 must be blank. usermain.read_string(''' [Theme] - default= 1 + default = True ''') - self.assertEqual(self.colorkeys('Theme'), 'IDLE Classic') + # IDLE omits 'name' for default old builtin theme. + self.assertEqual(self.colorkeys('Theme'), self.default_theme) + # IDLE adds 'name' for non-default old builtin theme. usermain['Theme']['name'] = 'IDLE New' self.assertEqual(self.colorkeys('Theme'), 'IDLE New') - usermain['Theme']['name'] = 'non-default' # error - self.assertEqual(self.colorkeys('Theme'), 'IDLE Classic') + # Erroneous non-default old builtin reverts to default. + usermain['Theme']['name'] = 'non-existent' + self.assertEqual(self.colorkeys('Theme'), self.default_theme) usermain.remove_section('Theme') - def test_new_default(self): - # name2 overrides name + def test_new_builtin_theme(self): + # IDLE writes name2 for new builtins. usermain.read_string(''' [Theme] - default= 1 - name= IDLE New - name2= IDLE Dark + default = True + name2 = IDLE Dark ''') self.assertEqual(self.colorkeys('Theme'), 'IDLE Dark') - usermain['Theme']['name2'] = 'non-default' # error - self.assertEqual(self.colorkeys('Theme'), 'IDLE Classic') + # Leftover 'name', not removed, is ignored. + usermain['Theme']['name'] = 'IDLE New' + self.assertEqual(self.colorkeys('Theme'), 'IDLE Dark') + # Erroneous non-default new builtin reverts to default. + usermain['Theme']['name2'] = 'non-existent' + self.assertEqual(self.colorkeys('Theme'), self.default_theme) usermain.remove_section('Theme') - def test_user_override(self): - # name2 does not matter + def test_user_override_theme(self): + # Erroneous custom name (no definition) reverts to default. usermain.read_string(''' [Theme] - default= 0 - name= Custom Dark - ''') # error until set userhigh - self.assertEqual(self.colorkeys('Theme'), 'IDLE Classic') + default = False + name = Custom Dark + ''') + self.assertEqual(self.colorkeys('Theme'), self.default_theme) + # Custom name is valid with matching Section name. userhigh.read_string('[Custom Dark]\na=b') self.assertEqual(self.colorkeys('Theme'), 'Custom Dark') - usermain['Theme']['name2'] = 'IDLE Dark' + # Name2 is ignored. + usermain['Theme']['name2'] = 'non-existent' self.assertEqual(self.colorkeys('Theme'), 'Custom Dark') usermain.remove_section('Theme') userhigh.remove_section('Custom Dark') + def test_old_builtin_keys(self): + # On initial installation, user main is blank. + self.assertEqual(self.colorkeys('Keys'), self.default_keys) + # For old default, name2 must be blank, name is always used. + usermain.read_string(''' + [Keys] + default = True + name = IDLE Classic Unix + ''') + self.assertEqual(self.colorkeys('Keys'), 'IDLE Classic Unix') + # Erroneous non-default old builtin reverts to default. + usermain['Keys']['name'] = 'non-existent' + self.assertEqual(self.colorkeys('Keys'), self.default_keys) + usermain.remove_section('Keys') + + def test_new_builtin_keys(self): + # IDLE writes name2 for new builtins. + usermain.read_string(''' + [Keys] + default = True + name2 = IDLE Modern Unix + ''') + self.assertEqual(self.colorkeys('Keys'), 'IDLE Modern Unix') + # Leftover 'name', not removed, is ignored. + usermain['Keys']['name'] = 'IDLE Classic Unix' + self.assertEqual(self.colorkeys('Keys'), 'IDLE Modern Unix') + # Erroneous non-default new builtin reverts to default. + usermain['Keys']['name2'] = 'non-existent' + self.assertEqual(self.colorkeys('Keys'), self.default_keys) + usermain.remove_section('Keys') + + def test_user_override_keys(self): + # Erroneous custom name (no definition) reverts to default. + usermain.read_string(''' + [Keys] + default = False + name = Custom Keys + ''') + self.assertEqual(self.colorkeys('Keys'), self.default_keys) + # Custom name is valid with matching Section name. + userkeys.read_string('[Custom Keys]\na=b') + self.assertEqual(self.colorkeys('Keys'), 'Custom Keys') + # Name2 is ignored. + usermain['Keys']['name2'] = 'non-existent' + self.assertEqual(self.colorkeys('Keys'), 'Custom Keys') + usermain.remove_section('Keys') + userkeys.remove_section('Custom Keys') + class WarningTest(unittest.TestCase): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Aug 25 01:23:18 2016 From: python-checkins at python.org (terry.reedy) Date: Thu, 25 Aug 2016 05:23:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1NTY0?= =?utf-8?q?=3A_Mention_exec_and_=5F=5Fbuiltins=5F=5F_in_IDLE-console_diffe?= =?utf-8?q?rence_section=2E?= Message-ID: <20160825052318.113360.48577.7F22A37B@psf.io> https://hg.python.org/cpython/rev/641852513b8e changeset: 102907:641852513b8e branch: 3.5 parent: 102895:37335860e0f7 user: Terry Jan Reedy date: Thu Aug 25 01:22:30 2016 -0400 summary: Issue #25564: Mention exec and __builtins__ in IDLE-console difference section. files: Doc/library/idle.rst | 6 ++- Lib/idlelib/help.html | 43 +++++++++++++++--------------- 2 files changed, 25 insertions(+), 24 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -562,11 +562,13 @@ When this window has the focus, it controls the keyboard and screen. This is normally transparent, but functions that directly access the keyboard and screen will not work. If ``sys`` is reset with ``importlib.reload(sys)``, -IDLE's changes are lost and things li ke ``input``, ``raw_input``, and +IDLE's changes are lost and things like ``input``, ``raw_input``, and ``print`` will not work correctly. With IDLE's Shell, one enters, edits, and recalls complete statements. -Some consoles only work with a single physical line at a time. +Some consoles only work with a single physical line at a time. IDLE uses +``exec`` to run each statement. As a result, ``'__builtins__'`` is always +defined for each statement. Running without a subprocess ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -6,7 +6,7 @@ - 25.5. IDLE — Python 3.5.1 documentation + 25.5. IDLE — Python 3.5.2 documentation @@ -14,7 +14,7 @@ - + @@ -58,13 +58,13 @@ accesskey="P">previous |
    • -
  • Python »
    • +
  • Python »
  • - 3.5.1 Documentation » + 3.5.2 Documentation »
    • -
  • The Python Standard Library »
    • -
  • 25. Graphical User Interfaces with Tk »
    • +
  • The Python Standard Library »
    • +
  • 25. Graphical User Interfaces with Tk »
    • @@ -492,7 +492,7 @@ functions to be used from IDLE’s Python shell.

    25.5.3.1. Command line usage?

    -
    idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ...
+
    idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ...
 
 -c command  run command in the shell window
 -d          enable debugger and open shell window
@@ -526,10 +526,12 @@
 When this window has the focus, it controls the keyboard and screen.
 This is normally transparent, but functions that directly access the keyboard
 and screen will not work.  If sys is reset with importlib.reload(sys),
-IDLE’s changes are lost and things li ke input, raw_input, and
+IDLE’s changes are lost and things like input, raw_input, and
 print will not work correctly.
    
 
    With IDLE’s Shell, one enters, edits, and recalls complete statements.
-Some consoles only work with a single physical line at a time.
    
+Some consoles only work with a single physical line at a time.  IDLE uses
+exec to run each statement.  As a result, '__builtins__' is always
+defined for each statement.
    
 
    
 
    
 
    25.5.3.3. Running without a subprocess?
    
@@ -657,14 +659,11 @@
 
 
         
    
@@ -688,13 +687,13 @@
              >previous |
         
    • 
-        
  • Python »
    • 
+        
  • Python »
    • 
         
  • 
-          3.5.1 Documentation »
+          3.5.2 Documentation »
         
    • 
 
-          
  • The Python Standard Library »
    • 
-          
  • 25. Graphical User Interfaces with Tk »
    • 
+          
  • The Python Standard Library »
    • 
+          
  • 25. Graphical User Interfaces with Tk »
    • 
       
     
    
     
    
@@ -703,10 +702,10 @@
     The Python Software Foundation is a non-profit corporation.
     Please donate.
     
    
-    Last updated on Jun 11, 2016.
+    Last updated on Aug 25, 2016.
     Found a bug?
     
    
-    Created using Sphinx 1.3.3.
+    Created using Sphinx 1.4.6.
     
    
 
   

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Aug 25 01:23:20 2016
From: python-checkins at python.org (terry.reedy)
Date: Thu, 25 Aug 2016 05:23:20 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_with_3=2E5?=
Message-ID: <20160825052318.17847.55687.36F766D1@psf.io>

https://hg.python.org/cpython/rev/48f8148428d7
changeset:   102908:48f8148428d7
parent:      102905:f198457d1475
parent:      102907:641852513b8e
user:        Terry Jan Reedy 
date:        Thu Aug 25 01:22:44 2016 -0400
summary:
  Merge with 3.5

files:
  Doc/library/idle.rst  |   6 ++-
  Lib/idlelib/help.html |  43 +++++++++++++++---------------
  2 files changed, 25 insertions(+), 24 deletions(-)


diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst
--- a/Doc/library/idle.rst
+++ b/Doc/library/idle.rst
@@ -562,11 +562,13 @@
 When this window has the focus, it controls the keyboard and screen.
 This is normally transparent, but functions that directly access the keyboard
 and screen will not work.  If ``sys`` is reset with ``importlib.reload(sys)``,
-IDLE's changes are lost and things li ke ``input``, ``raw_input``, and
+IDLE's changes are lost and things like ``input``, ``raw_input``, and
 ``print`` will not work correctly.
 
 With IDLE's Shell, one enters, edits, and recalls complete statements.
-Some consoles only work with a single physical line at a time.
+Some consoles only work with a single physical line at a time.  IDLE uses
+``exec`` to run each statement.  As a result, ``'__builtins__'`` is always
+defined for each statement.
 
 Running without a subprocess
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html
--- a/Lib/idlelib/help.html
+++ b/Lib/idlelib/help.html
@@ -6,7 +6,7 @@
   
     
 
-    25.5. IDLE — Python 3.5.1 documentation
+    25.5. IDLE — Python 3.5.2 documentation
 
     
     
@@ -14,7 +14,7 @@
     
     
     
     
     
-    
+    
     
     
     
@@ -58,13 +58,13 @@
              accesskey="P">previous |
         
    • 
-        
  • Python »
    • 
+        
  • Python »
    • 
         
  • 
-          3.5.1 Documentation »
+          3.5.2 Documentation »
         
    • 
 
-          
  • The Python Standard Library »
    • 
-          
  • 25. Graphical User Interfaces with Tk »
    • 
+          
  • The Python Standard Library »
    • 
+          
  • 25. Graphical User Interfaces with Tk »
    •
    @@ -492,7 +492,7 @@ functions to be used from IDLE’s Python shell.

    25.5.3.1. Command line usage?

    -
    idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ...
+
    idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ...
 
 -c command  run command in the shell window
 -d          enable debugger and open shell window
@@ -526,10 +526,12 @@
 When this window has the focus, it controls the keyboard and screen.
 This is normally transparent, but functions that directly access the keyboard
 and screen will not work.  If sys is reset with importlib.reload(sys),
-IDLE’s changes are lost and things li ke input, raw_input, and
+IDLE’s changes are lost and things like input, raw_input, and
 print will not work correctly.
    
 
    With IDLE’s Shell, one enters, edits, and recalls complete statements.
-Some consoles only work with a single physical line at a time.
    
+Some consoles only work with a single physical line at a time.  IDLE uses
+exec to run each statement.  As a result, '__builtins__' is always
+defined for each statement.
    
 
    
 
    
 
    25.5.3.3. Running without a subprocess?
    
@@ -657,14 +659,11 @@
 
 
         
    
@@ -688,13 +687,13 @@
              >previous |
         
    • 
-        
  • Python »
    • 
+        
  • Python »
    • 
         
  • 
-          3.5.1 Documentation »
+          3.5.2 Documentation »
         
    • 
 
-          
  • The Python Standard Library »
    • 
-          
  • 25. Graphical User Interfaces with Tk »
    • 
+          
  • The Python Standard Library »
    • 
+          
  • 25. Graphical User Interfaces with Tk »
    • 
       
     
    
     
    
@@ -703,10 +702,10 @@
     The Python Software Foundation is a non-profit corporation.
     Please donate.
     
    
-    Last updated on Jun 11, 2016.
+    Last updated on Aug 25, 2016.
     Found a bug?
     
    
-    Created using Sphinx 1.3.3.
+    Created using Sphinx 1.4.6.
     
    
 
   

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Aug 25 01:23:18 2016
From: python-checkins at python.org (terry.reedy)
Date: Thu, 25 Aug 2016 05:23:18 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1NTY0?=
 =?utf-8?q?=3A_Mention_exec_and_=5F=5Fbuiltins=5F=5F_in_IDLE-console_diffe?=
 =?utf-8?q?rence_section=2E?=
Message-ID: <20160825052318.113453.94968.AD0B0500@psf.io>

https://hg.python.org/cpython/rev/862761e4376e
changeset:   102906:862761e4376e
branch:      2.7
parent:      102889:ef13efb89afe
user:        Terry Jan Reedy 
date:        Thu Aug 25 01:21:54 2016 -0400
summary:
  Issue #25564: Mention exec and __builtins__ in IDLE-console difference section.
Do not print charrefs outside of the text proper (like '????? ').

files:
  Doc/library/idle.rst  |   4 ++-
  Lib/idlelib/help.html |  41 +++++++++++++++---------------
  Lib/idlelib/help.py   |   3 +-
  3 files changed, 25 insertions(+), 23 deletions(-)


diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst
--- a/Doc/library/idle.rst
+++ b/Doc/library/idle.rst
@@ -562,7 +562,9 @@
 ``print`` will not work correctly.
 
 With IDLE's Shell, one enters, edits, and recalls complete statements.
-Some consoles only work with a single physical line at a time.
+Some consoles only work with a single physical line at a time.  IDLE uses
+``exec`` to run each statement.  As a result, ``'__builtins__'`` is always
+defined for each statement.
 
 Running without a subprocess
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html
--- a/Lib/idlelib/help.html
+++ b/Lib/idlelib/help.html
@@ -6,7 +6,7 @@
   
     
 
-    24.6. IDLE — Python 2.7.11 documentation
+    24.6. IDLE — Python 2.7.12 documentation
 
     
     
@@ -14,7 +14,7 @@
     
     
     
     
     
-    
+    
     
     
     
@@ -58,13 +58,13 @@
              accesskey="P">previous |
         
    • 
-        
  • Python »
    • 
+        
  • Python »
    • 
         
  • 
-          Python 2.7.11 documentation »
+          Python 2.7.12 documentation »
         
    • 
 
-          
  • The Python Standard Library »
    • 
-          
  • 24. Graphical User Interfaces with Tk »
    • 
+          
  • The Python Standard Library »
    • 
+          
  • 24. Graphical User Interfaces with Tk »
    •
    @@ -490,7 +490,7 @@ functions to be used from IDLE’s Python shell.

    24.6.3.1. Command line usage?

    -
    idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ...
+
    idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ...
 
 -c command  run command in the shell window
 -d          enable debugger and open shell window
@@ -527,7 +527,9 @@
 IDLE’s changes are lost and things like input, raw_input, and
 print will not work correctly.
    
 
    With IDLE’s Shell, one enters, edits, and recalls complete statements.
-Some consoles only work with a single physical line at a time.
    
+Some consoles only work with a single physical line at a time.  IDLE uses
+exec to run each statement.  As a result, '__builtins__' is always
+defined for each statement.
    
 
    
 
    
 
    24.6.3.3. Running without a subprocess?
    
@@ -655,14 +657,11 @@
 
 
         
    
@@ -686,13 +685,13 @@
              >previous |
         
    • 
-        
  • Python »
    • 
+        
  • Python »
    • 
         
  • 
-          Python 2.7.11 documentation »
+          Python 2.7.12 documentation »
         
    • 
 
-          
  • The Python Standard Library »
    • 
-          
  • 24. Graphical User Interfaces with Tk »
    • 
+          
  • The Python Standard Library »
    • 
+          
  • 24. Graphical User Interfaces with Tk »
    • 
       
     
    
     
    
@@ -701,10 +700,10 @@
     The Python Software Foundation is a non-profit corporation.
     Please donate.
     
    
-    Last updated on May 02, 2016.
+    Last updated on Aug 25, 2016.
     Found a bug?
     
    
-    Created using Sphinx 1.3.3.
+    Created using Sphinx 1.4.6.
     
    
 
   
diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py
--- a/Lib/idlelib/help.py
+++ b/Lib/idlelib/help.py
@@ -150,7 +150,8 @@
             self.text.insert('end', d, (self.tags, self.chartags))
 
     def handle_charref(self, name):
-        self.text.insert('end', unichr(int(name)))
+        if self.show:
+            self.text.insert('end', unichr(int(name)))
 
 
 class HelpText(Text):

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Thu Aug 25 08:47:50 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Thu, 25 Aug 2016 13:47:50 +0100
Subject: [Python-checkins] GOOD Benchmark Results for Python Default
 2016-08-25
Message-ID: 

Results for project Python default, build date 2016-08-25 02:00:13 +0000
commit:		316e5de8a96b
previous commit:	ed5f5d490490
revision date:	2016-08-24 23:04:14 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.15%          0.76%         10.89%            15.69%
:-)             pybench      0.77%         -0.02%          5.95%             7.88%
:-)            regex_v8      2.76%          2.65%         -1.94%             7.19%
:-|               nbody      0.12%          0.30%          0.94%             6.89%
:-|        json_dump_v2      0.29%          1.53%         -1.80%            12.02%
:-|      normal_startup      0.45%         -0.35%         -0.04%             6.56%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-08-25/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Thu Aug 25 08:45:42 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Thu, 25 Aug 2016 13:45:42 +0100
Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7
 2016-08-25
Message-ID: 

Results for project Python 2.7, build date 2016-08-25 09:09:54 +0000
commit:		862761e4376e
previous commit:	75111791110b
revision date:	2016-08-25 05:21:54 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.14%          0.44%          5.34%             4.34%
:-)             pybench      0.21%          0.03%          6.14%             3.99%
:-(            regex_v8      0.60%         -0.08%         -2.05%            10.28%
:-)               nbody      0.09%          0.00%          7.73%             2.49%
:-)        json_dump_v2      0.29%         -0.60%          2.99%            11.43%
:-|      normal_startup      1.38%          0.55%         -0.08%             1.40%
:-)             ssbench      0.18%          0.30%          2.70%             0.65%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-08-25/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Thu Aug 25 10:14:34 2016
From: python-checkins at python.org (vinay.sajip)
Date: Thu, 25 Aug 2016 14:14:34 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIzNzQ2?=
 =?utf-8?q?=3A_Clarified_is=5Fpython=5Fbuild=28=29_documentation=2E?=
Message-ID: <20160825141416.18273.4938.1E561861@psf.io>

https://hg.python.org/cpython/rev/bf51f1f3569a
changeset:   102909:bf51f1f3569a
branch:      3.5
parent:      102907:641852513b8e
user:        Vinay Sajip 
date:        Thu Aug 25 15:13:24 2016 +0100
summary:
  Issue #23746: Clarified is_python_build() documentation.

files:
  Doc/library/sysconfig.rst |  4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Doc/library/sysconfig.rst b/Doc/library/sysconfig.rst
--- a/Doc/library/sysconfig.rst
+++ b/Doc/library/sysconfig.rst
@@ -204,7 +204,9 @@
 
 .. function:: is_python_build()
 
-   Return ``True`` if the current Python installation was built from source.
+   Return ``True`` if the running Python interpreter was built from source and
+   is being run from its built location, and not from a location resulting from
+   e.g. running ``make install`` or installing via a binary installer.
 
 
 .. function:: parse_config_h(fp[, vars])

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Aug 25 10:14:34 2016
From: python-checkins at python.org (vinay.sajip)
Date: Thu, 25 Aug 2016 14:14:34 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
 =?utf-8?q?=29=3A_Closes_=2323746=3A_merged_update_from_3=2E5=2E?=
Message-ID: <20160825141416.24096.58909.7CBA1E49@psf.io>

https://hg.python.org/cpython/rev/a0a7c5b1af97
changeset:   102910:a0a7c5b1af97
parent:      102908:48f8148428d7
parent:      102909:bf51f1f3569a
user:        Vinay Sajip 
date:        Thu Aug 25 15:14:07 2016 +0100
summary:
  Closes #23746: merged update from 3.5.

files:
  Doc/library/sysconfig.rst |  4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Doc/library/sysconfig.rst b/Doc/library/sysconfig.rst
--- a/Doc/library/sysconfig.rst
+++ b/Doc/library/sysconfig.rst
@@ -204,7 +204,9 @@
 
 .. function:: is_python_build()
 
-   Return ``True`` if the current Python installation was built from source.
+   Return ``True`` if the running Python interpreter was built from source and
+   is being run from its built location, and not from a location resulting from
+   e.g. running ``make install`` or installing via a binary installer.
 
 
 .. function:: parse_config_h(fp[, vars])

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Aug 25 17:30:39 2016
From: python-checkins at python.org (victor.stinner)
Date: Thu, 25 Aug 2016 21:30:39 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327830=3A_Revert?=
 =?utf-8?q?=2C_remove_=5FPyFunction=5FFastCallKeywords=28=29?=
Message-ID: <20160825213038.113360.33190.6C8F4C7A@psf.io>

https://hg.python.org/cpython/rev/ffcfa4f005a3
changeset:   102911:ffcfa4f005a3
user:        Victor Stinner 
date:        Thu Aug 25 23:26:50 2016 +0200
summary:
  Issue #27830: Revert, remove _PyFunction_FastCallKeywords()

files:
  Include/abstract.h   |  17 ------------
  Include/funcobject.h |   6 ----
  Objects/abstract.c   |  45 --------------------------------
  Python/ceval.c       |  12 ++++----
  4 files changed, 6 insertions(+), 74 deletions(-)


diff --git a/Include/abstract.h b/Include/abstract.h
--- a/Include/abstract.h
+++ b/Include/abstract.h
@@ -292,23 +292,6 @@
 #define _PyObject_CallArg1(func, arg) \
     _PyObject_FastCall((func), &(arg), 1)
 
-     /* Call the callable object func with the "fast call" calling convention:
-        args is a C array for positional arguments followed by (key, value)
-        pairs for keyword arguments.
-
-        nargs is the number of positional parameters at the beginning of stack.
-        nkwargs is the number of (key, value) pairs at the end of stack.
-
-        If nargs and nkwargs are equal to zero, stack can be NULL.
-
-        Return the result on success. Raise an exception and return NULL on
-        error. */
-     PyAPI_FUNC(PyObject *) _PyObject_FastCallKeywords(
-                                PyObject *func,
-                                PyObject **stack,
-                                Py_ssize_t nargs,
-                                Py_ssize_t nkwargs);
-
     PyAPI_FUNC(PyObject *) _PyObject_Call_Prepend(PyObject *func,
                                                   PyObject *obj, PyObject *args,
                                                   PyObject *kwargs);
diff --git a/Include/funcobject.h b/Include/funcobject.h
--- a/Include/funcobject.h
+++ b/Include/funcobject.h
@@ -64,12 +64,6 @@
     PyObject **args,
     Py_ssize_t nargs,
     PyObject *kwargs);
-
-PyAPI_FUNC(PyObject *) _PyFunction_FastCallKeywords(
-    PyObject *func,
-    PyObject **stack,
-    Py_ssize_t nargs,
-    Py_ssize_t nkwargs);
 #endif
 
 /* Macros for direct access to these values. Type checks are *not*
diff --git a/Objects/abstract.c b/Objects/abstract.c
--- a/Objects/abstract.c
+++ b/Objects/abstract.c
@@ -2343,51 +2343,6 @@
     return kwdict;
 }
 
-PyObject *
-_PyObject_FastCallKeywords(PyObject *func, PyObject **stack, Py_ssize_t nargs,
-                           Py_ssize_t nkwargs)
-{
-    PyObject *args, *kwdict, *result;
-
-    /* _PyObject_FastCallKeywords() must not be called with an exception set,
-       because it may clear it (directly or indirectly) and so the
-       caller loses its exception */
-    assert(!PyErr_Occurred());
-
-    assert(func != NULL);
-    assert(nargs >= 0);
-    assert(nkwargs >= 0);
-    assert((nargs == 0 && nkwargs == 0) || stack != NULL);
-
-    if (PyFunction_Check(func)) {
-        /* Fast-path: avoid temporary tuple or dict */
-        return _PyFunction_FastCallKeywords(func, stack, nargs, nkwargs);
-    }
-
-    if (PyCFunction_Check(func) && nkwargs == 0) {
-        return _PyCFunction_FastCallDict(func, stack, nargs, NULL);
-    }
-
-    /* Slow-path: build temporary tuple and/or dict */
-    args = _PyStack_AsTuple(stack, nargs);
-
-    if (nkwargs > 0) {
-        kwdict = _PyStack_AsDict(stack + nargs, nkwargs, func);
-        if (kwdict == NULL) {
-            Py_DECREF(args);
-            return NULL;
-        }
-    }
-    else {
-        kwdict = NULL;
-    }
-
-    result = PyObject_Call(func, args, kwdict);
-    Py_DECREF(args);
-    Py_XDECREF(kwdict);
-    return result;
-}
-
 /* Positional arguments are obj followed args. */
 PyObject *
 _PyObject_Call_Prepend(PyObject *func,
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -113,6 +113,7 @@
 #else
 static PyObject * call_function(PyObject ***, int);
 #endif
+static PyObject * fast_function(PyObject *, PyObject **, Py_ssize_t, Py_ssize_t);
 static PyObject * do_call(PyObject *, PyObject ***, Py_ssize_t, Py_ssize_t);
 static PyObject * ext_do_call(PyObject *, PyObject ***, int, Py_ssize_t, Py_ssize_t);
 static PyObject * update_keyword_args(PyObject *, Py_ssize_t, PyObject ***,
@@ -4766,7 +4767,7 @@
         }
         READ_TIMESTAMP(*pintr0);
         if (PyFunction_Check(func)) {
-            x = _PyFunction_FastCallKeywords(func, (*pp_stack) - n, nargs, nkwargs);
+            x = fast_function(func, (*pp_stack) - n, nargs, nkwargs);
         }
         else {
             x = do_call(func, pp_stack, nargs, nkwargs);
@@ -4779,7 +4780,7 @@
 
     /* Clear the stack of the function object.  Also removes
        the arguments in case they weren't consumed already
-       (_PyFunction_FastCallKeywords() and err_args() leave them on the stack).
+       (fast_function() and err_args() leave them on the stack).
      */
     while ((*pp_stack) > pfunc) {
         w = EXT_POP(*pp_stack);
@@ -4791,7 +4792,7 @@
     return x;
 }
 
-/* The _PyFunction_FastCallKeywords() function optimize calls for which no argument
+/* The fast_function() function optimize calls for which no argument
    tuple is necessary; the objects are passed directly from the stack.
    For the simplest case -- a function that takes only positional
    arguments and is called with only positional arguments -- it
@@ -4839,9 +4840,8 @@
 
 /* Similar to _PyFunction_FastCall() but keywords are passed a (key, value)
    pairs in stack */
-PyObject *
-_PyFunction_FastCallKeywords(PyObject *func, PyObject **stack,
-                             Py_ssize_t nargs, Py_ssize_t nkwargs)
+static PyObject *
+fast_function(PyObject *func, PyObject **stack, Py_ssize_t nargs, Py_ssize_t nkwargs)
 {
     PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func);
     PyObject *globals = PyFunction_GET_GLOBALS(func);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Aug 25 20:05:46 2016
From: python-checkins at python.org (terry.reedy)
Date: Fri, 26 Aug 2016 00:05:46 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Add_version_to?=
 =?utf-8?q?_title_of_IDLE_help_window=2E?=
Message-ID: <20160826000546.82688.70279.17C7196C@psf.io>

https://hg.python.org/cpython/rev/c75fcd094dcf
changeset:   102913:c75fcd094dcf
branch:      3.5
parent:      102909:bf51f1f3569a
user:        Terry Jan Reedy 
date:        Thu Aug 25 20:04:14 2016 -0400
summary:
  Add version to title of IDLE help window.

files:
  Lib/idlelib/help.py |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py
--- a/Lib/idlelib/help.py
+++ b/Lib/idlelib/help.py
@@ -26,6 +26,7 @@
 """
 from html.parser import HTMLParser
 from os.path import abspath, dirname, isdir, isfile, join
+from platform import python_version
 from tkinter import Tk, Toplevel, Frame, Text, Scrollbar, Menu, Menubutton
 from tkinter import font as tkfont
 from idlelib.configHandler import idleConf
@@ -265,7 +266,7 @@
     if not isfile(filename):
         # try copy_strip, present message
         return
-    HelpWindow(parent, filename, 'IDLE Help')
+    HelpWindow(parent, filename, 'IDLE Help (%s)' % python_version())
 
 if __name__ == '__main__':
     from idlelib.idle_test.htest import run

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Aug 25 20:05:46 2016
From: python-checkins at python.org (terry.reedy)
Date: Fri, 26 Aug 2016 00:05:46 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Add_version_to?=
 =?utf-8?q?_title_of_IDLE_help_window=2E?=
Message-ID: <20160826000545.18218.20293.6C7B995F@psf.io>

https://hg.python.org/cpython/rev/24e580e667fb
changeset:   102912:24e580e667fb
branch:      2.7
parent:      102906:862761e4376e
user:        Terry Jan Reedy 
date:        Thu Aug 25 20:04:08 2016 -0400
summary:
  Add version to title of IDLE help window.

files:
  Lib/idlelib/help.py |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py
--- a/Lib/idlelib/help.py
+++ b/Lib/idlelib/help.py
@@ -26,6 +26,7 @@
 """
 from HTMLParser import HTMLParser
 from os.path import abspath, dirname, isdir, isfile, join
+from platform import python_version
 from Tkinter import Tk, Toplevel, Frame, Text, Scrollbar, Menu, Menubutton
 import tkFont as tkfont
 from idlelib.configHandler import idleConf
@@ -269,7 +270,7 @@
     if not isfile(filename):
         # try copy_strip, present message
         return
-    HelpWindow(parent, filename, 'IDLE Help')
+    HelpWindow(parent, filename, 'IDLE Help (%s)' % python_version())
 
 if __name__ == '__main__':
     from idlelib.idle_test.htest import run

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Aug 25 20:05:46 2016
From: python-checkins at python.org (terry.reedy)
Date: Fri, 26 Aug 2016 00:05:46 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
 =?utf-8?q?=29=3A_Add_version_to_title_of_IDLE_help_window=2E?=
Message-ID: <20160826000546.68019.98924.5A435881@psf.io>

https://hg.python.org/cpython/rev/10a73648c92c
changeset:   102914:10a73648c92c
parent:      102911:ffcfa4f005a3
parent:      102913:c75fcd094dcf
user:        Terry Jan Reedy 
date:        Thu Aug 25 20:05:30 2016 -0400
summary:
  Add version to title of IDLE help window.

files:
  Lib/idlelib/help.py |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py
--- a/Lib/idlelib/help.py
+++ b/Lib/idlelib/help.py
@@ -26,6 +26,7 @@
 """
 from html.parser import HTMLParser
 from os.path import abspath, dirname, isfile, join
+from platform import python_version
 from tkinter import Toplevel, Frame, Text, Menu
 from tkinter.ttk import Menubutton, Scrollbar
 from tkinter import font as tkfont
@@ -265,7 +266,7 @@
     if not isfile(filename):
         # try copy_strip, present message
         return
-    return HelpWindow(parent, filename, 'IDLE Help')
+    HelpWindow(parent, filename, 'IDLE Help (%s)' % python_version())
 
 if __name__ == '__main__':
     from idlelib.idle_test.htest import run

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Aug 26 00:12:22 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Fri, 26 Aug 2016 04:12:22 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgMTk1MDQ6?=
 =?utf-8?q?__Change_=22customise=22_to_=22customize=22_American_spelling?=
 =?utf-8?q?=2E?=
Message-ID: <20160826041221.27150.84479.5F9ED4BA@psf.io>

https://hg.python.org/cpython/rev/579360b1c8fe
changeset:   102915:579360b1c8fe
branch:      3.5
parent:      102913:c75fcd094dcf
user:        Raymond Hettinger 
date:        Thu Aug 25 21:11:50 2016 -0700
summary:
  Issue 19504:  Change "customise" to "customize" American spelling.

files:
  Doc/reference/datamodel.rst         |  4 ++--
  Doc/whatsnew/2.0.rst                |  4 ++--
  Lib/http/cookiejar.py               |  2 +-
  Lib/test/cfgparser.2                |  2 +-
  Lib/test/test_gdb.py                |  2 +-
  Lib/unittest/test/test_discovery.py |  2 +-
  Objects/exceptions.c                |  2 +-
  Python/bltinmodule.c                |  4 ++--
  8 files changed, 11 insertions(+), 11 deletions(-)


diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst
--- a/Doc/reference/datamodel.rst
+++ b/Doc/reference/datamodel.rst
@@ -1117,7 +1117,7 @@
    instance; for example: ``BaseClass.__init__(self, [args...])``.
 
    Because :meth:`__new__` and :meth:`__init__` work together in constructing
-   objects (:meth:`__new__` to create it, and :meth:`__init__` to customise it),
+   objects (:meth:`__new__` to create it, and :meth:`__init__` to customize it),
    no non-``None`` value may be returned by :meth:`__init__`; doing so will
    cause a :exc:`TypeError` to be raised at runtime.
 
@@ -1637,7 +1637,7 @@
 executed in a new namespace and the class name is bound locally to the
 result of ``type(name, bases, namespace)``.
 
-The class creation process can be customised by passing the ``metaclass``
+The class creation process can be customized by passing the ``metaclass``
 keyword argument in the class definition line, or by inheriting from an
 existing class that included such an argument. In the following example,
 both ``MyClass`` and ``MySubclass`` are instances of ``Meta``::
diff --git a/Doc/whatsnew/2.0.rst b/Doc/whatsnew/2.0.rst
--- a/Doc/whatsnew/2.0.rst
+++ b/Doc/whatsnew/2.0.rst
@@ -166,7 +166,7 @@
 registering new encodings that are then available throughout a Python program.
 If an encoding isn't specified, the default encoding is usually 7-bit ASCII,
 though it can be changed for your Python installation by calling the
-``sys.setdefaultencoding(encoding)`` function in a customised version of
+``sys.setdefaultencoding(encoding)`` function in a customized version of
 :file:`site.py`.
 
 Combining 8-bit and Unicode strings always coerces to Unicode, using the default
@@ -1139,7 +1139,7 @@
   Unix, not to be confused with :program:`gzip`\ -format files (which are
   supported by the :mod:`gzip` module) (Contributed by James C. Ahlstrom.)
 
-* :mod:`imputil`: A module that provides a simpler way for writing customised
+* :mod:`imputil`: A module that provides a simpler way for writing customized
   import hooks, in comparison to the existing :mod:`ihooks` module.  (Implemented
   by Greg Stein, with much discussion on python-dev along the way.)
 
diff --git a/Lib/http/cookiejar.py b/Lib/http/cookiejar.py
--- a/Lib/http/cookiejar.py
+++ b/Lib/http/cookiejar.py
@@ -833,7 +833,7 @@
     May also modify cookies, though this is probably a bad idea.
 
     The subclass DefaultCookiePolicy defines the standard rules for Netscape
-    and RFC 2965 cookies -- override that if you want a customised policy.
+    and RFC 2965 cookies -- override that if you want a customized policy.
 
     """
     def set_ok(self, cookie, request):
diff --git a/Lib/test/cfgparser.2 b/Lib/test/cfgparser.2
--- a/Lib/test/cfgparser.2
+++ b/Lib/test/cfgparser.2
@@ -129,7 +129,7 @@
 # Unix users can map to different SMB User names
 ;  username map = /etc/samba/smbusers
 
-# Using the following line enables you to customise your configuration
+# Using the following line enables you to customize your configuration
 # on a per machine basis. The %m gets replaced with the netbios name
 # of the machine that is connecting
 ;   include = /etc/samba/smb.conf.%m
diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py
--- a/Lib/test/test_gdb.py
+++ b/Lib/test/test_gdb.py
@@ -91,7 +91,7 @@
     raise unittest.SkipTest("gdb not built with embedded python support")
 
 # Verify that "gdb" can load our custom hooks, as OS security settings may
-# disallow this without a customised .gdbinit.
+# disallow this without a customized .gdbinit.
 _, gdbpy_errors = run_gdb('--args', sys.executable)
 if "auto-loading has been declined" in gdbpy_errors:
     msg = "gdb security settings prevent use of custom hooks: "
diff --git a/Lib/unittest/test/test_discovery.py b/Lib/unittest/test/test_discovery.py
--- a/Lib/unittest/test/test_discovery.py
+++ b/Lib/unittest/test/test_discovery.py
@@ -275,7 +275,7 @@
         self.assertEqual(Module.load_tests_args,
                          [(loader, [], 'test*.py')])
 
-    def test_find_tests_customise_via_package_pattern(self):
+    def test_find_tests_customize_via_package_pattern(self):
         # This test uses the example 'do-nothing' load_tests from
         # https://docs.python.org/3/library/unittest.html#load-tests-protocol
         # to make sure that that actually works.
diff --git a/Objects/exceptions.c b/Objects/exceptions.c
--- a/Objects/exceptions.c
+++ b/Objects/exceptions.c
@@ -1263,7 +1263,7 @@
  *    SyntaxError extends Exception
  */
 
-/* Helper function to customise error message for some syntax errors */
+/* Helper function to customize error message for some syntax errors */
 static int _report_missing_parentheses(PySyntaxErrorObject *self);
 
 static int
diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c
--- a/Python/bltinmodule.c
+++ b/Python/bltinmodule.c
@@ -2068,7 +2068,7 @@
 
 Return a new list containing all items from the iterable in ascending order.
 
-A custom key function can be supplied to customise the sort order, and the
+A custom key function can be supplied to customize the sort order, and the
 reverse flag can be set to request the result in descending order.
 [end disabled clinic input]*/
 
@@ -2078,7 +2078,7 @@
 "\n"
 "Return a new list containing all items from the iterable in ascending order.\n"
 "\n"
-"A custom key function can be supplied to customise the sort order, and the\n"
+"A custom key function can be supplied to customize the sort order, and the\n"
 "reverse flag can be set to request the result in descending order.");
 
 #define BUILTIN_SORTED_METHODDEF    \

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Aug 26 00:12:23 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Fri, 26 Aug 2016 04:12:23 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
 =?utf-8?q?=29=3A_Merge?=
Message-ID: <20160826041222.82583.43367.C1D49CA7@psf.io>

https://hg.python.org/cpython/rev/1c38457da5a1
changeset:   102916:1c38457da5a1
parent:      102914:10a73648c92c
parent:      102915:579360b1c8fe
user:        Raymond Hettinger 
date:        Thu Aug 25 21:12:16 2016 -0700
summary:
  Merge

files:
  Doc/reference/datamodel.rst         |  4 ++--
  Doc/whatsnew/2.0.rst                |  4 ++--
  Lib/http/cookiejar.py               |  2 +-
  Lib/test/cfgparser.2                |  2 +-
  Lib/test/test_gdb.py                |  2 +-
  Lib/unittest/test/test_discovery.py |  2 +-
  Objects/exceptions.c                |  2 +-
  Python/bltinmodule.c                |  4 ++--
  8 files changed, 11 insertions(+), 11 deletions(-)


diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst
--- a/Doc/reference/datamodel.rst
+++ b/Doc/reference/datamodel.rst
@@ -1123,7 +1123,7 @@
    instance; for example: ``BaseClass.__init__(self, [args...])``.
 
    Because :meth:`__new__` and :meth:`__init__` work together in constructing
-   objects (:meth:`__new__` to create it, and :meth:`__init__` to customise it),
+   objects (:meth:`__new__` to create it, and :meth:`__init__` to customize it),
    no non-``None`` value may be returned by :meth:`__init__`; doing so will
    cause a :exc:`TypeError` to be raised at runtime.
 
@@ -1697,7 +1697,7 @@
 executed in a new namespace and the class name is bound locally to the
 result of ``type(name, bases, namespace)``.
 
-The class creation process can be customised by passing the ``metaclass``
+The class creation process can be customized by passing the ``metaclass``
 keyword argument in the class definition line, or by inheriting from an
 existing class that included such an argument. In the following example,
 both ``MyClass`` and ``MySubclass`` are instances of ``Meta``::
diff --git a/Doc/whatsnew/2.0.rst b/Doc/whatsnew/2.0.rst
--- a/Doc/whatsnew/2.0.rst
+++ b/Doc/whatsnew/2.0.rst
@@ -166,7 +166,7 @@
 registering new encodings that are then available throughout a Python program.
 If an encoding isn't specified, the default encoding is usually 7-bit ASCII,
 though it can be changed for your Python installation by calling the
-``sys.setdefaultencoding(encoding)`` function in a customised version of
+``sys.setdefaultencoding(encoding)`` function in a customized version of
 :file:`site.py`.
 
 Combining 8-bit and Unicode strings always coerces to Unicode, using the default
@@ -1139,7 +1139,7 @@
   Unix, not to be confused with :program:`gzip`\ -format files (which are
   supported by the :mod:`gzip` module) (Contributed by James C. Ahlstrom.)
 
-* :mod:`imputil`: A module that provides a simpler way for writing customised
+* :mod:`imputil`: A module that provides a simpler way for writing customized
   import hooks, in comparison to the existing :mod:`ihooks` module.  (Implemented
   by Greg Stein, with much discussion on python-dev along the way.)
 
diff --git a/Lib/http/cookiejar.py b/Lib/http/cookiejar.py
--- a/Lib/http/cookiejar.py
+++ b/Lib/http/cookiejar.py
@@ -833,7 +833,7 @@
     May also modify cookies, though this is probably a bad idea.
 
     The subclass DefaultCookiePolicy defines the standard rules for Netscape
-    and RFC 2965 cookies -- override that if you want a customised policy.
+    and RFC 2965 cookies -- override that if you want a customized policy.
 
     """
     def set_ok(self, cookie, request):
diff --git a/Lib/test/cfgparser.2 b/Lib/test/cfgparser.2
--- a/Lib/test/cfgparser.2
+++ b/Lib/test/cfgparser.2
@@ -129,7 +129,7 @@
 # Unix users can map to different SMB User names
 ;  username map = /etc/samba/smbusers
 
-# Using the following line enables you to customise your configuration
+# Using the following line enables you to customize your configuration
 # on a per machine basis. The %m gets replaced with the netbios name
 # of the machine that is connecting
 ;   include = /etc/samba/smb.conf.%m
diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py
--- a/Lib/test/test_gdb.py
+++ b/Lib/test/test_gdb.py
@@ -90,7 +90,7 @@
     raise unittest.SkipTest("gdb not built with embedded python support")
 
 # Verify that "gdb" can load our custom hooks, as OS security settings may
-# disallow this without a customised .gdbinit.
+# disallow this without a customized .gdbinit.
 _, gdbpy_errors = run_gdb('--args', sys.executable)
 if "auto-loading has been declined" in gdbpy_errors:
     msg = "gdb security settings prevent use of custom hooks: "
diff --git a/Lib/unittest/test/test_discovery.py b/Lib/unittest/test/test_discovery.py
--- a/Lib/unittest/test/test_discovery.py
+++ b/Lib/unittest/test/test_discovery.py
@@ -275,7 +275,7 @@
         self.assertEqual(Module.load_tests_args,
                          [(loader, [], 'test*.py')])
 
-    def test_find_tests_customise_via_package_pattern(self):
+    def test_find_tests_customize_via_package_pattern(self):
         # This test uses the example 'do-nothing' load_tests from
         # https://docs.python.org/3/library/unittest.html#load-tests-protocol
         # to make sure that that actually works.
diff --git a/Objects/exceptions.c b/Objects/exceptions.c
--- a/Objects/exceptions.c
+++ b/Objects/exceptions.c
@@ -1256,7 +1256,7 @@
  *    SyntaxError extends Exception
  */
 
-/* Helper function to customise error message for some syntax errors */
+/* Helper function to customize error message for some syntax errors */
 static int _report_missing_parentheses(PySyntaxErrorObject *self);
 
 static int
diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c
--- a/Python/bltinmodule.c
+++ b/Python/bltinmodule.c
@@ -2072,7 +2072,7 @@
 
 Return a new list containing all items from the iterable in ascending order.
 
-A custom key function can be supplied to customise the sort order, and the
+A custom key function can be supplied to customize the sort order, and the
 reverse flag can be set to request the result in descending order.
 [end disabled clinic input]*/
 
@@ -2082,7 +2082,7 @@
 "\n"
 "Return a new list containing all items from the iterable in ascending order.\n"
 "\n"
-"A custom key function can be supplied to customise the sort order, and the\n"
+"A custom key function can be supplied to customize the sort order, and the\n"
 "reverse flag can be set to request the result in descending order.");
 
 #define BUILTIN_SORTED_METHODDEF    \

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Fri Aug 26 08:00:06 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Fri, 26 Aug 2016 13:00:06 +0100
Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-08-26
Message-ID: 

Results for project Python 2.7, build date 2016-08-26 02:46:52 +0000
commit:		24e580e667fb
previous commit:	862761e4376e
revision date:	2016-08-26 00:04:08 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.13%         -1.14%          4.26%             7.98%
:-)             pybench      0.21%          0.01%          6.15%             4.26%
:-|            regex_v8      0.61%          0.10%         -1.95%            10.66%
:-)               nbody      0.10%         -0.02%          7.71%             3.39%
:-)        json_dump_v2      0.50%          0.28%          3.27%            10.71%
:-|      normal_startup      0.52%         -0.78%         -0.87%             2.81%
:-)             ssbench      0.16%          0.31%          3.02%             0.49%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-08-26/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Fri Aug 26 08:01:48 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Fri, 26 Aug 2016 13:01:48 +0100
Subject: [Python-checkins] BAD Benchmark Results for Python Default
 2016-08-26
Message-ID: <177836b4-6a36-4e52-9b9a-0e64db29e997@irsmsx151.ger.corp.intel.com>

Results for project Python default, build date 2016-08-26 02:00:31 +0000
commit:		10a73648c92c
previous commit:	316e5de8a96b
revision date:	2016-08-26 00:05:30 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.15%         -2.94%          8.27%            18.86%
:-)             pybench      0.76%          0.68%          6.59%             7.76%
:-(            regex_v8      2.67%         -2.56%         -4.55%             6.03%
:-|               nbody      0.11%         -0.92%          0.02%            11.13%
:-|        json_dump_v2      0.30%          0.26%         -1.54%            11.19%
:-|      normal_startup      0.97%          0.72%          0.57%             6.71%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-default-2016-08-26/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Fri Aug 26 15:07:35 2016
From: python-checkins at python.org (berker.peksag)
Date: Fri, 26 Aug 2016 19:07:35 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2310513=3A_Merge_from_3=2E5?=
Message-ID: <20160826190735.20148.59235.B82EE147@psf.io>

https://hg.python.org/cpython/rev/685f32972c11
changeset:   102918:685f32972c11
parent:      102916:1c38457da5a1
parent:      102917:81f614dd8136
user:        Berker Peksag 
date:        Fri Aug 26 22:08:51 2016 +0300
summary:
  Issue #10513: Merge from 3.5

files:
  Lib/sqlite3/test/regression.py |  31 ++++++++++++++++++++++
  Misc/NEWS                      |   3 ++
  Modules/_sqlite/connection.c   |   1 -
  3 files changed, 34 insertions(+), 1 deletions(-)


diff --git a/Lib/sqlite3/test/regression.py b/Lib/sqlite3/test/regression.py
--- a/Lib/sqlite3/test/regression.py
+++ b/Lib/sqlite3/test/regression.py
@@ -322,6 +322,37 @@
         self.assertRaises(ValueError, cur.execute, " \0select 2")
         self.assertRaises(ValueError, cur.execute, "select 2\0")
 
+    def CheckCommitCursorReset(self):
+        """
+        Connection.commit() did reset cursors, which made sqlite3
+        to return rows multiple times when fetched from cursors
+        after commit. See issues 10513 and 23129 for details.
+        """
+        con = sqlite.connect(":memory:")
+        con.executescript("""
+        create table t(c);
+        create table t2(c);
+        insert into t values(0);
+        insert into t values(1);
+        insert into t values(2);
+        """)
+
+        self.assertEqual(con.isolation_level, "")
+
+        counter = 0
+        for i, row in enumerate(con.execute("select c from t")):
+            with self.subTest(i=i, row=row):
+                con.execute("insert into t2(c) values (?)", (i,))
+                con.commit()
+                if counter == 0:
+                    self.assertEqual(row[0], 0)
+                elif counter == 1:
+                    self.assertEqual(row[0], 1)
+                elif counter == 2:
+                    self.assertEqual(row[0], 2)
+                counter += 1
+        self.assertEqual(counter, 3, "should have returned exactly three rows")
+
 
 def suite():
     regression_suite = unittest.makeSuite(RegressionTests, "Check")
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -46,6 +46,9 @@
 Library
 -------
 
+- Issue #10513: Fix a regression in Connection.commit().  Statements should
+  not be reset after a commit.
+
 - Issue #12319: Chunked transfer encoding support added to
   http.client.HTTPConnection requests.  The
   urllib.request.AbstractHTTPHandler class does not enforce a Content-Length
diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c
--- a/Modules/_sqlite/connection.c
+++ b/Modules/_sqlite/connection.c
@@ -425,7 +425,6 @@
     }
 
     if (self->inTransaction) {
-        pysqlite_do_all_statements(self, ACTION_RESET, 0);
 
         Py_BEGIN_ALLOW_THREADS
         rc = sqlite3_prepare(self->db, "COMMIT", -1, &statement, &tail);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Aug 26 15:07:35 2016
From: python-checkins at python.org (berker.peksag)
Date: Fri, 26 Aug 2016 19:07:35 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzEwNTEz?=
 =?utf-8?q?=3A_Fix_a_regression_in_Connection=2Ecommit=28=29?=
Message-ID: <20160826190735.39106.80497.4603A50C@psf.io>

https://hg.python.org/cpython/rev/81f614dd8136
changeset:   102917:81f614dd8136
branch:      3.5
parent:      102915:579360b1c8fe
user:        Berker Peksag 
date:        Fri Aug 26 22:07:51 2016 +0300
summary:
  Issue #10513: Fix a regression in Connection.commit()

Statements should not be reset after a commit.

Backported from https://github.com/ghaering/pysqlite/commit/029050896b1e6058573abeef5a8970384c0c7faa

files:
  Lib/sqlite3/test/regression.py |  31 ++++++++++++++++++++++
  Misc/NEWS                      |   3 ++
  Modules/_sqlite/connection.c   |   1 -
  3 files changed, 34 insertions(+), 1 deletions(-)


diff --git a/Lib/sqlite3/test/regression.py b/Lib/sqlite3/test/regression.py
--- a/Lib/sqlite3/test/regression.py
+++ b/Lib/sqlite3/test/regression.py
@@ -322,6 +322,37 @@
         self.assertRaises(ValueError, cur.execute, " \0select 2")
         self.assertRaises(ValueError, cur.execute, "select 2\0")
 
+    def CheckCommitCursorReset(self):
+        """
+        Connection.commit() did reset cursors, which made sqlite3
+        to return rows multiple times when fetched from cursors
+        after commit. See issues 10513 and 23129 for details.
+        """
+        con = sqlite.connect(":memory:")
+        con.executescript("""
+        create table t(c);
+        create table t2(c);
+        insert into t values(0);
+        insert into t values(1);
+        insert into t values(2);
+        """)
+
+        self.assertEqual(con.isolation_level, "")
+
+        counter = 0
+        for i, row in enumerate(con.execute("select c from t")):
+            with self.subTest(i=i, row=row):
+                con.execute("insert into t2(c) values (?)", (i,))
+                con.commit()
+                if counter == 0:
+                    self.assertEqual(row[0], 0)
+                elif counter == 1:
+                    self.assertEqual(row[0], 1)
+                elif counter == 2:
+                    self.assertEqual(row[0], 2)
+                counter += 1
+        self.assertEqual(counter, 3, "should have returned exactly three rows")
+
 
 def suite():
     regression_suite = unittest.makeSuite(RegressionTests, "Check")
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -50,6 +50,9 @@
 Library
 -------
 
+- Issue #10513: Fix a regression in Connection.commit().  Statements should
+  not be reset after a commit.
+
 - A new version of typing.py from https://github.com/python/typing:
   - Collection (only for 3.6) (Issue #27598)
   - Add FrozenSet to __all__ (upstream #261)
diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c
--- a/Modules/_sqlite/connection.c
+++ b/Modules/_sqlite/connection.c
@@ -426,7 +426,6 @@
     }
 
     if (self->inTransaction) {
-        pysqlite_do_all_statements(self, ACTION_RESET, 0);
 
         Py_BEGIN_ALLOW_THREADS
         rc = sqlite3_prepare(self->db, "COMMIT", -1, &statement, &tail);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Aug 26 15:17:47 2016
From: python-checkins at python.org (berker.peksag)
Date: Fri, 26 Aug 2016 19:17:47 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEwNTEz?=
 =?utf-8?q?=3A_Fix_a_regression_in_Connection=2Ecommit=28=29?=
Message-ID: <20160826191747.2296.56831.5D2E40BB@psf.io>

https://hg.python.org/cpython/rev/030e100f048a
changeset:   102919:030e100f048a
branch:      2.7
parent:      102912:24e580e667fb
user:        Berker Peksag 
date:        Fri Aug 26 22:19:05 2016 +0300
summary:
  Issue #10513: Fix a regression in Connection.commit()

Statements should not be reset after a commit.

Backported from https://github.com/ghaering/pysqlite/commit/029050896b1e6058573abeef5a8970384c0c7faa

files:
  Lib/sqlite3/test/regression.py |  30 ++++++++++++++++++++++
  Misc/NEWS                      |   3 ++
  Modules/_sqlite/connection.c   |   1 -
  3 files changed, 33 insertions(+), 1 deletions(-)


diff --git a/Lib/sqlite3/test/regression.py b/Lib/sqlite3/test/regression.py
--- a/Lib/sqlite3/test/regression.py
+++ b/Lib/sqlite3/test/regression.py
@@ -328,6 +328,36 @@
         self.assertRaises(ValueError, cur.execute, " \0select 2")
         self.assertRaises(ValueError, cur.execute, "select 2\0")
 
+    def CheckCommitCursorReset(self):
+        """
+        Connection.commit() did reset cursors, which made sqlite3
+        to return rows multiple times when fetched from cursors
+        after commit. See issues 10513 and 23129 for details.
+        """
+        con = sqlite.connect(":memory:")
+        con.executescript("""
+        create table t(c);
+        create table t2(c);
+        insert into t values(0);
+        insert into t values(1);
+        insert into t values(2);
+        """)
+
+        self.assertEqual(con.isolation_level, "")
+
+        counter = 0
+        for i, row in enumerate(con.execute("select c from t")):
+            con.execute("insert into t2(c) values (?)", (i,))
+            con.commit()
+            if counter == 0:
+                self.assertEqual(row[0], 0)
+            elif counter == 1:
+                self.assertEqual(row[0], 1)
+            elif counter == 2:
+                self.assertEqual(row[0], 2)
+            counter += 1
+        self.assertEqual(counter, 3, "should have returned exactly three rows")
+
 
 def suite():
     regression_suite = unittest.makeSuite(RegressionTests, "Check")
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -33,6 +33,9 @@
 Library
 -------
 
+- Issue #10513: Fix a regression in Connection.commit().  Statements should
+  not be reset after a commit.
+
 - Issue #2466: posixpath.ismount now correctly recognizes mount points which
   the user does not have permission to access.
 
diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c
--- a/Modules/_sqlite/connection.c
+++ b/Modules/_sqlite/connection.c
@@ -458,7 +458,6 @@
     }
 
     if (self->inTransaction) {
-        pysqlite_do_all_statements(self, ACTION_RESET, 0);
 
         Py_BEGIN_ALLOW_THREADS
         rc = sqlite3_prepare(self->db, "COMMIT", -1, &statement, &tail);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Aug 26 17:45:22 2016
From: python-checkins at python.org (brett.cannon)
Date: Fri, 26 Aug 2016 21:45:22 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326027=2C_=2327524?=
 =?utf-8?q?=3A_Add_PEP_519/=5F=5Ffspath=5F=5F=28=29_support_to_os_and?=
Message-ID: <20160826214521.1752.7043.4A5B5A22@psf.io>

https://hg.python.org/cpython/rev/b64f83d6ff24
changeset:   102920:b64f83d6ff24
parent:      102918:685f32972c11
user:        Brett Cannon 
date:        Fri Aug 26 14:44:48 2016 -0700
summary:
  Issue #26027, #27524: Add PEP 519/__fspath__() support to os and
os.path.

Thanks to Jelle Zijlstra for the initial patch against posixmodule.c.

files:
  Lib/genericpath.py           |    6 +
  Lib/ntpath.py                |   18 +++-
  Lib/os.py                    |    4 +-
  Lib/posixpath.py             |   19 +++-
  Lib/test/test_genericpath.py |   64 +++++++++++--
  Lib/test/test_ntpath.py      |   83 +++++++++++++++++
  Lib/test/test_os.py          |   83 ++++++++++++++++-
  Lib/test/test_posix.py       |   10 +-
  Lib/test/test_posixpath.py   |   80 ++++++++++++++++
  Misc/NEWS                    |    5 +-
  Modules/posixmodule.c        |  112 ++++++++++++++++------
  11 files changed, 428 insertions(+), 56 deletions(-)


diff --git a/Lib/genericpath.py b/Lib/genericpath.py
--- a/Lib/genericpath.py
+++ b/Lib/genericpath.py
@@ -69,6 +69,12 @@
 def commonprefix(m):
     "Given a list of pathnames, returns the longest common leading component"
     if not m: return ''
+    # Some people pass in a list of pathname parts to operate in an OS-agnostic
+    # fashion; don't try to translate in that case as that's an abuse of the
+    # API and they are already doing what they need to be OS-agnostic and so
+    # they most likely won't be using an os.PathLike object in the sublists.
+    if not isinstance(m[0], (list, tuple)):
+        m = tuple(map(os.fspath, m))
     s1 = min(m)
     s2 = max(m)
     for i, c in enumerate(s1):
diff --git a/Lib/ntpath.py b/Lib/ntpath.py
--- a/Lib/ntpath.py
+++ b/Lib/ntpath.py
@@ -46,6 +46,7 @@
     """Normalize case of pathname.
 
     Makes all characters lowercase and all slashes into backslashes."""
+    s = os.fspath(s)
     try:
         if isinstance(s, bytes):
             return s.replace(b'/', b'\\').lower()
@@ -66,12 +67,14 @@
 
 def isabs(s):
     """Test whether a path is absolute"""
+    s = os.fspath(s)
     s = splitdrive(s)[1]
     return len(s) > 0 and s[0] in _get_bothseps(s)
 
 
 # Join two (or more) paths.
 def join(path, *paths):
+    path = os.fspath(path)
     if isinstance(path, bytes):
         sep = b'\\'
         seps = b'\\/'
@@ -84,7 +87,7 @@
         if not paths:
             path[:0] + sep  #23780: Ensure compatible data type even if p is null.
         result_drive, result_path = splitdrive(path)
-        for p in paths:
+        for p in map(os.fspath, paths):
             p_drive, p_path = splitdrive(p)
             if p_path and p_path[0] in seps:
                 # Second path is absolute
@@ -136,6 +139,7 @@
     Paths cannot contain both a drive letter and a UNC path.
 
     """
+    p = os.fspath(p)
     if len(p) >= 2:
         if isinstance(p, bytes):
             sep = b'\\'
@@ -199,7 +203,7 @@
 
     Return tuple (head, tail) where tail is everything after the final slash.
     Either part may be empty."""
-
+    p = os.fspath(p)
     seps = _get_bothseps(p)
     d, p = splitdrive(p)
     # set i to index beyond p's last slash
@@ -218,6 +222,7 @@
 # It is always true that root + ext == p.
 
 def splitext(p):
+    p = os.fspath(p)
     if isinstance(p, bytes):
         return genericpath._splitext(p, b'\\', b'/', b'.')
     else:
@@ -278,6 +283,7 @@
 def ismount(path):
     """Test whether a path is a mount point (a drive root, the root of a
     share, or a mounted volume)"""
+    path = os.fspath(path)
     seps = _get_bothseps(path)
     path = abspath(path)
     root, rest = splitdrive(path)
@@ -305,6 +311,7 @@
     """Expand ~ and ~user constructs.
 
     If user or $HOME is unknown, do nothing."""
+    path = os.fspath(path)
     if isinstance(path, bytes):
         tilde = b'~'
     else:
@@ -354,6 +361,7 @@
     """Expand shell variables of the forms $var, ${var} and %var%.
 
     Unknown variables are left unchanged."""
+    path = os.fspath(path)
     if isinstance(path, bytes):
         if b'$' not in path and b'%' not in path:
             return path
@@ -464,6 +472,7 @@
 
 def normpath(path):
     """Normalize path, eliminating double slashes, etc."""
+    path = os.fspath(path)
     if isinstance(path, bytes):
         sep = b'\\'
         altsep = b'/'
@@ -518,6 +527,7 @@
 except ImportError: # not running on Windows - mock up something sensible
     def abspath(path):
         """Return the absolute version of a path."""
+        path = os.fspath(path)
         if not isabs(path):
             if isinstance(path, bytes):
                 cwd = os.getcwdb()
@@ -531,6 +541,7 @@
         """Return the absolute version of a path."""
 
         if path: # Empty path must return current working directory.
+            path = os.fspath(path)
             try:
                 path = _getfullpathname(path)
             except OSError:
@@ -549,6 +560,7 @@
 
 def relpath(path, start=None):
     """Return a relative version of a path"""
+    path = os.fspath(path)
     if isinstance(path, bytes):
         sep = b'\\'
         curdir = b'.'
@@ -564,6 +576,7 @@
     if not path:
         raise ValueError("no path specified")
 
+    start = os.fspath(start)
     try:
         start_abs = abspath(normpath(start))
         path_abs = abspath(normpath(path))
@@ -607,6 +620,7 @@
     if not paths:
         raise ValueError('commonpath() arg is an empty sequence')
 
+    paths = tuple(map(os.fspath, paths))
     if isinstance(paths[0], bytes):
         sep = b'\\'
         altsep = b'/'
diff --git a/Lib/os.py b/Lib/os.py
--- a/Lib/os.py
+++ b/Lib/os.py
@@ -353,7 +353,7 @@
             dirs.remove('CVS')  # don't visit CVS directories
 
     """
-
+    top = fspath(top)
     dirs = []
     nondirs = []
     walk_dirs = []
@@ -536,6 +536,8 @@
             if 'CVS' in dirs:
                 dirs.remove('CVS')  # don't visit CVS directories
         """
+        if not isinstance(top, int) or not hasattr(top, '__index__'):
+            top = fspath(top)
         # Note: To guard against symlink races, we use the standard
         # lstat()/open()/fstat() trick.
         orig_st = stat(top, follow_symlinks=False, dir_fd=dir_fd)
diff --git a/Lib/posixpath.py b/Lib/posixpath.py
--- a/Lib/posixpath.py
+++ b/Lib/posixpath.py
@@ -49,6 +49,7 @@
 
 def normcase(s):
     """Normalize case of pathname.  Has no effect under Posix"""
+    s = os.fspath(s)
     if not isinstance(s, (bytes, str)):
         raise TypeError("normcase() argument must be str or bytes, "
                         "not '{}'".format(s.__class__.__name__))
@@ -60,6 +61,7 @@
 
 def isabs(s):
     """Test whether a path is absolute"""
+    s = os.fspath(s)
     sep = _get_sep(s)
     return s.startswith(sep)
 
@@ -73,12 +75,13 @@
     If any component is an absolute path, all previous path components
     will be discarded.  An empty last part will result in a path that
     ends with a separator."""
+    a = os.fspath(a)
     sep = _get_sep(a)
     path = a
     try:
         if not p:
             path[:0] + sep  #23780: Ensure compatible data type even if p is null.
-        for b in p:
+        for b in map(os.fspath, p):
             if b.startswith(sep):
                 path = b
             elif not path or path.endswith(sep):
@@ -99,6 +102,7 @@
 def split(p):
     """Split a pathname.  Returns tuple "(head, tail)" where "tail" is
     everything after the final slash.  Either part may be empty."""
+    p = os.fspath(p)
     sep = _get_sep(p)
     i = p.rfind(sep) + 1
     head, tail = p[:i], p[i:]
@@ -113,6 +117,7 @@
 # It is always true that root + ext == p.
 
 def splitext(p):
+    p = os.fspath(p)
     if isinstance(p, bytes):
         sep = b'/'
         extsep = b'.'
@@ -128,6 +133,7 @@
 def splitdrive(p):
     """Split a pathname into drive and path. On Posix, drive is always
     empty."""
+    p = os.fspath(p)
     return p[:0], p
 
 
@@ -135,6 +141,7 @@
 
 def basename(p):
     """Returns the final component of a pathname"""
+    p = os.fspath(p)
     sep = _get_sep(p)
     i = p.rfind(sep) + 1
     return p[i:]
@@ -144,6 +151,7 @@
 
 def dirname(p):
     """Returns the directory component of a pathname"""
+    p = os.fspath(p)
     sep = _get_sep(p)
     i = p.rfind(sep) + 1
     head = p[:i]
@@ -222,6 +230,7 @@
 def expanduser(path):
     """Expand ~ and ~user constructions.  If user or $HOME is unknown,
     do nothing."""
+    path = os.fspath(path)
     if isinstance(path, bytes):
         tilde = b'~'
     else:
@@ -267,6 +276,7 @@
 def expandvars(path):
     """Expand shell variables of form $var and ${var}.  Unknown variables
     are left unchanged."""
+    path = os.fspath(path)
     global _varprog, _varprogb
     if isinstance(path, bytes):
         if b'$' not in path:
@@ -318,6 +328,7 @@
 
 def normpath(path):
     """Normalize path, eliminating double slashes, etc."""
+    path = os.fspath(path)
     if isinstance(path, bytes):
         sep = b'/'
         empty = b''
@@ -355,6 +366,7 @@
 
 def abspath(path):
     """Return an absolute path."""
+    path = os.fspath(path)
     if not isabs(path):
         if isinstance(path, bytes):
             cwd = os.getcwdb()
@@ -370,6 +382,7 @@
 def realpath(filename):
     """Return the canonical path of the specified filename, eliminating any
 symbolic links encountered in the path."""
+    filename = os.fspath(filename)
     path, ok = _joinrealpath(filename[:0], filename, {})
     return abspath(path)
 
@@ -434,6 +447,7 @@
     if not path:
         raise ValueError("no path specified")
 
+    path = os.fspath(path)
     if isinstance(path, bytes):
         curdir = b'.'
         sep = b'/'
@@ -445,6 +459,8 @@
 
     if start is None:
         start = curdir
+    else:
+        start = os.fspath(start)
 
     try:
         start_list = [x for x in abspath(start).split(sep) if x]
@@ -472,6 +488,7 @@
     if not paths:
         raise ValueError('commonpath() arg is an empty sequence')
 
+    paths = tuple(map(os.fspath, paths))
     if isinstance(paths[0], bytes):
         sep = b'/'
         curdir = b'.'
diff --git a/Lib/test/test_genericpath.py b/Lib/test/test_genericpath.py
--- a/Lib/test/test_genericpath.py
+++ b/Lib/test/test_genericpath.py
@@ -450,16 +450,15 @@
             with self.assertRaisesRegex(TypeError, errmsg):
                 self.pathmodule.join('str', b'bytes')
             # regression, see #15377
-            errmsg = r'join\(\) argument must be str or bytes, not %r'
-            with self.assertRaisesRegex(TypeError, errmsg % 'int'):
+            with self.assertRaisesRegex(TypeError, 'int'):
                 self.pathmodule.join(42, 'str')
-            with self.assertRaisesRegex(TypeError, errmsg % 'int'):
+            with self.assertRaisesRegex(TypeError, 'int'):
                 self.pathmodule.join('str', 42)
-            with self.assertRaisesRegex(TypeError, errmsg % 'int'):
+            with self.assertRaisesRegex(TypeError, 'int'):
                 self.pathmodule.join(42)
-            with self.assertRaisesRegex(TypeError, errmsg % 'list'):
+            with self.assertRaisesRegex(TypeError, 'list'):
                 self.pathmodule.join([])
-            with self.assertRaisesRegex(TypeError, errmsg % 'bytearray'):
+            with self.assertRaisesRegex(TypeError, 'bytearray'):
                 self.pathmodule.join(bytearray(b'foo'), bytearray(b'bar'))
 
     def test_relpath_errors(self):
@@ -471,14 +470,59 @@
                 self.pathmodule.relpath(b'bytes', 'str')
             with self.assertRaisesRegex(TypeError, errmsg):
                 self.pathmodule.relpath('str', b'bytes')
-            errmsg = r'relpath\(\) argument must be str or bytes, not %r'
-            with self.assertRaisesRegex(TypeError, errmsg % 'int'):
+            with self.assertRaisesRegex(TypeError, 'int'):
                 self.pathmodule.relpath(42, 'str')
-            with self.assertRaisesRegex(TypeError, errmsg % 'int'):
+            with self.assertRaisesRegex(TypeError, 'int'):
                 self.pathmodule.relpath('str', 42)
-            with self.assertRaisesRegex(TypeError, errmsg % 'bytearray'):
+            with self.assertRaisesRegex(TypeError, 'bytearray'):
                 self.pathmodule.relpath(bytearray(b'foo'), bytearray(b'bar'))
 
 
+class PathLikeTests(unittest.TestCase):
+
+    class PathLike:
+        def __init__(self, path=''):
+            self.path = path
+        def __fspath__(self):
+            if isinstance(self.path, BaseException):
+                raise self.path
+            else:
+                return self.path
+
+    def setUp(self):
+        self.file_name = support.TESTFN.lower()
+        self.file_path = self.PathLike(support.TESTFN)
+        self.addCleanup(support.unlink, self.file_name)
+        create_file(self.file_name, b"test_genericpath.PathLikeTests")
+
+    def assertPathEqual(self, func):
+        self.assertEqual(func(self.file_path), func(self.file_name))
+
+    def test_path_exists(self):
+        self.assertPathEqual(os.path.exists)
+
+    def test_path_isfile(self):
+        self.assertPathEqual(os.path.isfile)
+
+    def test_path_isdir(self):
+        self.assertPathEqual(os.path.isdir)
+
+    def test_path_commonprefix(self):
+        self.assertEqual(os.path.commonprefix([self.file_path, self.file_name]),
+                         self.file_name)
+
+    def test_path_getsize(self):
+        self.assertPathEqual(os.path.getsize)
+
+    def test_path_getmtime(self):
+        self.assertPathEqual(os.path.getatime)
+
+    def test_path_getctime(self):
+        self.assertPathEqual(os.path.getctime)
+
+    def test_path_samefile(self):
+        self.assertTrue(os.path.samefile(self.file_path, self.file_name))
+
+
 if __name__=="__main__":
     unittest.main()
diff --git a/Lib/test/test_ntpath.py b/Lib/test/test_ntpath.py
--- a/Lib/test/test_ntpath.py
+++ b/Lib/test/test_ntpath.py
@@ -452,5 +452,88 @@
     attributes = ['relpath', 'splitunc']
 
 
+class PathLikeTests(unittest.TestCase):
+
+    path = ntpath
+
+    class PathLike:
+        def __init__(self, path=''):
+            self.path = path
+        def __fspath__(self):
+            if isinstance(self.path, BaseException):
+                raise self.path
+            else:
+                return self.path
+
+    def setUp(self):
+        self.file_name = support.TESTFN.lower()
+        self.file_path = self.PathLike(support.TESTFN)
+        self.addCleanup(support.unlink, self.file_name)
+        with open(self.file_name, 'xb', 0) as file:
+            file.write(b"test_ntpath.PathLikeTests")
+
+    def assertPathEqual(self, func):
+        self.assertEqual(func(self.file_path), func(self.file_name))
+
+    def test_path_normcase(self):
+        self.assertPathEqual(self.path.normcase)
+
+    def test_path_isabs(self):
+        self.assertPathEqual(self.path.isabs)
+
+    def test_path_join(self):
+        self.assertEqual(self.path.join('a', self.PathLike('b'), 'c'),
+                         self.path.join('a', 'b', 'c'))
+
+    def test_path_split(self):
+        self.assertPathEqual(self.path.split)
+
+    def test_path_splitext(self):
+        self.assertPathEqual(self.path.splitext)
+
+    def test_path_splitdrive(self):
+        self.assertPathEqual(self.path.splitdrive)
+
+    def test_path_basename(self):
+        self.assertPathEqual(self.path.basename)
+
+    def test_path_dirname(self):
+        self.assertPathEqual(self.path.dirname)
+
+    def test_path_islink(self):
+        self.assertPathEqual(self.path.islink)
+
+    def test_path_lexists(self):
+        self.assertPathEqual(self.path.lexists)
+
+    def test_path_ismount(self):
+        self.assertPathEqual(self.path.ismount)
+
+    def test_path_expanduser(self):
+        self.assertPathEqual(self.path.expanduser)
+
+    def test_path_expandvars(self):
+        self.assertPathEqual(self.path.expandvars)
+
+    def test_path_normpath(self):
+        self.assertPathEqual(self.path.normpath)
+
+    def test_path_abspath(self):
+        self.assertPathEqual(self.path.abspath)
+
+    def test_path_realpath(self):
+        self.assertPathEqual(self.path.realpath)
+
+    def test_path_relpath(self):
+        self.assertPathEqual(self.path.relpath)
+
+    def test_path_commonpath(self):
+        common_path = self.path.commonpath([self.file_path, self.file_name])
+        self.assertEqual(common_path, self.file_name)
+
+    def test_path_isdir(self):
+        self.assertPathEqual(self.path.isdir)
+
+
 if __name__ == "__main__":
     unittest.main()
diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
--- a/Lib/test/test_os.py
+++ b/Lib/test/test_os.py
@@ -874,10 +874,12 @@
         self.assertEqual(all[2 + flipped], (self.sub11_path, [], []))
         self.assertEqual(all[3 - 2 * flipped], self.sub2_tree)
 
-    def test_walk_prune(self):
+    def test_walk_prune(self, walk_path=None):
+        if walk_path is None:
+            walk_path = self.walk_path
         # Prune the search.
         all = []
-        for root, dirs, files in self.walk(self.walk_path):
+        for root, dirs, files in self.walk(walk_path):
             all.append((root, dirs, files))
             # Don't descend into SUB1.
             if 'SUB1' in dirs:
@@ -886,11 +888,22 @@
 
         self.assertEqual(len(all), 2)
         self.assertEqual(all[0],
-                         (self.walk_path, ["SUB2"], ["tmp1"]))
+                         (str(walk_path), ["SUB2"], ["tmp1"]))
 
         all[1][-1].sort()
         self.assertEqual(all[1], self.sub2_tree)
 
+    def test_file_like_path(self):
+        class FileLike:
+            def __init__(self, path):
+                self._path = path
+            def __str__(self):
+                return str(self._path)
+            def __fspath__(self):
+                return self._path
+
+        self.test_walk_prune(FileLike(self.walk_path))
+
     def test_walk_bottom_up(self):
         # Walk bottom-up.
         all = list(self.walk(self.walk_path, topdown=False))
@@ -2807,6 +2820,70 @@
         self.assertEqual(os.get_inheritable(slave_fd), False)
 
 
+class PathTConverterTests(unittest.TestCase):
+    # tuples of (function name, allows fd arguments, additional arguments to
+    # function, cleanup function)
+    functions = [
+        ('stat', True, (), None),
+        ('lstat', False, (), None),
+        ('access', True, (os.F_OK,), None),
+        ('chflags', False, (0,), None),
+        ('lchflags', False, (0,), None),
+        ('open', False, (0,), getattr(os, 'close', None)),
+    ]
+
+    def test_path_t_converter(self):
+        class PathLike:
+            def __init__(self, path):
+                self.path = path
+
+            def __fspath__(self):
+                return self.path
+
+        str_filename = support.TESTFN
+        bytes_filename = support.TESTFN.encode('ascii')
+        bytearray_filename = bytearray(bytes_filename)
+        fd = os.open(PathLike(str_filename), os.O_WRONLY|os.O_CREAT)
+        self.addCleanup(os.close, fd)
+        self.addCleanup(support.unlink, support.TESTFN)
+
+        int_fspath = PathLike(fd)
+        str_fspath = PathLike(str_filename)
+        bytes_fspath = PathLike(bytes_filename)
+        bytearray_fspath = PathLike(bytearray_filename)
+
+        for name, allow_fd, extra_args, cleanup_fn in self.functions:
+            with self.subTest(name=name):
+                try:
+                    fn = getattr(os, name)
+                except AttributeError:
+                    continue
+
+                for path in (str_filename, bytes_filename, bytearray_filename,
+                             str_fspath, bytes_fspath):
+                    with self.subTest(name=name, path=path):
+                        result = fn(path, *extra_args)
+                        if cleanup_fn is not None:
+                            cleanup_fn(result)
+
+                with self.assertRaisesRegex(
+                        TypeError, 'should be string, bytes'):
+                    fn(int_fspath, *extra_args)
+                with self.assertRaisesRegex(
+                        TypeError, 'should be string, bytes'):
+                    fn(bytearray_fspath, *extra_args)
+
+                if allow_fd:
+                    result = fn(fd, *extra_args)  # should not fail
+                    if cleanup_fn is not None:
+                        cleanup_fn(result)
+                else:
+                    with self.assertRaisesRegex(
+                            TypeError,
+                            'os.PathLike'):
+                        fn(fd, *extra_args)
+
+
 @unittest.skipUnless(hasattr(os, 'get_blocking'),
                      'needs os.get_blocking() and os.set_blocking()')
 class BlockingTests(unittest.TestCase):
diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py
--- a/Lib/test/test_posix.py
+++ b/Lib/test/test_posix.py
@@ -397,7 +397,7 @@
             self.assertTrue(posix.stat(fp.fileno()))
 
             self.assertRaisesRegex(TypeError,
-                    'should be string, bytes or integer, not',
+                    'should be string, bytes, os.PathLike or integer, not',
                     posix.stat, float(fp.fileno()))
         finally:
             fp.close()
@@ -409,16 +409,16 @@
         self.assertTrue(posix.stat(os.fsencode(support.TESTFN)))
 
         self.assertWarnsRegex(DeprecationWarning,
-                'should be string, bytes or integer, not',
+                'should be string, bytes, os.PathLike or integer, not',
                 posix.stat, bytearray(os.fsencode(support.TESTFN)))
         self.assertRaisesRegex(TypeError,
-                'should be string, bytes or integer, not',
+                'should be string, bytes, os.PathLike or integer, not',
                 posix.stat, None)
         self.assertRaisesRegex(TypeError,
-                'should be string, bytes or integer, not',
+                'should be string, bytes, os.PathLike or integer, not',
                 posix.stat, list(support.TESTFN))
         self.assertRaisesRegex(TypeError,
-                'should be string, bytes or integer, not',
+                'should be string, bytes, os.PathLike or integer, not',
                 posix.stat, list(os.fsencode(support.TESTFN)))
 
     @unittest.skipUnless(hasattr(posix, 'mkfifo'), "don't have mkfifo()")
diff --git a/Lib/test/test_posixpath.py b/Lib/test/test_posixpath.py
--- a/Lib/test/test_posixpath.py
+++ b/Lib/test/test_posixpath.py
@@ -596,5 +596,85 @@
     attributes = ['relpath', 'samefile', 'sameopenfile', 'samestat']
 
 
+class PathLikeTests(unittest.TestCase):
+
+    path = posixpath
+
+    class PathLike:
+        def __init__(self, path=''):
+            self.path = path
+        def __fspath__(self):
+            if isinstance(self.path, BaseException):
+                raise self.path
+            else:
+                return self.path
+
+    def setUp(self):
+        self.file_name = support.TESTFN.lower()
+        self.file_path = self.PathLike(support.TESTFN)
+        self.addCleanup(support.unlink, self.file_name)
+        with open(self.file_name, 'xb', 0) as file:
+            file.write(b"test_posixpath.PathLikeTests")
+
+    def assertPathEqual(self, func):
+        self.assertEqual(func(self.file_path), func(self.file_name))
+
+    def test_path_normcase(self):
+        self.assertPathEqual(self.path.normcase)
+
+    def test_path_isabs(self):
+        self.assertPathEqual(self.path.isabs)
+
+    def test_path_join(self):
+        self.assertEqual(self.path.join('a', self.PathLike('b'), 'c'),
+                         self.path.join('a', 'b', 'c'))
+
+    def test_path_split(self):
+        self.assertPathEqual(self.path.split)
+
+    def test_path_splitext(self):
+        self.assertPathEqual(self.path.splitext)
+
+    def test_path_splitdrive(self):
+        self.assertPathEqual(self.path.splitdrive)
+
+    def test_path_basename(self):
+        self.assertPathEqual(self.path.basename)
+
+    def test_path_dirname(self):
+        self.assertPathEqual(self.path.dirname)
+
+    def test_path_islink(self):
+        self.assertPathEqual(self.path.islink)
+
+    def test_path_lexists(self):
+        self.assertPathEqual(self.path.lexists)
+
+    def test_path_ismount(self):
+        self.assertPathEqual(self.path.ismount)
+
+    def test_path_expanduser(self):
+        self.assertPathEqual(self.path.expanduser)
+
+    def test_path_expandvars(self):
+        self.assertPathEqual(self.path.expandvars)
+
+    def test_path_normpath(self):
+        self.assertPathEqual(self.path.normpath)
+
+    def test_path_abspath(self):
+        self.assertPathEqual(self.path.abspath)
+
+    def test_path_realpath(self):
+        self.assertPathEqual(self.path.realpath)
+
+    def test_path_relpath(self):
+        self.assertPathEqual(self.path.relpath)
+
+    def test_path_commonpath(self):
+        common_path = self.path.commonpath([self.file_path, self.file_name])
+        self.assertEqual(common_path, self.file_name)
+
+
 if __name__=="__main__":
     unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -142,7 +142,10 @@
 Library
 -------
 
-- Issue 27598: Add Collections to collections.abc.
+- Issue #26027, #27524: Add PEP 519/__fspath__() support to the os and os.path
+  modules. Includes code from Jelle Zijlstra.
+
+- Issue #27598: Add Collections to collections.abc.
   Patch by Ivan Levkivskyi, docs by Neil Girdhar.
 
 - Issue #25958: Support "anti-registration" of special methods from
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -834,8 +834,11 @@
 path_converter(PyObject *o, void *p)
 {
     path_t *path = (path_t *)p;
-    PyObject *bytes;
+    PyObject *bytes, *to_cleanup = NULL;
     Py_ssize_t length;
+    int is_index, is_buffer, is_bytes, is_unicode;
+    /* Default to failure, forcing explicit signaling of succcess. */
+    int ret = 0;
     const char *narrow;
 
 #define FORMAT_EXCEPTION(exc, fmt) \
@@ -850,7 +853,7 @@
         return 1;
     }
 
-    /* ensure it's always safe to call path_cleanup() */
+    /* Ensure it's always safe to call path_cleanup(). */
     path->cleanup = NULL;
 
     if ((o == Py_None) && path->nullable) {
@@ -862,21 +865,54 @@
         return 1;
     }
 
-    if (PyUnicode_Check(o)) {
+    /* Only call this here so that we don't treat the return value of
+       os.fspath() as an fd or buffer. */
+    is_index = path->allow_fd && PyIndex_Check(o);
+    is_buffer = PyObject_CheckBuffer(o);
+    is_bytes = PyBytes_Check(o);
+    is_unicode = PyUnicode_Check(o);
+
+    if (!is_index && !is_buffer && !is_unicode && !is_bytes) {
+        /* Inline PyOS_FSPath() for better error messages. */
+        _Py_IDENTIFIER(__fspath__);
+        PyObject *func = NULL;
+
+        func = _PyObject_LookupSpecial(o, &PyId___fspath__);
+        if (NULL == func) {
+            goto error_exit;
+        }
+
+        o = to_cleanup = PyObject_CallFunctionObjArgs(func, NULL);
+        Py_DECREF(func);
+        if (NULL == o) {
+            goto error_exit;
+        }
+        else if (PyUnicode_Check(o)) {
+            is_unicode = 1;
+        }
+        else if (PyBytes_Check(o)) {
+            is_bytes = 1;
+        }
+        else {
+            goto error_exit;
+        }
+    }
+
+    if (is_unicode) {
 #ifdef MS_WINDOWS
         const wchar_t *wide;
 
         wide = PyUnicode_AsUnicodeAndSize(o, &length);
         if (!wide) {
-            return 0;
+            goto exit;
         }
         if (length > 32767) {
             FORMAT_EXCEPTION(PyExc_ValueError, "%s too long for Windows");
-            return 0;
+            goto exit;
         }
         if (wcslen(wide) != length) {
             FORMAT_EXCEPTION(PyExc_ValueError, "embedded null character in %s");
-            return 0;
+            goto exit;
         }
 
         path->wide = wide;
@@ -884,66 +920,71 @@
         path->length = length;
         path->object = o;
         path->fd = -1;
-        return 1;
+        ret = 1;
+        goto exit;
 #else
         if (!PyUnicode_FSConverter(o, &bytes)) {
-            return 0;
-        }
-#endif
-    }
-    else if (PyBytes_Check(o)) {
+            goto exit;
+        }
+#endif
+    }
+    else if (is_bytes) {
 #ifdef MS_WINDOWS
         if (win32_warn_bytes_api()) {
-            return 0;
+            goto exit;
         }
 #endif
         bytes = o;
         Py_INCREF(bytes);
     }
-    else if (PyObject_CheckBuffer(o)) {
+    else if (is_buffer) {
         if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1,
             "%s%s%s should be %s, not %.200s",
             path->function_name ? path->function_name : "",
             path->function_name ? ": "                : "",
             path->argument_name ? path->argument_name : "path",
-            path->allow_fd && path->nullable ? "string, bytes, integer or None" :
-            path->allow_fd ? "string, bytes or integer" :
-            path->nullable ? "string, bytes or None" :
-                             "string or bytes",
+            path->allow_fd && path->nullable ? "string, bytes, os.PathLike, "
+                                               "integer or None" :
+            path->allow_fd ? "string, bytes, os.PathLike or integer" :
+            path->nullable ? "string, bytes, os.PathLike or None" :
+                             "string, bytes or os.PathLike",
             Py_TYPE(o)->tp_name)) {
-            return 0;
+            goto exit;
         }
 #ifdef MS_WINDOWS
         if (win32_warn_bytes_api()) {
-            return 0;
+            goto exit;
         }
 #endif
         bytes = PyBytes_FromObject(o);
         if (!bytes) {
-            return 0;
+            goto exit;
         }
     }
     else if (path->allow_fd && PyIndex_Check(o)) {
         if (!_fd_converter(o, &path->fd)) {
-            return 0;
+            goto exit;
         }
         path->wide = NULL;
         path->narrow = NULL;
         path->length = 0;
         path->object = o;
-        return 1;
+        ret = 1;
+        goto exit;
     }
     else {
+ error_exit:
         PyErr_Format(PyExc_TypeError, "%s%s%s should be %s, not %.200s",
             path->function_name ? path->function_name : "",
             path->function_name ? ": "                : "",
             path->argument_name ? path->argument_name : "path",
-            path->allow_fd && path->nullable ? "string, bytes, integer or None" :
-            path->allow_fd ? "string, bytes or integer" :
-            path->nullable ? "string, bytes or None" :
-                             "string or bytes",
+            path->allow_fd && path->nullable ? "string, bytes, os.PathLike, "
+                                               "integer or None" :
+            path->allow_fd ? "string, bytes, os.PathLike or integer" :
+            path->nullable ? "string, bytes, os.PathLike or None" :
+                             "string, bytes or os.PathLike",
             Py_TYPE(o)->tp_name);
-        return 0;
+        goto exit;
     }
 
     length = PyBytes_GET_SIZE(bytes);
@@ -951,7 +992,7 @@
     if (length > MAX_PATH-1) {
         FORMAT_EXCEPTION(PyExc_ValueError, "%s too long for Windows");
         Py_DECREF(bytes);
-        return 0;
+        goto exit;
     }
 #endif
 
@@ -959,7 +1000,7 @@
     if ((size_t)length != strlen(narrow)) {
         FORMAT_EXCEPTION(PyExc_ValueError, "embedded null character in %s");
         Py_DECREF(bytes);
-        return 0;
+        goto exit;
     }
 
     path->wide = NULL;
@@ -969,12 +1010,15 @@
     path->fd = -1;
     if (bytes == o) {
         Py_DECREF(bytes);
-        return 1;
+        ret = 1;
     }
     else {
         path->cleanup = bytes;
-        return Py_CLEANUP_SUPPORTED;
-    }
+        ret = Py_CLEANUP_SUPPORTED;
+    }
+ exit:
+    Py_XDECREF(to_cleanup);
+    return ret;
 }
 
 static void
@@ -12329,6 +12373,8 @@
 PyObject *
 PyOS_FSPath(PyObject *path)
 {
+    /* For error message reasons, this function is manually inlined in
+       path_converter(). */
     _Py_IDENTIFIER(__fspath__);
     PyObject *func = NULL;
     PyObject *path_repr = NULL;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Aug 26 17:45:23 2016
From: python-checkins at python.org (brett.cannon)
Date: Fri, 26 Aug 2016 21:45:23 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_a_What=27s_New_entry_f?=
 =?utf-8?q?or_PEP_519?=
Message-ID: <20160826214521.113175.3938.9D71A32B@psf.io>

https://hg.python.org/cpython/rev/95361959d451
changeset:   102921:95361959d451
user:        Brett Cannon 
date:        Fri Aug 26 14:45:15 2016 -0700
summary:
  Add a What's New entry for PEP 519

files:
  Doc/whatsnew/3.6.rst |  67 ++++++++++++++++++++++++++++++++
  1 files changed, 67 insertions(+), 0 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -66,6 +66,10 @@
 
 * PEP 498: :ref:`Formatted string literals `
 
+Standard library improvements:
+
+* PEP 519: :ref:`Adding a file system path protocol `
+
 Windows improvements:
 
 * The ``py.exe`` launcher, when used interactively, no longer prefers
@@ -92,6 +96,69 @@
 New Features
 ============
 
+.. _pep-519:
+
+PEP 519: Adding a file system path protocol
+===========================================
+
+File system paths have historically been represented as :class:`str`
+or :class:`bytes` objects. This has led to people who write code which
+operate on file system paths to assume that such objects are only one
+of those two types (an :class:`int` representing a file descriptor
+does not count as that is not a file path). Unfortunately that
+assumption prevents alternative object representations of file system
+paths like :mod:`pathlib` from working with pre-existing code,
+including Python's standard library.
+
+To fix this situation, a new interface represented by
+:class:`os.PathLike` has been defined. By implementing the
+:meth:`~os.PathLike.__fspath__` method, an object signals that it
+represents a path. An object can then provide a low-level
+representation of a file system path as a :class:`str` or
+:class:`bytes` object. This means an object is considered
+:term:`path-like ` if it implements
+:class:`os.PathLike` or is a :class:`str` or :class:`bytes` object
+which represents a file system path. Code can use :func:`os.fspath`,
+:func:`os.fsdecode`, or :func:`os.fsencode` to explicitly get a
+:class:`str` and/or :class:`bytes` representation of a path-like
+object.
+
+The built-in :func:`open` function has been updated to accept
+:class:`os.PathLike` objects as have all relevant functions in the
+:mod:`os` and :mod:`os.path` modules. The :class:`os.DirEntry` class
+and relevant classes in :mod:`pathlib` have also been updated to
+implement :class:`os.PathLike`. The hope is that updating the
+fundamental functions for operating on file system paths will lead
+to third-party code to implicitly support all
+:term:`path-like objects ` without any code changes
+or at least very minimal ones (e.g. calling :func:`os.fspath` at the
+beginning of code before operating on a path-like object).
+
+Here are some examples of how the new interface allows for
+:class:`pathlib.Path` to be used more easily and transparently with
+pre-existing code::
+
+  >>> import pathlib
+  >>> with open(pathlib.Path("README")) as f:
+  ...     contents = f.read()
+  ...
+  >>> import os.path
+  >>> os.path.splitext(pathlib.Path("some_file.txt"))
+  ('some_file', '.txt')
+  >>> os.path.join("/a/b", pathlib.Path("c"))
+  '/a/b/c'
+  >>> import os
+  >>> os.fspath(pathlib.Path("some_file.txt"))
+  'some_file.txt'
+
+(Implemented by Brett Cannon, Ethan Furman, Dusty Phillips, and Jelle Zijlstra.)
+
+.. seealso::
+
+    :pep:`519` - Adding a file system path protocol
+       PEP written by Brett Cannon and Koos Zevenhoven.
+
+
 .. _whatsnew-fstrings:
 
 PEP 498: Formatted string literals

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Aug 26 22:30:16 2016
From: python-checkins at python.org (brett.cannon)
Date: Sat, 27 Aug 2016 02:30:16 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326027=3A_Don=27t_?=
 =?utf-8?q?test_for_bytearray_in_path=5Ft_as_that=27s_now?=
Message-ID: <20160827023016.27094.9061.FAFC556C@psf.io>

https://hg.python.org/cpython/rev/32b93ba32aa0
changeset:   102922:32b93ba32aa0
user:        Brett Cannon 
date:        Fri Aug 26 19:30:11 2016 -0700
summary:
  Issue #26027: Don't test for bytearray in path_t as that's now
deprecated.

files:
  Lib/test/test_os.py |  9 ++-------
  1 files changed, 2 insertions(+), 7 deletions(-)


diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
--- a/Lib/test/test_os.py
+++ b/Lib/test/test_os.py
@@ -2842,7 +2842,6 @@
 
         str_filename = support.TESTFN
         bytes_filename = support.TESTFN.encode('ascii')
-        bytearray_filename = bytearray(bytes_filename)
         fd = os.open(PathLike(str_filename), os.O_WRONLY|os.O_CREAT)
         self.addCleanup(os.close, fd)
         self.addCleanup(support.unlink, support.TESTFN)
@@ -2850,7 +2849,6 @@
         int_fspath = PathLike(fd)
         str_fspath = PathLike(str_filename)
         bytes_fspath = PathLike(bytes_filename)
-        bytearray_fspath = PathLike(bytearray_filename)
 
         for name, allow_fd, extra_args, cleanup_fn in self.functions:
             with self.subTest(name=name):
@@ -2859,8 +2857,8 @@
                 except AttributeError:
                     continue
 
-                for path in (str_filename, bytes_filename, bytearray_filename,
-                             str_fspath, bytes_fspath):
+                for path in (str_filename, bytes_filename, str_fspath,
+                             bytes_fspath):
                     with self.subTest(name=name, path=path):
                         result = fn(path, *extra_args)
                         if cleanup_fn is not None:
@@ -2869,9 +2867,6 @@
                 with self.assertRaisesRegex(
                         TypeError, 'should be string, bytes'):
                     fn(int_fspath, *extra_args)
-                with self.assertRaisesRegex(
-                        TypeError, 'should be string, bytes'):
-                    fn(bytearray_fspath, *extra_args)
 
                 if allow_fd:
                     result = fn(fd, *extra_args)  # should not fail

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Aug 26 22:42:38 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 27 Aug 2016 02:42:38 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2312319=3A_Always_s?=
 =?utf-8?q?end_file_request_bodies_using_chunked_encoding?=
Message-ID: <20160827024238.17947.49518.97E3ECCF@psf.io>

https://hg.python.org/cpython/rev/216f5451c35a
changeset:   102923:216f5451c35a
user:        Martin Panter 
date:        Sat Aug 27 01:39:26 2016 +0000
summary:
  Issue #12319: Always send file request bodies using chunked encoding

The previous attempt to determine the file?s Content-Length gave a false
positive for pipes on Windows.

Also, drop the special case for sending zero-length iterable bodies.

files:
  Doc/library/http.client.rst    |  31 +++++------
  Doc/library/urllib.request.rst |  15 ++---
  Doc/whatsnew/3.6.rst           |  11 +++-
  Lib/http/client.py             |  31 +++---------
  Lib/test/test_httplib.py       |  27 +++++++---
  Lib/test/test_urllib2.py       |  56 +++++++++++++--------
  Misc/NEWS                      |   7 +-
  7 files changed, 96 insertions(+), 82 deletions(-)


diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst
--- a/Doc/library/http.client.rst
+++ b/Doc/library/http.client.rst
@@ -240,17 +240,17 @@
    The *headers* argument should be a mapping of extra HTTP headers to send
    with the request.
 
-   If *headers* contains neither Content-Length nor Transfer-Encoding, a
-   Content-Length header will be added automatically if possible.  If
+   If *headers* contains neither Content-Length nor Transfer-Encoding,
+   but there is a request body, one of those
+   header fields will be added automatically.  If
    *body* is ``None``, the Content-Length header is set to ``0`` for
    methods that expect a body (``PUT``, ``POST``, and ``PATCH``).  If
-   *body* is a string or bytes-like object, the Content-Length header is
-   set to its length.  If *body* is a binary :term:`file object`
-   supporting :meth:`~io.IOBase.seek`, this will be used to determine
-   its size.  Otherwise, the Content-Length header is not added
-   automatically.  In cases where determining the Content-Length up
-   front is not possible, the body will be chunk-encoded and the
-   Transfer-Encoding header will automatically be set.
+   *body* is a string or a bytes-like object that is not also a
+   :term:`file `, the Content-Length header is
+   set to its length.  Any other type of *body* (files
+   and iterables in general) will be chunk-encoded, and the
+   Transfer-Encoding header will automatically be set instead of
+   Content-Length.
 
    The *encode_chunked* argument is only relevant if Transfer-Encoding is
    specified in *headers*.  If *encode_chunked* is ``False``, the
@@ -260,19 +260,18 @@
    .. note::
       Chunked transfer encoding has been added to the HTTP protocol
       version 1.1.  Unless the HTTP server is known to handle HTTP 1.1,
-      the caller must either specify the Content-Length or must use a
-      body representation whose length can be determined automatically.
+      the caller must either specify the Content-Length, or must pass a
+      :class:`str` or bytes-like object that is not also a file as the
+      body representation.
 
    .. versionadded:: 3.2
       *body* can now be an iterable.
 
    .. versionchanged:: 3.6
       If neither Content-Length nor Transfer-Encoding are set in
-      *headers* and Content-Length cannot be determined, *body* will now
-      be automatically chunk-encoded.  The *encode_chunked* argument
-      was added.
-      The Content-Length for binary file objects is determined with seek.
-      No attempt is made to determine the Content-Length for text file
+      *headers*, file and iterable *body* objects are now chunk-encoded.
+      The *encode_chunked* argument was added.
+      No attempt is made to determine the Content-Length for file
       objects.
 
 .. method:: HTTPConnection.getresponse()
diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst
--- a/Doc/library/urllib.request.rst
+++ b/Doc/library/urllib.request.rst
@@ -187,12 +187,11 @@
    server, or ``None`` if no such data is needed.  Currently HTTP
    requests are the only ones that use *data*.  The supported object
    types include bytes, file-like objects, and iterables.  If no
-   ``Content-Length`` header has been provided, :class:`HTTPHandler` will
-   try to determine the length of *data* and set this header accordingly.
-   If this fails, ``Transfer-Encoding: chunked`` as specified in
-   :rfc:`7230`, Section 3.3.1 will be used to send the data.  See
-   :meth:`http.client.HTTPConnection.request` for details on the
-   supported object types and on how the content length is determined.
+   ``Content-Length`` nor ``Transfer-Encoding`` header field
+   has been provided, :class:`HTTPHandler` will set these headers according
+   to the type of *data*.  ``Content-Length`` will be used to send
+   bytes objects, while ``Transfer-Encoding: chunked`` as specified in
+   :rfc:`7230`, Section 3.3.1 will be used to send files and other iterables.
 
    For an HTTP POST request method, *data* should be a buffer in the
    standard :mimetype:`application/x-www-form-urlencoded` format.  The
@@ -256,8 +255,8 @@
 
    .. versionchanged:: 3.6
       Do not raise an error if the ``Content-Length`` has not been
-      provided and could not be determined.  Fall back to use chunked
-      transfer encoding instead.
+      provided and *data* is neither ``None`` nor a bytes object.
+      Fall back to use chunked transfer encoding instead.
 
 .. class:: OpenerDirector()
 
diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -579,8 +579,8 @@
 urllib.request
 --------------
 
-If a HTTP request has a non-empty body but no Content-Length header
-and the content length cannot be determined up front, rather than
+If a HTTP request has a file or iterable body (other than a
+bytes object) but no Content-Length header, rather than
 throwing an error, :class:`~urllib.request.AbstractHTTPHandler` now
 falls back to use chunked transfer encoding.
 (Contributed by Demian Brecht and Rolf Krahl in :issue:`12319`.)
@@ -935,6 +935,13 @@
   This behavior has also been backported to earlier Python versions
   by Setuptools 26.0.0.
 
+* In the :mod:`urllib.request` module and the
+  :meth:`http.client.HTTPConnection.request` method, if no Content-Length
+  header field has been specified and the request body is a file object,
+  it is now sent with HTTP 1.1 chunked encoding. If a file object has to
+  be sent to a HTTP 1.0 server, the Content-Length value now has to be
+  specified by the caller. See :issue:`12319`.
+
 Changes in the C API
 --------------------
 
diff --git a/Lib/http/client.py b/Lib/http/client.py
--- a/Lib/http/client.py
+++ b/Lib/http/client.py
@@ -805,35 +805,21 @@
     def _get_content_length(body, method):
         """Get the content-length based on the body.
 
-        If the body is "empty", we set Content-Length: 0 for methods
-        that expect a body (RFC 7230, Section 3.3.2). If the body is
-        set for other methods, we set the header provided we can
-        figure out what the length is.
+        If the body is None, we set Content-Length: 0 for methods that expect
+        a body (RFC 7230, Section 3.3.2). We also set the Content-Length for
+        any method if the body is a str or bytes-like object and not a file.
         """
-        if not body:
+        if body is None:
             # do an explicit check for not None here to distinguish
             # between unset and set but empty
-            if method.upper() in _METHODS_EXPECTING_BODY or body is not None:
+            if method.upper() in _METHODS_EXPECTING_BODY:
                 return 0
             else:
                 return None
 
         if hasattr(body, 'read'):
             # file-like object.
-            if HTTPConnection._is_textIO(body):
-                # text streams are unpredictable because it depends on
-                # character encoding and line ending translation.
-                return None
-            else:
-                # Is it seekable?
-                try:
-                    curpos = body.tell()
-                    sz = body.seek(0, io.SEEK_END)
-                except (TypeError, AttributeError, OSError):
-                    return None
-                else:
-                    body.seek(curpos)
-                    return sz - curpos
+            return None
 
         try:
             # does it implement the buffer protocol (bytes, bytearray, array)?
@@ -1266,8 +1252,7 @@
         # the caller passes encode_chunked=True or the following
         # conditions hold:
         # 1. content-length has not been explicitly set
-        # 2. the length of the body cannot be determined
-        #    (e.g. it is a generator or unseekable file)
+        # 2. the body is a file or iterable, but not a str or bytes-like
         # 3. Transfer-Encoding has NOT been explicitly set by the caller
 
         if 'content-length' not in header_names:
@@ -1280,7 +1265,7 @@
                 encode_chunked = False
                 content_length = self._get_content_length(body, method)
                 if content_length is None:
-                    if body:
+                    if body is not None:
                         if self.debuglevel > 0:
                             print('Unable to determine size of %r' % body)
                         encode_chunked = True
diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py
--- a/Lib/test/test_httplib.py
+++ b/Lib/test/test_httplib.py
@@ -381,6 +381,16 @@
             # same request
             self.assertNotIn('content-length', [k.lower() for k in headers])
 
+    def test_empty_body(self):
+        # Zero-length iterable should be treated like any other iterable
+        conn = client.HTTPConnection('example.com')
+        conn.sock = FakeSocket(b'')
+        conn.request('POST', '/', ())
+        _, headers, body = self._parse_request(conn.sock.data)
+        self.assertEqual(headers['Transfer-Encoding'], 'chunked')
+        self.assertNotIn('content-length', [k.lower() for k in headers])
+        self.assertEqual(body, b"0\r\n\r\n")
+
     def _make_body(self, empty_lines=False):
         lines = self.expected_body.split(b' ')
         for idx, line in enumerate(lines):
@@ -652,7 +662,9 @@
 
     def test_send_file(self):
         expected = (b'GET /foo HTTP/1.1\r\nHost: example.com\r\n'
-                    b'Accept-Encoding: identity\r\nContent-Length:')
+                    b'Accept-Encoding: identity\r\n'
+                    b'Transfer-Encoding: chunked\r\n'
+                    b'\r\n')
 
         with open(__file__, 'rb') as body:
             conn = client.HTTPConnection('example.com')
@@ -1717,7 +1729,7 @@
         self.assertEqual("5", message.get("content-length"))
         self.assertEqual(b'body\xc1', f.read())
 
-    def test_file_body(self):
+    def test_text_file_body(self):
         self.addCleanup(support.unlink, support.TESTFN)
         with open(support.TESTFN, "w") as f:
             f.write("body")
@@ -1726,10 +1738,8 @@
             message, f = self.get_headers_and_fp()
             self.assertEqual("text/plain", message.get_content_type())
             self.assertIsNone(message.get_charset())
-            # Note that the length of text files is unpredictable
-            # because it depends on character encoding and line ending
-            # translation.  No content-length will be set, the body
-            # will be sent using chunked transfer encoding.
+            # No content-length will be determined for files; the body
+            # will be sent using chunked transfer encoding instead.
             self.assertIsNone(message.get("content-length"))
             self.assertEqual("chunked", message.get("transfer-encoding"))
             self.assertEqual(b'4\r\nbody\r\n0\r\n\r\n', f.read())
@@ -1743,8 +1753,9 @@
             message, f = self.get_headers_and_fp()
             self.assertEqual("text/plain", message.get_content_type())
             self.assertIsNone(message.get_charset())
-            self.assertEqual("5", message.get("content-length"))
-            self.assertEqual(b'body\xc1', f.read())
+            self.assertEqual("chunked", message.get("Transfer-Encoding"))
+            self.assertNotIn("Content-Length", message)
+            self.assertEqual(b'5\r\nbody\xc1\r\n0\r\n\r\n', f.read())
 
 
 class HTTPResponseTest(TestCase):
diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py
--- a/Lib/test/test_urllib2.py
+++ b/Lib/test/test_urllib2.py
@@ -913,40 +913,50 @@
             self.assertEqual(req.unredirected_hdrs["Spam"], "foo")
 
     def test_http_body_file(self):
-        # A regular file - Content Length is calculated unless already set.
+        # A regular file - chunked encoding is used unless Content Length is
+        # already set.
 
         h = urllib.request.AbstractHTTPHandler()
         o = h.parent = MockOpener()
 
         file_obj = tempfile.NamedTemporaryFile(mode='w+b', delete=False)
         file_path = file_obj.name
-        file_obj.write(b"Something\nSomething\nSomething\n")
         file_obj.close()
+        self.addCleanup(os.unlink, file_path)
 
-        for headers in {}, {"Content-Length": 30}:
-            with open(file_path, "rb") as f:
-                req = Request("http://example.com/", f, headers)
-                newreq = h.do_request_(req)
-                self.assertEqual(int(newreq.get_header('Content-length')), 30)
+        with open(file_path, "rb") as f:
+            req = Request("http://example.com/", f, {})
+            newreq = h.do_request_(req)
+            te = newreq.get_header('Transfer-encoding')
+            self.assertEqual(te, "chunked")
+            self.assertFalse(newreq.has_header('Content-length'))
 
-        os.unlink(file_path)
+        with open(file_path, "rb") as f:
+            req = Request("http://example.com/", f, {"Content-Length": 30})
+            newreq = h.do_request_(req)
+            self.assertEqual(int(newreq.get_header('Content-length')), 30)
+            self.assertFalse(newreq.has_header("Transfer-encoding"))
 
     def test_http_body_fileobj(self):
-        # A file object - Content Length is calculated unless already set.
+        # A file object - chunked encoding is used
+        # unless Content Length is already set.
         # (Note that there are some subtle differences to a regular
         # file, that is why we are testing both cases.)
 
         h = urllib.request.AbstractHTTPHandler()
         o = h.parent = MockOpener()
+        file_obj = io.BytesIO()
 
-        file_obj = io.BytesIO()
-        file_obj.write(b"Something\nSomething\nSomething\n")
+        req = Request("http://example.com/", file_obj, {})
+        newreq = h.do_request_(req)
+        self.assertEqual(newreq.get_header('Transfer-encoding'), 'chunked')
+        self.assertFalse(newreq.has_header('Content-length'))
 
-        for headers in {}, {"Content-Length": 30}:
-            file_obj.seek(0)
-            req = Request("http://example.com/", file_obj, headers)
-            newreq = h.do_request_(req)
-            self.assertEqual(int(newreq.get_header('Content-length')), 30)
+        headers = {"Content-Length": 30}
+        req = Request("http://example.com/", file_obj, headers)
+        newreq = h.do_request_(req)
+        self.assertEqual(int(newreq.get_header('Content-length')), 30)
+        self.assertFalse(newreq.has_header("Transfer-encoding"))
 
         file_obj.close()
 
@@ -959,9 +969,7 @@
         h = urllib.request.AbstractHTTPHandler()
         o = h.parent = MockOpener()
 
-        cmd = [sys.executable, "-c",
-               r"import sys; "
-               r"sys.stdout.buffer.write(b'Something\nSomething\nSomething\n')"]
+        cmd = [sys.executable, "-c", r"pass"]
         for headers in {}, {"Content-Length": 30}:
             with subprocess.Popen(cmd, stdout=subprocess.PIPE) as proc:
                 req = Request("http://example.com/", proc.stdout, headers)
@@ -983,8 +991,6 @@
 
         def iterable_body():
             yield b"one"
-            yield b"two"
-            yield b"three"
 
         for headers in {}, {"Content-Length": 11}:
             req = Request("http://example.com/", iterable_body(), headers)
@@ -996,6 +1002,14 @@
             else:
                 self.assertEqual(int(newreq.get_header('Content-length')), 11)
 
+    def test_http_body_empty_seq(self):
+        # Zero-length iterable body should be treated like any other iterable
+        h = urllib.request.AbstractHTTPHandler()
+        h.parent = MockOpener()
+        req = h.do_request_(Request("http://example.com/", ()))
+        self.assertEqual(req.get_header("Transfer-encoding"), "chunked")
+        self.assertFalse(req.has_header("Content-length"))
+
     def test_http_body_array(self):
         # array.array Iterable - Content Length is calculated
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -52,10 +52,9 @@
 - Issue #12319: Chunked transfer encoding support added to
   http.client.HTTPConnection requests.  The
   urllib.request.AbstractHTTPHandler class does not enforce a Content-Length
-  header any more.  If a HTTP request has a non-empty body, but no
-  Content-Length header, and the content length cannot be determined
-  up front, rather than throwing an error, the library now falls back
-  to use chunked transfer encoding.
+  header any more.  If a HTTP request has a file or iterable body, but no
+  Content-Length header, the library now falls back to use chunked transfer-
+  encoding.
 
 - A new version of typing.py from https://github.com/python/typing:
   - Collection (only for 3.6) (Issue #27598)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Aug 27 00:10:52 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 27 Aug 2016 04:10:52 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE5ODg0?=
 =?utf-8?q?=3A_Avoid_spurious_output_on_OS_X_with_Gnu_Readline?=
Message-ID: <20160827041052.38965.84186.C6E5DE3A@psf.io>

https://hg.python.org/cpython/rev/55ec5fdc3099
changeset:   102924:55ec5fdc3099
branch:      2.7
parent:      102919:030e100f048a
user:        Martin Panter 
date:        Sat Aug 27 02:54:43 2016 +0000
summary:
  Issue #19884: Avoid spurious output on OS X with Gnu Readline

Also adjust the test condition, because enable-meta-key was only added in
6.1, not 6.0.

files:
  Lib/test/test_readline.py |   2 +-
  Misc/NEWS                 |   2 ++
  Modules/readline.c        |  25 ++++++++++++++-----------
  3 files changed, 17 insertions(+), 12 deletions(-)


diff --git a/Lib/test/test_readline.py b/Lib/test/test_readline.py
--- a/Lib/test/test_readline.py
+++ b/Lib/test/test_readline.py
@@ -44,7 +44,7 @@
 
 class TestReadline(unittest.TestCase):
 
-    @unittest.skipIf(readline._READLINE_VERSION < 0x0600
+    @unittest.skipIf(readline._READLINE_VERSION < 0x0601
                      and "libedit" not in readline.__doc__,
                      "not supported in this library version")
     def test_init(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -33,6 +33,8 @@
 Library
 -------
 
+- Issue #19884: Avoid spurious output on OS X with Gnu Readline.
+
 - Issue #10513: Fix a regression in Connection.commit().  Statements should
   not be reset after a commit.
 
diff --git a/Modules/readline.c b/Modules/readline.c
--- a/Modules/readline.c
+++ b/Modules/readline.c
@@ -961,19 +961,22 @@
     begidx = PyInt_FromLong(0L);
     endidx = PyInt_FromLong(0L);
 
-#ifndef __APPLE__
-    if (!isatty(STDOUT_FILENO)) {
-        /* Issue #19884: stdout is not a terminal. Disable meta modifier
-           keys to not write the ANSI sequence "\033[1034h" into stdout. On
-           terminals supporting 8 bit characters like TERM=xterm-256color
-           (which is now the default Fedora since Fedora 18), the meta key is
-           used to enable support of 8 bit characters (ANSI sequence
-           "\033[1034h").
+#ifdef __APPLE__
+    if (!using_libedit_emulation)
+#endif
+    {
+        if (!isatty(STDOUT_FILENO)) {
+            /* Issue #19884: stdout is not a terminal. Disable meta modifier
+               keys to not write the ANSI sequence "\033[1034h" into stdout. On
+               terminals supporting 8 bit characters like TERM=xterm-256color
+               (which is now the default Fedora since Fedora 18), the meta key is
+               used to enable support of 8 bit characters (ANSI sequence
+               "\033[1034h").
 
-           With libedit, this call makes readline() crash. */
-        rl_variable_bind ("enable-meta-key", "off");
+               With libedit, this call makes readline() crash. */
+            rl_variable_bind ("enable-meta-key", "off");
+        }
     }
-#endif
 
     /* Initialize (allows .inputrc to override)
      *

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Aug 27 00:10:53 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 27 Aug 2016 04:10:53 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2325825=3A_Merge_=24=28LIBPL=29_fix_from_3=2E5?=
Message-ID: <20160827041053.20148.26070.E41D6C5E@psf.io>

https://hg.python.org/cpython/rev/5a05c0eeefc3
changeset:   102928:5a05c0eeefc3
parent:      102927:72e034afeb55
parent:      102926:ca1ddd365f5f
user:        Martin Panter 
date:        Sat Aug 27 04:07:54 2016 +0000
summary:
  Issue #25825: Merge $(LIBPL) fix from 3.5

files:
  Misc/NEWS    |  6 ++++++
  configure    |  2 +-
  configure.ac |  2 +-
  3 files changed, 8 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -89,6 +89,12 @@
 - Issue #27787: Call gc.collect() before checking each test for "dangling
   threads", since the dangling threads are weak references.
 
+Build
+-----
+
+- Issue #25825: Update references to the $(LIBPL) installation path on AIX.
+  This path was changed in 3.2a4.
+
 
 What's New in Python 3.6.0 alpha 4
 ==================================
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -9079,7 +9079,7 @@
 	case $ac_sys_system/$ac_sys_release in
 	AIX*)
 		BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:\$(srcdir)/Modules/python.exp"
-		LDSHARED="\$(BINLIBDEST)/config/ld_so_aix \$(CC) -bI:\$(BINLIBDEST)/config/python.exp"
+		LDSHARED="\$(LIBPL)/ld_so_aix \$(CC) -bI:\$(LIBPL)/python.exp"
 		;;
 	IRIX/5*) LDSHARED="ld -shared";;
 	IRIX*/6*) LDSHARED="ld ${SGI_ABI} -shared -all";;
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -2315,7 +2315,7 @@
 	case $ac_sys_system/$ac_sys_release in
 	AIX*)
 		BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:\$(srcdir)/Modules/python.exp"
-		LDSHARED="\$(BINLIBDEST)/config/ld_so_aix \$(CC) -bI:\$(BINLIBDEST)/config/python.exp"
+		LDSHARED="\$(LIBPL)/ld_so_aix \$(CC) -bI:\$(LIBPL)/python.exp"
 		;;
 	IRIX/5*) LDSHARED="ld -shared";;
 	IRIX*/6*) LDSHARED="ld ${SGI_ABI} -shared -all";;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Aug 27 00:10:52 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 27 Aug 2016 04:10:52 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzE5ODg0?=
 =?utf-8?q?=3A_Avoid_spurious_output_on_OS_X_with_Gnu_Readline?=
Message-ID: <20160827041052.20534.6272.29C7A3C3@psf.io>

https://hg.python.org/cpython/rev/782d9b5d2e90
changeset:   102925:782d9b5d2e90
branch:      3.5
parent:      102917:81f614dd8136
user:        Martin Panter 
date:        Sat Aug 27 03:23:11 2016 +0000
summary:
  Issue #19884: Avoid spurious output on OS X with Gnu Readline

Also adjust the test condition, because enable-meta-key was only added in
6.1, not 6.0.

files:
  Lib/test/test_readline.py |   2 +-
  Misc/NEWS                 |   2 ++
  Modules/readline.c        |  25 ++++++++++++++-----------
  3 files changed, 17 insertions(+), 12 deletions(-)


diff --git a/Lib/test/test_readline.py b/Lib/test/test_readline.py
--- a/Lib/test/test_readline.py
+++ b/Lib/test/test_readline.py
@@ -111,7 +111,7 @@
 
 class TestReadline(unittest.TestCase):
 
-    @unittest.skipIf(readline._READLINE_VERSION < 0x0600 and not is_editline,
+    @unittest.skipIf(readline._READLINE_VERSION < 0x0601 and not is_editline,
                      "not supported in this library version")
     def test_init(self):
         # Issue #19884: Ensure that the ANSI sequence "\033[1034h" is not
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -50,6 +50,8 @@
 Library
 -------
 
+- Issue #19884: Avoid spurious output on OS X with Gnu Readline.
+
 - Issue #10513: Fix a regression in Connection.commit().  Statements should
   not be reset after a commit.
 
diff --git a/Modules/readline.c b/Modules/readline.c
--- a/Modules/readline.c
+++ b/Modules/readline.c
@@ -1129,19 +1129,22 @@
     mod_state->begidx = PyLong_FromLong(0L);
     mod_state->endidx = PyLong_FromLong(0L);
 
-#ifndef __APPLE__
-    if (!isatty(STDOUT_FILENO)) {
-        /* Issue #19884: stdout is not a terminal. Disable meta modifier
-           keys to not write the ANSI sequence "\033[1034h" into stdout. On
-           terminals supporting 8 bit characters like TERM=xterm-256color
-           (which is now the default Fedora since Fedora 18), the meta key is
-           used to enable support of 8 bit characters (ANSI sequence
-           "\033[1034h").
+#ifdef __APPLE__
+    if (!using_libedit_emulation)
+#endif
+    {
+        if (!isatty(STDOUT_FILENO)) {
+            /* Issue #19884: stdout is not a terminal. Disable meta modifier
+               keys to not write the ANSI sequence "\033[1034h" into stdout. On
+               terminals supporting 8 bit characters like TERM=xterm-256color
+               (which is now the default Fedora since Fedora 18), the meta key is
+               used to enable support of 8 bit characters (ANSI sequence
+               "\033[1034h").
 
-           With libedit, this call makes readline() crash. */
-        rl_variable_bind ("enable-meta-key", "off");
+               With libedit, this call makes readline() crash. */
+            rl_variable_bind ("enable-meta-key", "off");
+        }
     }
-#endif
 
     /* Initialize (allows .inputrc to override)
      *

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Aug 27 00:10:53 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 27 Aug 2016 04:10:53 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2319884=3A_Merge_Readline_updates_from_3=2E5?=
Message-ID: <20160827041053.27050.37543.C608BA42@psf.io>

https://hg.python.org/cpython/rev/72e034afeb55
changeset:   102927:72e034afeb55
parent:      102923:216f5451c35a
parent:      102925:782d9b5d2e90
user:        Martin Panter 
date:        Sat Aug 27 04:03:26 2016 +0000
summary:
  Issue #19884: Merge Readline updates from 3.5

files:
  Lib/test/test_readline.py |   2 +-
  Misc/NEWS                 |   2 ++
  Modules/readline.c        |  25 ++++++++++++++-----------
  3 files changed, 17 insertions(+), 12 deletions(-)


diff --git a/Lib/test/test_readline.py b/Lib/test/test_readline.py
--- a/Lib/test/test_readline.py
+++ b/Lib/test/test_readline.py
@@ -111,7 +111,7 @@
 
 class TestReadline(unittest.TestCase):
 
-    @unittest.skipIf(readline._READLINE_VERSION < 0x0600 and not is_editline,
+    @unittest.skipIf(readline._READLINE_VERSION < 0x0601 and not is_editline,
                      "not supported in this library version")
     def test_init(self):
         # Issue #19884: Ensure that the ANSI sequence "\033[1034h" is not
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -46,6 +46,8 @@
 Library
 -------
 
+- Issue #19884: Avoid spurious output on OS X with Gnu Readline.
+
 - Issue #10513: Fix a regression in Connection.commit().  Statements should
   not be reset after a commit.
 
diff --git a/Modules/readline.c b/Modules/readline.c
--- a/Modules/readline.c
+++ b/Modules/readline.c
@@ -1146,19 +1146,22 @@
     mod_state->begidx = PyLong_FromLong(0L);
     mod_state->endidx = PyLong_FromLong(0L);
 
-#ifndef __APPLE__
-    if (!isatty(STDOUT_FILENO)) {
-        /* Issue #19884: stdout is not a terminal. Disable meta modifier
-           keys to not write the ANSI sequence "\033[1034h" into stdout. On
-           terminals supporting 8 bit characters like TERM=xterm-256color
-           (which is now the default Fedora since Fedora 18), the meta key is
-           used to enable support of 8 bit characters (ANSI sequence
-           "\033[1034h").
+#ifdef __APPLE__
+    if (!using_libedit_emulation)
+#endif
+    {
+        if (!isatty(STDOUT_FILENO)) {
+            /* Issue #19884: stdout is not a terminal. Disable meta modifier
+               keys to not write the ANSI sequence "\033[1034h" into stdout. On
+               terminals supporting 8 bit characters like TERM=xterm-256color
+               (which is now the default Fedora since Fedora 18), the meta key is
+               used to enable support of 8 bit characters (ANSI sequence
+               "\033[1034h").
 
-           With libedit, this call makes readline() crash. */
-        rl_variable_bind ("enable-meta-key", "off");
+               With libedit, this call makes readline() crash. */
+            rl_variable_bind ("enable-meta-key", "off");
+        }
     }
-#endif
 
     /* Initialize (allows .inputrc to override)
      *

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Aug 27 00:10:53 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 27 Aug 2016 04:10:53 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1ODI1?=
 =?utf-8?q?=3A_Fix_references_to_=24=28LIBPL=29_installation_path_on_AIX?=
Message-ID: <20160827041052.63797.3789.EE28E09E@psf.io>

https://hg.python.org/cpython/rev/ca1ddd365f5f
changeset:   102926:ca1ddd365f5f
branch:      3.5
user:        Martin Panter 
date:        Sat Aug 27 04:00:19 2016 +0000
summary:
  Issue #25825: Fix references to $(LIBPL) installation path on AIX

In Subversion r86731, the path was changed from $(BINLIBDEST)/config to
$(LIBDEST)/config-$(LDVERSION).

files:
  Misc/NEWS    |  6 ++++--
  configure    |  2 +-
  configure.ac |  2 +-
  3 files changed, 6 insertions(+), 4 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -220,8 +220,10 @@
 - Issue #27713: Suppress spurious build warnings when updating importlib's
   bootstrap files.  Patch by Xiang Zhang
 
-- Issue #25825: Correct the references to Modules/python.exp, which is
-  required on AIX.  The references were accidentally changed in 3.5.0a1.
+- Issue #25825: Correct the references to Modules/python.exp and ld_so_aix,
+  which are required on AIX.  This updates references to an installation path
+  that was changed in 3.2a4, and undoes changed references to the build tree
+  that were made in 3.5.0a1.
 
 - Issue #27453: CPP invocation in configure must use CPPFLAGS. Patch by
   Chi Hsuan Yen.
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -9040,7 +9040,7 @@
 	case $ac_sys_system/$ac_sys_release in
 	AIX*)
 		BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:\$(srcdir)/Modules/python.exp"
-		LDSHARED="\$(BINLIBDEST)/config/ld_so_aix \$(CC) -bI:\$(BINLIBDEST)/config/python.exp"
+		LDSHARED="\$(LIBPL)/ld_so_aix \$(CC) -bI:\$(LIBPL)/python.exp"
 		;;
 	IRIX/5*) LDSHARED="ld -shared";;
 	IRIX*/6*) LDSHARED="ld ${SGI_ABI} -shared -all";;
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -2284,7 +2284,7 @@
 	case $ac_sys_system/$ac_sys_release in
 	AIX*)
 		BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:\$(srcdir)/Modules/python.exp"
-		LDSHARED="\$(BINLIBDEST)/config/ld_so_aix \$(CC) -bI:\$(BINLIBDEST)/config/python.exp"
+		LDSHARED="\$(LIBPL)/ld_so_aix \$(CC) -bI:\$(LIBPL)/python.exp"
 		;;
 	IRIX/5*) LDSHARED="ld -shared";;
 	IRIX*/6*) LDSHARED="ld ${SGI_ABI} -shared -all";;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Aug 27 05:09:51 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 27 Aug 2016 09:09:51 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327506=3A_Support_?=
 =?utf-8?q?bytes/bytearray=2Etranslate=28=29_delete_as_keyword_argument?=
Message-ID: <20160827090951.113201.25135.950F7B5F@psf.io>

https://hg.python.org/cpython/rev/6ab1b54245d5
changeset:   102929:6ab1b54245d5
user:        Martin Panter 
date:        Sat Aug 27 08:35:02 2016 +0000
summary:
  Issue #27506: Support bytes/bytearray.translate() delete as keyword argument

Patch by Xiang Zhang.

files:
  Doc/library/stdtypes.rst           |   7 +-
  Lib/test/test_bytes.py             |  49 +++++++++++------
  Misc/NEWS                          |   3 +
  Objects/bytearrayobject.c          |  13 +--
  Objects/bytesobject.c              |  10 +--
  Objects/clinic/bytearrayobject.c.h |  37 +++++--------
  Objects/clinic/bytesobject.c.h     |  37 +++++--------
  7 files changed, 76 insertions(+), 80 deletions(-)


diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst
--- a/Doc/library/stdtypes.rst
+++ b/Doc/library/stdtypes.rst
@@ -2631,8 +2631,8 @@
    The prefix(es) to search for may be any :term:`bytes-like object`.
 
 
-.. method:: bytes.translate(table[, delete])
-            bytearray.translate(table[, delete])
+.. method:: bytes.translate(table, delete=b'')
+            bytearray.translate(table, delete=b'')
 
    Return a copy of the bytes or bytearray object where all bytes occurring in
    the optional argument *delete* are removed, and the remaining bytes have
@@ -2648,6 +2648,9 @@
       >>> b'read this short text'.translate(None, b'aeiou')
       b'rd ths shrt txt'
 
+   .. versionchanged:: 3.6
+      *delete* is now supported as a keyword argument.
+
 
 The following methods on bytes and bytearray objects have default behaviours
 that assume the use of ASCII compatible binary formats, but can still be used
diff --git a/Lib/test/test_bytes.py b/Lib/test/test_bytes.py
--- a/Lib/test/test_bytes.py
+++ b/Lib/test/test_bytes.py
@@ -689,6 +689,37 @@
         test.support.check_free_after_iterating(self, iter, self.type2test)
         test.support.check_free_after_iterating(self, reversed, self.type2test)
 
+    def test_translate(self):
+        b = self.type2test(b'hello')
+        rosetta = bytearray(range(256))
+        rosetta[ord('o')] = ord('e')
+
+        self.assertRaises(TypeError, b.translate)
+        self.assertRaises(TypeError, b.translate, None, None)
+        self.assertRaises(ValueError, b.translate, bytes(range(255)))
+
+        c = b.translate(rosetta, b'hello')
+        self.assertEqual(b, b'hello')
+        self.assertIsInstance(c, self.type2test)
+
+        c = b.translate(rosetta)
+        d = b.translate(rosetta, b'')
+        self.assertEqual(c, d)
+        self.assertEqual(c, b'helle')
+
+        c = b.translate(rosetta, b'l')
+        self.assertEqual(c, b'hee')
+        c = b.translate(None, b'e')
+        self.assertEqual(c, b'hllo')
+
+        # test delete as a keyword argument
+        c = b.translate(rosetta, delete=b'')
+        self.assertEqual(c, b'helle')
+        c = b.translate(rosetta, delete=b'l')
+        self.assertEqual(c, b'hee')
+        c = b.translate(None, delete=b'e')
+        self.assertEqual(c, b'hllo')
+
 
 class BytesTest(BaseBytesTest, unittest.TestCase):
     type2test = bytes
@@ -1449,24 +1480,6 @@
             self.assertRaises(SyntaxError, eval,
                               'b"%s"' % chr(c))
 
-    def test_translate(self):
-        b = b'hello'
-        ba = bytearray(b)
-        rosetta = bytearray(range(0, 256))
-        rosetta[ord('o')] = ord('e')
-        c = b.translate(rosetta, b'l')
-        self.assertEqual(b, b'hello')
-        self.assertEqual(c, b'hee')
-        c = ba.translate(rosetta, b'l')
-        self.assertEqual(ba, b'hello')
-        self.assertEqual(c, b'hee')
-        c = b.translate(None, b'e')
-        self.assertEqual(c, b'hllo')
-        c = ba.translate(None, b'e')
-        self.assertEqual(c, b'hllo')
-        self.assertRaises(TypeError, b.translate, None, None)
-        self.assertRaises(TypeError, ba.translate, None, None)
-
     def test_split_bytearray(self):
         self.assertEqual(b'a b'.split(memoryview(b' ')), [b'a', b'b'])
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #27506: Support passing the bytes/bytearray.translate() "delete"
+  argument by keyword.
+
 - Issue #27587: Fix another issue found by PVS-Studio: Null pointer check
   after use of 'def' in _PyState_AddModule().
   Initial patch by Christian Heimes.
diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c
--- a/Objects/bytearrayobject.c
+++ b/Objects/bytearrayobject.c
@@ -1175,21 +1175,19 @@
 
     table: object
         Translation table, which must be a bytes object of length 256.
-    [
-    deletechars: object
-    ]
     /
+    delete as deletechars: object(c_default="NULL") = b''
 
 Return a copy with each character mapped by the given translation table.
 
-All characters occurring in the optional argument deletechars are removed.
+All characters occurring in the optional argument delete are removed.
 The remaining characters are mapped through the given translation table.
 [clinic start generated code]*/
 
 static PyObject *
 bytearray_translate_impl(PyByteArrayObject *self, PyObject *table,
-                         int group_right_1, PyObject *deletechars)
-/*[clinic end generated code: output=2bebc86a9a1ff083 input=846a01671bccc1c5]*/
+                         PyObject *deletechars)
+/*[clinic end generated code: output=b6a8f01c2a74e446 input=cfff956d4d127a9b]*/
 {
     char *input, *output;
     const char *table_chars;
@@ -1258,8 +1256,7 @@
     for (i = inlen; --i >= 0; ) {
         c = Py_CHARMASK(*input++);
         if (trans_table[c] != -1)
-            if (Py_CHARMASK(*output++ = (char)trans_table[c]) == c)
-                    continue;
+            *output++ = (char)trans_table[c];
     }
     /* Fix the size of the resulting string */
     if (inlen > 0)
diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c
--- a/Objects/bytesobject.c
+++ b/Objects/bytesobject.c
@@ -2045,21 +2045,19 @@
 
     table: object
         Translation table, which must be a bytes object of length 256.
-    [
-    deletechars: object
-    ]
     /
+    delete as deletechars: object(c_default="NULL") = b''
 
 Return a copy with each character mapped by the given translation table.
 
-All characters occurring in the optional argument deletechars are removed.
+All characters occurring in the optional argument delete are removed.
 The remaining characters are mapped through the given translation table.
 [clinic start generated code]*/
 
 static PyObject *
-bytes_translate_impl(PyBytesObject *self, PyObject *table, int group_right_1,
+bytes_translate_impl(PyBytesObject *self, PyObject *table,
                      PyObject *deletechars)
-/*[clinic end generated code: output=233df850eb50bf8d input=ca20edf39d780d49]*/
+/*[clinic end generated code: output=43be3437f1956211 input=0ecdf159f654233c]*/
 {
     char *input, *output;
     Py_buffer table_view = {NULL, NULL};
diff --git a/Objects/clinic/bytearrayobject.c.h b/Objects/clinic/bytearrayobject.c.h
--- a/Objects/clinic/bytearrayobject.c.h
+++ b/Objects/clinic/bytearrayobject.c.h
@@ -39,47 +39,38 @@
 }
 
 PyDoc_STRVAR(bytearray_translate__doc__,
-"translate(table, [deletechars])\n"
+"translate($self, table, /, delete=b\'\')\n"
+"--\n"
+"\n"
 "Return a copy with each character mapped by the given translation table.\n"
 "\n"
 "  table\n"
 "    Translation table, which must be a bytes object of length 256.\n"
 "\n"
-"All characters occurring in the optional argument deletechars are removed.\n"
+"All characters occurring in the optional argument delete are removed.\n"
 "The remaining characters are mapped through the given translation table.");
 
 #define BYTEARRAY_TRANSLATE_METHODDEF    \
-    {"translate", (PyCFunction)bytearray_translate, METH_VARARGS, bytearray_translate__doc__},
+    {"translate", (PyCFunction)bytearray_translate, METH_VARARGS|METH_KEYWORDS, bytearray_translate__doc__},
 
 static PyObject *
 bytearray_translate_impl(PyByteArrayObject *self, PyObject *table,
-                         int group_right_1, PyObject *deletechars);
+                         PyObject *deletechars);
 
 static PyObject *
-bytearray_translate(PyByteArrayObject *self, PyObject *args)
+bytearray_translate(PyByteArrayObject *self, PyObject *args, PyObject *kwargs)
 {
     PyObject *return_value = NULL;
+    static const char * const _keywords[] = {"", "delete", NULL};
+    static _PyArg_Parser _parser = {"O|O:translate", _keywords, 0};
     PyObject *table;
-    int group_right_1 = 0;
     PyObject *deletechars = NULL;
 
-    switch (PyTuple_GET_SIZE(args)) {
-        case 1:
-            if (!PyArg_ParseTuple(args, "O:translate", &table)) {
-                goto exit;
-            }
-            break;
-        case 2:
-            if (!PyArg_ParseTuple(args, "OO:translate", &table, &deletechars)) {
-                goto exit;
-            }
-            group_right_1 = 1;
-            break;
-        default:
-            PyErr_SetString(PyExc_TypeError, "bytearray.translate requires 1 to 2 arguments");
-            goto exit;
+    if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser,
+        &table, &deletechars)) {
+        goto exit;
     }
-    return_value = bytearray_translate_impl(self, table, group_right_1, deletechars);
+    return_value = bytearray_translate_impl(self, table, deletechars);
 
 exit:
     return return_value;
@@ -720,4 +711,4 @@
 {
     return bytearray_sizeof_impl(self);
 }
-/*[clinic end generated code: output=0af30f8c0b1ecd76 input=a9049054013a1b77]*/
+/*[clinic end generated code: output=59a0c86b29ff06d1 input=a9049054013a1b77]*/
diff --git a/Objects/clinic/bytesobject.c.h b/Objects/clinic/bytesobject.c.h
--- a/Objects/clinic/bytesobject.c.h
+++ b/Objects/clinic/bytesobject.c.h
@@ -269,47 +269,38 @@
 }
 
 PyDoc_STRVAR(bytes_translate__doc__,
-"translate(table, [deletechars])\n"
+"translate($self, table, /, delete=b\'\')\n"
+"--\n"
+"\n"
 "Return a copy with each character mapped by the given translation table.\n"
 "\n"
 "  table\n"
 "    Translation table, which must be a bytes object of length 256.\n"
 "\n"
-"All characters occurring in the optional argument deletechars are removed.\n"
+"All characters occurring in the optional argument delete are removed.\n"
 "The remaining characters are mapped through the given translation table.");
 
 #define BYTES_TRANSLATE_METHODDEF    \
-    {"translate", (PyCFunction)bytes_translate, METH_VARARGS, bytes_translate__doc__},
+    {"translate", (PyCFunction)bytes_translate, METH_VARARGS|METH_KEYWORDS, bytes_translate__doc__},
 
 static PyObject *
-bytes_translate_impl(PyBytesObject *self, PyObject *table, int group_right_1,
+bytes_translate_impl(PyBytesObject *self, PyObject *table,
                      PyObject *deletechars);
 
 static PyObject *
-bytes_translate(PyBytesObject *self, PyObject *args)
+bytes_translate(PyBytesObject *self, PyObject *args, PyObject *kwargs)
 {
     PyObject *return_value = NULL;
+    static const char * const _keywords[] = {"", "delete", NULL};
+    static _PyArg_Parser _parser = {"O|O:translate", _keywords, 0};
     PyObject *table;
-    int group_right_1 = 0;
     PyObject *deletechars = NULL;
 
-    switch (PyTuple_GET_SIZE(args)) {
-        case 1:
-            if (!PyArg_ParseTuple(args, "O:translate", &table)) {
-                goto exit;
-            }
-            break;
-        case 2:
-            if (!PyArg_ParseTuple(args, "OO:translate", &table, &deletechars)) {
-                goto exit;
-            }
-            group_right_1 = 1;
-            break;
-        default:
-            PyErr_SetString(PyExc_TypeError, "bytes.translate requires 1 to 2 arguments");
-            goto exit;
+    if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser,
+        &table, &deletechars)) {
+        goto exit;
     }
-    return_value = bytes_translate_impl(self, table, group_right_1, deletechars);
+    return_value = bytes_translate_impl(self, table, deletechars);
 
 exit:
     return return_value;
@@ -508,4 +499,4 @@
 exit:
     return return_value;
 }
-/*[clinic end generated code: output=637c2c14610d3c8d input=a9049054013a1b77]*/
+/*[clinic end generated code: output=5618c05c24c1e617 input=a9049054013a1b77]*/

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Aug 27 12:42:44 2016
From: python-checkins at python.org (brett.cannon)
Date: Sat, 27 Aug 2016 16:42:44 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Don=27t_test_for_path-like?=
 =?utf-8?q?_bytes_paths_on_Windows?=
Message-ID: <20160827164244.1752.78867.343C02FC@psf.io>

https://hg.python.org/cpython/rev/775158408ecb
changeset:   102930:775158408ecb
user:        Brett Cannon 
date:        Sat Aug 27 09:42:40 2016 -0700
summary:
  Don't test for path-like bytes paths on Windows

files:
  Lib/test/test_os.py |  9 +++++++--
  1 files changed, 7 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
--- a/Lib/test/test_os.py
+++ b/Lib/test/test_os.py
@@ -2841,14 +2841,17 @@
                 return self.path
 
         str_filename = support.TESTFN
-        bytes_filename = support.TESTFN.encode('ascii')
+        if os.name == 'nt':
+            bytes_fspath = bytes_filename = None
+        else:
+            bytes_filename = support.TESTFN.encode('ascii')
+            bytes_fspath = PathLike(bytes_filename)
         fd = os.open(PathLike(str_filename), os.O_WRONLY|os.O_CREAT)
         self.addCleanup(os.close, fd)
         self.addCleanup(support.unlink, support.TESTFN)
 
         int_fspath = PathLike(fd)
         str_fspath = PathLike(str_filename)
-        bytes_fspath = PathLike(bytes_filename)
 
         for name, allow_fd, extra_args, cleanup_fn in self.functions:
             with self.subTest(name=name):
@@ -2859,6 +2862,8 @@
 
                 for path in (str_filename, bytes_filename, str_fspath,
                              bytes_fspath):
+                    if path is None:
+                        continue
                     with self.subTest(name=name, path=path):
                         result = fn(path, *extra_args)
                         if cleanup_fn is not None:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Aug 27 14:25:15 2016
From: python-checkins at python.org (berker.peksag)
Date: Sat, 27 Aug 2016 18:25:15 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326027=3A_Fix_test?=
 =?utf-8?q?=5Fpath=5Ft=5Fconverter_on_Windows?=
Message-ID: <20160827182515.27150.90239.5E4C8986@psf.io>

https://hg.python.org/cpython/rev/8ec5a00e5d75
changeset:   102931:8ec5a00e5d75
user:        Berker Peksag 
date:        Sat Aug 27 21:26:35 2016 +0300
summary:
  Issue #26027: Fix test_path_t_converter on Windows

files:
  Lib/test/test_os.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
--- a/Lib/test/test_os.py
+++ b/Lib/test/test_os.py
@@ -2847,8 +2847,8 @@
             bytes_filename = support.TESTFN.encode('ascii')
             bytes_fspath = PathLike(bytes_filename)
         fd = os.open(PathLike(str_filename), os.O_WRONLY|os.O_CREAT)
+        self.addCleanup(support.unlink, support.TESTFN)
         self.addCleanup(os.close, fd)
-        self.addCleanup(support.unlink, support.TESTFN)
 
         int_fspath = PathLike(fd)
         str_fspath = PathLike(str_filename)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Aug 29 08:12:25 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 29 Aug 2016 12:12:25 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2327861=3A_Fixed_a_crash_in_sqlite3=2EConnection?=
 =?utf-8?q?=2Ecursor=28=29_when_a_factory?=
Message-ID: <20160829121225.18218.22260.5E623A6B@psf.io>

https://hg.python.org/cpython/rev/afcec2d11e9e
changeset:   102934:afcec2d11e9e
parent:      102931:8ec5a00e5d75
parent:      102932:c046cbb24f98
user:        Serhiy Storchaka 
date:        Mon Aug 29 15:11:52 2016 +0300
summary:
  Issue #27861: Fixed a crash in sqlite3.Connection.cursor() when a factory
creates not a cursor.  Patch by Xiang Zhang.

files:
  Doc/library/sqlite3.rst      |   8 ++++----
  Lib/sqlite3/test/factory.py  |  20 +++++++++++++++++---
  Misc/NEWS                    |   3 +++
  Modules/_sqlite/connection.c |  13 +++++++++++--
  4 files changed, 35 insertions(+), 9 deletions(-)


diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst
--- a/Doc/library/sqlite3.rst
+++ b/Doc/library/sqlite3.rst
@@ -285,11 +285,11 @@
 
       .. versionadded:: 3.2
 
-   .. method:: cursor([cursorClass])
+   .. method:: cursor(factory=Cursor)
 
-      The cursor method accepts a single optional parameter *cursorClass*. If
-      supplied, this must be a custom cursor class that extends
-      :class:`sqlite3.Cursor`.
+      The cursor method accepts a single optional parameter *factory*. If
+      supplied, this must be a callable returning an instance of :class:`Cursor`
+      or its subclasses.
 
    .. method:: commit()
 
diff --git a/Lib/sqlite3/test/factory.py b/Lib/sqlite3/test/factory.py
--- a/Lib/sqlite3/test/factory.py
+++ b/Lib/sqlite3/test/factory.py
@@ -58,8 +58,20 @@
         self.con.close()
 
     def CheckIsInstance(self):
-        cur = self.con.cursor(factory=MyCursor)
+        cur = self.con.cursor()
+        self.assertIsInstance(cur, sqlite.Cursor)
+        cur = self.con.cursor(MyCursor)
         self.assertIsInstance(cur, MyCursor)
+        cur = self.con.cursor(factory=lambda con: MyCursor(con))
+        self.assertIsInstance(cur, MyCursor)
+
+    def CheckInvalidFactory(self):
+        # not a callable at all
+        self.assertRaises(TypeError, self.con.cursor, None)
+        # invalid callable with not exact one argument
+        self.assertRaises(TypeError, self.con.cursor, lambda: None)
+        # invalid callable returning non-cursor
+        self.assertRaises(TypeError, self.con.cursor, lambda con: None)
 
 class RowFactoryTestsBackwardsCompat(unittest.TestCase):
     def setUp(self):
@@ -183,10 +195,12 @@
     def CheckFakeCursorClass(self):
         # Issue #24257: Incorrect use of PyObject_IsInstance() caused
         # segmentation fault.
+        # Issue #27861: Also applies for cursor factory.
         class FakeCursor(str):
             __class__ = sqlite.Cursor
-        cur = self.con.cursor(factory=FakeCursor)
-        self.assertRaises(TypeError, sqlite.Row, cur, ())
+        self.con.row_factory = sqlite.Row
+        self.assertRaises(TypeError, self.con.cursor, FakeCursor)
+        self.assertRaises(TypeError, sqlite.Row, FakeCursor(), ())
 
     def tearDown(self):
         self.con.close()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -49,6 +49,9 @@
 Library
 -------
 
+- Issue #27861: Fixed a crash in sqlite3.Connection.cursor() when a factory
+  creates not a cursor.  Patch by Xiang Zhang.
+
 - Issue #19884: Avoid spurious output on OS X with Gnu Readline.
 
 - Issue #10513: Fix a regression in Connection.commit().  Statements should
diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c
--- a/Modules/_sqlite/connection.c
+++ b/Modules/_sqlite/connection.c
@@ -300,7 +300,7 @@
 
 PyObject* pysqlite_connection_cursor(pysqlite_Connection* self, PyObject* args, PyObject* kwargs)
 {
-    static char *kwlist[] = {"factory", NULL, NULL};
+    static char *kwlist[] = {"factory", NULL};
     PyObject* factory = NULL;
     PyObject* cursor;
 
@@ -317,7 +317,16 @@
         factory = (PyObject*)&pysqlite_CursorType;
     }
 
-    cursor = PyObject_CallFunction(factory, "O", self);
+    cursor = PyObject_CallFunctionObjArgs(factory, (PyObject *)self, NULL);
+    if (cursor == NULL)
+        return NULL;
+    if (!PyObject_TypeCheck(cursor, &pysqlite_CursorType)) {
+        PyErr_Format(PyExc_TypeError,
+                     "factory must return a cursor, not %.100s",
+                     Py_TYPE(cursor)->tp_name);
+        Py_DECREF(cursor);
+        return NULL;
+    }
 
     _pysqlite_drop_unused_cursor_references(self);
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Aug 29 08:12:25 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 29 Aug 2016 12:12:25 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3ODYx?=
 =?utf-8?q?=3A_Fixed_a_crash_in_sqlite3=2EConnection=2Ecursor=28=29_when_a?=
 =?utf-8?q?_factory?=
Message-ID: <20160829121225.20168.15378.AD24AE3D@psf.io>

https://hg.python.org/cpython/rev/97dbba8a6d4a
changeset:   102933:97dbba8a6d4a
branch:      2.7
parent:      102924:55ec5fdc3099
user:        Serhiy Storchaka 
date:        Mon Aug 29 14:29:55 2016 +0300
summary:
  Issue #27861: Fixed a crash in sqlite3.Connection.cursor() when a factory
creates not a cursor.  Patch by Xiang Zhang.

files:
  Doc/library/sqlite3.rst      |   8 ++++----
  Lib/sqlite3/test/factory.py  |  20 +++++++++++++++++---
  Misc/NEWS                    |   3 +++
  Modules/_sqlite/connection.c |  13 +++++++++++--
  4 files changed, 35 insertions(+), 9 deletions(-)


diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst
--- a/Doc/library/sqlite3.rst
+++ b/Doc/library/sqlite3.rst
@@ -256,11 +256,11 @@
       :ref:`sqlite3-controlling-transactions` for a more detailed explanation.
 
 
-   .. method:: cursor([cursorClass])
+   .. method:: cursor(factory=Cursor)
 
-      The cursor method accepts a single optional parameter *cursorClass*. If
-      supplied, this must be a custom cursor class that extends
-      :class:`sqlite3.Cursor`.
+      The cursor method accepts a single optional parameter *factory*. If
+      supplied, this must be a callable returning an instance of :class:`Cursor`
+      or its subclasses.
 
    .. method:: commit()
 
diff --git a/Lib/sqlite3/test/factory.py b/Lib/sqlite3/test/factory.py
--- a/Lib/sqlite3/test/factory.py
+++ b/Lib/sqlite3/test/factory.py
@@ -58,8 +58,20 @@
         self.con.close()
 
     def CheckIsInstance(self):
-        cur = self.con.cursor(factory=MyCursor)
+        cur = self.con.cursor()
+        self.assertIsInstance(cur, sqlite.Cursor)
+        cur = self.con.cursor(MyCursor)
         self.assertIsInstance(cur, MyCursor)
+        cur = self.con.cursor(factory=lambda con: MyCursor(con))
+        self.assertIsInstance(cur, MyCursor)
+
+    def CheckInvalidFactory(self):
+        # not a callable at all
+        self.assertRaises(TypeError, self.con.cursor, None)
+        # invalid callable with not exact one argument
+        self.assertRaises(TypeError, self.con.cursor, lambda: None)
+        # invalid callable returning non-cursor
+        self.assertRaises(TypeError, self.con.cursor, lambda con: None)
 
 class RowFactoryTestsBackwardsCompat(unittest.TestCase):
     def setUp(self):
@@ -173,10 +185,12 @@
     def CheckFakeCursorClass(self):
         # Issue #24257: Incorrect use of PyObject_IsInstance() caused
         # segmentation fault.
+        # Issue #27861: Also applies for cursor factory.
         class FakeCursor(str):
             __class__ = sqlite.Cursor
-        cur = self.con.cursor(factory=FakeCursor)
-        self.assertRaises(TypeError, sqlite.Row, cur, ())
+        self.con.row_factory = sqlite.Row
+        self.assertRaises(TypeError, self.con.cursor, FakeCursor)
+        self.assertRaises(TypeError, sqlite.Row, FakeCursor(), ())
 
     def tearDown(self):
         self.con.close()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -33,6 +33,9 @@
 Library
 -------
 
+- Issue #27861: Fixed a crash in sqlite3.Connection.cursor() when a factory
+  creates not a cursor.  Patch by Xiang Zhang.
+
 - Issue #19884: Avoid spurious output on OS X with Gnu Readline.
 
 - Issue #10513: Fix a regression in Connection.commit().  Statements should
diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c
--- a/Modules/_sqlite/connection.c
+++ b/Modules/_sqlite/connection.c
@@ -324,7 +324,7 @@
 
 PyObject* pysqlite_connection_cursor(pysqlite_Connection* self, PyObject* args, PyObject* kwargs)
 {
-    static char *kwlist[] = {"factory", NULL, NULL};
+    static char *kwlist[] = {"factory", NULL};
     PyObject* factory = NULL;
     PyObject* cursor;
 
@@ -341,7 +341,16 @@
         factory = (PyObject*)&pysqlite_CursorType;
     }
 
-    cursor = PyObject_CallFunction(factory, "O", self);
+    cursor = PyObject_CallFunctionObjArgs(factory, (PyObject *)self, NULL);
+    if (cursor == NULL)
+        return NULL;
+    if (!PyObject_TypeCheck(cursor, &pysqlite_CursorType)) {
+        PyErr_Format(PyExc_TypeError,
+                     "factory must return a cursor, not %.100s",
+                     Py_TYPE(cursor)->tp_name);
+        Py_DECREF(cursor);
+        return NULL;
+    }
 
     _pysqlite_drop_unused_cursor_references(self);
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Aug 29 08:12:26 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 29 Aug 2016 12:12:26 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3ODYx?=
 =?utf-8?q?=3A_Fixed_a_crash_in_sqlite3=2EConnection=2Ecursor=28=29_when_a?=
 =?utf-8?q?_factory?=
Message-ID: <20160829121225.45636.24071.D209F0D0@psf.io>

https://hg.python.org/cpython/rev/c046cbb24f98
changeset:   102932:c046cbb24f98
branch:      3.5
parent:      102926:ca1ddd365f5f
user:        Serhiy Storchaka 
date:        Mon Aug 29 14:29:55 2016 +0300
summary:
  Issue #27861: Fixed a crash in sqlite3.Connection.cursor() when a factory
creates not a cursor.  Patch by Xiang Zhang.

files:
  Doc/library/sqlite3.rst      |   8 ++++----
  Lib/sqlite3/test/factory.py  |  20 +++++++++++++++++---
  Misc/NEWS                    |   3 +++
  Modules/_sqlite/connection.c |  13 +++++++++++--
  4 files changed, 35 insertions(+), 9 deletions(-)


diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst
--- a/Doc/library/sqlite3.rst
+++ b/Doc/library/sqlite3.rst
@@ -285,11 +285,11 @@
 
       .. versionadded:: 3.2
 
-   .. method:: cursor([cursorClass])
+   .. method:: cursor(factory=Cursor)
 
-      The cursor method accepts a single optional parameter *cursorClass*. If
-      supplied, this must be a custom cursor class that extends
-      :class:`sqlite3.Cursor`.
+      The cursor method accepts a single optional parameter *factory*. If
+      supplied, this must be a callable returning an instance of :class:`Cursor`
+      or its subclasses.
 
    .. method:: commit()
 
diff --git a/Lib/sqlite3/test/factory.py b/Lib/sqlite3/test/factory.py
--- a/Lib/sqlite3/test/factory.py
+++ b/Lib/sqlite3/test/factory.py
@@ -58,8 +58,20 @@
         self.con.close()
 
     def CheckIsInstance(self):
-        cur = self.con.cursor(factory=MyCursor)
+        cur = self.con.cursor()
+        self.assertIsInstance(cur, sqlite.Cursor)
+        cur = self.con.cursor(MyCursor)
         self.assertIsInstance(cur, MyCursor)
+        cur = self.con.cursor(factory=lambda con: MyCursor(con))
+        self.assertIsInstance(cur, MyCursor)
+
+    def CheckInvalidFactory(self):
+        # not a callable at all
+        self.assertRaises(TypeError, self.con.cursor, None)
+        # invalid callable with not exact one argument
+        self.assertRaises(TypeError, self.con.cursor, lambda: None)
+        # invalid callable returning non-cursor
+        self.assertRaises(TypeError, self.con.cursor, lambda con: None)
 
 class RowFactoryTestsBackwardsCompat(unittest.TestCase):
     def setUp(self):
@@ -183,10 +195,12 @@
     def CheckFakeCursorClass(self):
         # Issue #24257: Incorrect use of PyObject_IsInstance() caused
         # segmentation fault.
+        # Issue #27861: Also applies for cursor factory.
         class FakeCursor(str):
             __class__ = sqlite.Cursor
-        cur = self.con.cursor(factory=FakeCursor)
-        self.assertRaises(TypeError, sqlite.Row, cur, ())
+        self.con.row_factory = sqlite.Row
+        self.assertRaises(TypeError, self.con.cursor, FakeCursor)
+        self.assertRaises(TypeError, sqlite.Row, FakeCursor(), ())
 
     def tearDown(self):
         self.con.close()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -50,6 +50,9 @@
 Library
 -------
 
+- Issue #27861: Fixed a crash in sqlite3.Connection.cursor() when a factory
+  creates not a cursor.  Patch by Xiang Zhang.
+
 - Issue #19884: Avoid spurious output on OS X with Gnu Readline.
 
 - Issue #10513: Fix a regression in Connection.commit().  Statements should
diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c
--- a/Modules/_sqlite/connection.c
+++ b/Modules/_sqlite/connection.c
@@ -300,7 +300,7 @@
 
 PyObject* pysqlite_connection_cursor(pysqlite_Connection* self, PyObject* args, PyObject* kwargs)
 {
-    static char *kwlist[] = {"factory", NULL, NULL};
+    static char *kwlist[] = {"factory", NULL};
     PyObject* factory = NULL;
     PyObject* cursor;
 
@@ -317,7 +317,16 @@
         factory = (PyObject*)&pysqlite_CursorType;
     }
 
-    cursor = PyObject_CallFunction(factory, "O", self);
+    cursor = PyObject_CallFunctionObjArgs(factory, (PyObject *)self, NULL);
+    if (cursor == NULL)
+        return NULL;
+    if (!PyObject_TypeCheck(cursor, &pysqlite_CursorType)) {
+        PyErr_Format(PyExc_TypeError,
+                     "factory must return a cursor, not %.100s",
+                     Py_TYPE(cursor)->tp_name);
+        Py_DECREF(cursor);
+        return NULL;
+    }
 
     _pysqlite_drop_unused_cursor_references(self);
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Aug 29 08:57:12 2016
From: python-checkins at python.org (mark.dickinson)
Date: Mon, 29 Aug 2016 12:57:12 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_23229=3A_add_cmath?=
 =?utf-8?q?=2Einf=2C_cmath=2Enan=2C_cmath=2Einfj_and_cmath=2Enanj=2E?=
Message-ID: <20160829125711.67844.8914.B24EA9E5@psf.io>

https://hg.python.org/cpython/rev/4b25da63d1d0
changeset:   102935:4b25da63d1d0
user:        Mark Dickinson 
date:        Mon Aug 29 13:56:58 2016 +0100
summary:
  Issue 23229: add cmath.inf, cmath.nan, cmath.infj and cmath.nanj.

files:
  Doc/library/cmath.rst  |  28 +++++++++++++++
  Lib/test/test_cmath.py |  17 +++++++++
  Misc/NEWS              |   4 ++
  Modules/cmathmodule.c  |  54 ++++++++++++++++++++++++++++++
  4 files changed, 103 insertions(+), 0 deletions(-)


diff --git a/Doc/library/cmath.rst b/Doc/library/cmath.rst
--- a/Doc/library/cmath.rst
+++ b/Doc/library/cmath.rst
@@ -259,6 +259,34 @@
 
    .. versionadded:: 3.6
 
+.. data:: inf
+
+   Floating-point positive infinity. Equivalent to ``float('inf')``.
+
+   .. versionadded:: 3.6
+
+.. data:: infj
+
+   Complex number with zero real part and positive infinity imaginary
+   part. Equivalent to ``complex(0.0, float('inf'))``.
+
+   .. versionadded:: 3.6
+
+.. data:: nan
+
+   A floating-point "not a number" (NaN) value.  Equivalent to
+   ``float('nan')``.
+
+   .. versionadded:: 3.6
+
+.. data:: nanj
+
+   Complex number with zero real part and NaN imaginary part. Equivalent to
+   ``complex(0.0, float('nan'))``.
+
+   .. versionadded:: 3.6
+
+
 .. index:: module: math
 
 Note that the selection of functions is similar, but not identical, to that in
diff --git a/Lib/test/test_cmath.py b/Lib/test/test_cmath.py
--- a/Lib/test/test_cmath.py
+++ b/Lib/test/test_cmath.py
@@ -154,6 +154,23 @@
         self.assertAlmostEqual(cmath.e, e_expected, places=9,
             msg="cmath.e is {}; should be {}".format(cmath.e, e_expected))
 
+    def test_infinity_and_nan_constants(self):
+        self.assertEqual(cmath.inf.real, math.inf)
+        self.assertEqual(cmath.inf.imag, 0.0)
+        self.assertEqual(cmath.infj.real, 0.0)
+        self.assertEqual(cmath.infj.imag, math.inf)
+
+        self.assertTrue(math.isnan(cmath.nan.real))
+        self.assertEqual(cmath.nan.imag, 0.0)
+        self.assertEqual(cmath.nanj.real, 0.0)
+        self.assertTrue(math.isnan(cmath.nanj.imag))
+
+        # Check consistency with reprs.
+        self.assertEqual(repr(cmath.inf), "inf")
+        self.assertEqual(repr(cmath.infj), "infj")
+        self.assertEqual(repr(cmath.nan), "nan")
+        self.assertEqual(repr(cmath.nanj), "nanj")
+
     def test_user_object(self):
         # Test automatic calling of __complex__ and __float__ by cmath
         # functions
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -49,6 +49,10 @@
 Library
 -------
 
+- Issue #23229: Add new ``cmath`` constants: ``cmath.inf`` and ``cmath.nan`` to
+  match ``math.inf`` and ``math.nan``, and also ``cmath.infj`` and
+  ``cmath.nanj`` to match the format used by complex repr.
+
 - Issue #27861: Fixed a crash in sqlite3.Connection.cursor() when a factory
   creates not a cursor.  Patch by Xiang Zhang.
 
diff --git a/Modules/cmathmodule.c b/Modules/cmathmodule.c
--- a/Modules/cmathmodule.c
+++ b/Modules/cmathmodule.c
@@ -81,6 +81,54 @@
 #endif
 #define CM_SCALE_DOWN (-(CM_SCALE_UP+1)/2)
 
+/* Constants cmath.inf, cmath.infj, cmath.nan, cmath.nanj.
+   cmath.nan and cmath.nanj are defined only when either
+   PY_NO_SHORT_FLOAT_REPR is *not* defined (which should be
+   the most common situation on machines using an IEEE 754
+   representation), or Py_NAN is defined. */
+
+static double
+m_inf(void)
+{
+#ifndef PY_NO_SHORT_FLOAT_REPR
+    return _Py_dg_infinity(0);
+#else
+    return Py_HUGE_VAL;
+#endif
+}
+
+static Py_complex
+c_infj(void)
+{
+    Py_complex r;
+    r.real = 0.0;
+    r.imag = m_inf();
+    return r;
+}
+
+#if !defined(PY_NO_SHORT_FLOAT_REPR) || defined(Py_NAN)
+
+static double
+m_nan(void)
+{
+#ifndef PY_NO_SHORT_FLOAT_REPR
+    return _Py_dg_stdnan(0);
+#else
+    return Py_NAN;
+#endif
+}
+
+static Py_complex
+c_nanj(void)
+{
+    Py_complex r;
+    r.real = 0.0;
+    r.imag = m_nan();
+    return r;
+}
+
+#endif
+
 /* forward declarations */
 static Py_complex cmath_asinh_impl(PyObject *, Py_complex);
 static Py_complex cmath_atanh_impl(PyObject *, Py_complex);
@@ -1240,6 +1288,12 @@
                        PyFloat_FromDouble(Py_MATH_PI));
     PyModule_AddObject(m, "e", PyFloat_FromDouble(Py_MATH_E));
     PyModule_AddObject(m, "tau", PyFloat_FromDouble(Py_MATH_TAU)); /* 2pi */
+    PyModule_AddObject(m, "inf", PyFloat_FromDouble(m_inf()));
+    PyModule_AddObject(m, "infj", PyComplex_FromCComplex(c_infj()));
+#if !defined(PY_NO_SHORT_FLOAT_REPR) || defined(Py_NAN)
+    PyModule_AddObject(m, "nan", PyFloat_FromDouble(m_nan()));
+    PyModule_AddObject(m, "nanj", PyComplex_FromCComplex(c_nanj()));
+#endif
 
     /* initialize special value tables */
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Aug 29 08:59:59 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 29 Aug 2016 12:59:59 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327818=3A_Speed_up?=
 =?utf-8?q?_parsing_width_and_precision_in_format=28=29_strings_for?=
Message-ID: <20160829125959.1752.4449.77146473@psf.io>

https://hg.python.org/cpython/rev/9bddf9e72c96
changeset:   102936:9bddf9e72c96
parent:      102934:afcec2d11e9e
user:        Serhiy Storchaka 
date:        Mon Aug 29 15:57:26 2016 +0300
summary:
  Issue #27818: Speed up parsing width and precision in format() strings for
numbers.  Patch by Stefan Behnel.

files:
  Python/formatter_unicode.c |  25 ++++++++++++++++---------
  1 files changed, 16 insertions(+), 9 deletions(-)


diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c
--- a/Python/formatter_unicode.c
+++ b/Python/formatter_unicode.c
@@ -48,16 +48,17 @@
     returns -1 on error.
 */
 static int
-get_integer(PyObject *str, Py_ssize_t *pos, Py_ssize_t end,
+get_integer(PyObject *str, Py_ssize_t *ppos, Py_ssize_t end,
                   Py_ssize_t *result)
 {
-    Py_ssize_t accumulator, digitval;
+    Py_ssize_t accumulator, digitval, pos = *ppos;
     int numdigits;
+    int kind = PyUnicode_KIND(str);
+    void *data = PyUnicode_DATA(str);
+
     accumulator = numdigits = 0;
-    for (;;(*pos)++, numdigits++) {
-        if (*pos >= end)
-            break;
-        digitval = Py_UNICODE_TODECIMAL(PyUnicode_READ_CHAR(str, *pos));
+    for (; pos < end; pos++, numdigits++) {
+        digitval = Py_UNICODE_TODECIMAL(PyUnicode_READ(kind, data, pos));
         if (digitval < 0)
             break;
         /*
@@ -69,10 +70,12 @@
         if (accumulator > (PY_SSIZE_T_MAX - digitval) / 10) {
             PyErr_Format(PyExc_ValueError,
                          "Too many decimal digits in format string");
+            *ppos = pos;
             return -1;
         }
         accumulator = accumulator * 10 + digitval;
     }
+    *ppos = pos;
     *result = accumulator;
     return numdigits;
 }
@@ -150,9 +153,11 @@
                                   char default_align)
 {
     Py_ssize_t pos = start;
+    int kind = PyUnicode_KIND(format_spec);
+    void *data = PyUnicode_DATA(format_spec);
     /* end-pos is used throughout this code to specify the length of
        the input string */
-#define READ_spec(index) PyUnicode_READ_CHAR(format_spec, index)
+#define READ_spec(index) PyUnicode_READ(kind, data, index)
 
     Py_ssize_t consumed;
     int align_specified = 0;
@@ -402,13 +407,15 @@
              Py_ssize_t *n_remainder, int *has_decimal)
 {
     Py_ssize_t remainder;
+    int kind = PyUnicode_KIND(s);
+    void *data = PyUnicode_DATA(s);
 
-    while (pos

https://hg.python.org/cpython/rev/c6fd0f3aadc4
changeset:   102937:c6fd0f3aadc4
parent:      102936:9bddf9e72c96
parent:      102935:4b25da63d1d0
user:        Serhiy Storchaka 
date:        Mon Aug 29 15:59:48 2016 +0300
summary:
  Merge heads

files:
  Doc/library/cmath.rst  |  28 +++++++++++++++
  Lib/test/test_cmath.py |  17 +++++++++
  Misc/NEWS              |   4 ++
  Modules/cmathmodule.c  |  54 ++++++++++++++++++++++++++++++
  4 files changed, 103 insertions(+), 0 deletions(-)


diff --git a/Doc/library/cmath.rst b/Doc/library/cmath.rst
--- a/Doc/library/cmath.rst
+++ b/Doc/library/cmath.rst
@@ -259,6 +259,34 @@
 
    .. versionadded:: 3.6
 
+.. data:: inf
+
+   Floating-point positive infinity. Equivalent to ``float('inf')``.
+
+   .. versionadded:: 3.6
+
+.. data:: infj
+
+   Complex number with zero real part and positive infinity imaginary
+   part. Equivalent to ``complex(0.0, float('inf'))``.
+
+   .. versionadded:: 3.6
+
+.. data:: nan
+
+   A floating-point "not a number" (NaN) value.  Equivalent to
+   ``float('nan')``.
+
+   .. versionadded:: 3.6
+
+.. data:: nanj
+
+   Complex number with zero real part and NaN imaginary part. Equivalent to
+   ``complex(0.0, float('nan'))``.
+
+   .. versionadded:: 3.6
+
+
 .. index:: module: math
 
 Note that the selection of functions is similar, but not identical, to that in
diff --git a/Lib/test/test_cmath.py b/Lib/test/test_cmath.py
--- a/Lib/test/test_cmath.py
+++ b/Lib/test/test_cmath.py
@@ -154,6 +154,23 @@
         self.assertAlmostEqual(cmath.e, e_expected, places=9,
             msg="cmath.e is {}; should be {}".format(cmath.e, e_expected))
 
+    def test_infinity_and_nan_constants(self):
+        self.assertEqual(cmath.inf.real, math.inf)
+        self.assertEqual(cmath.inf.imag, 0.0)
+        self.assertEqual(cmath.infj.real, 0.0)
+        self.assertEqual(cmath.infj.imag, math.inf)
+
+        self.assertTrue(math.isnan(cmath.nan.real))
+        self.assertEqual(cmath.nan.imag, 0.0)
+        self.assertEqual(cmath.nanj.real, 0.0)
+        self.assertTrue(math.isnan(cmath.nanj.imag))
+
+        # Check consistency with reprs.
+        self.assertEqual(repr(cmath.inf), "inf")
+        self.assertEqual(repr(cmath.infj), "infj")
+        self.assertEqual(repr(cmath.nan), "nan")
+        self.assertEqual(repr(cmath.nanj), "nanj")
+
     def test_user_object(self):
         # Test automatic calling of __complex__ and __float__ by cmath
         # functions
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -49,6 +49,10 @@
 Library
 -------
 
+- Issue #23229: Add new ``cmath`` constants: ``cmath.inf`` and ``cmath.nan`` to
+  match ``math.inf`` and ``math.nan``, and also ``cmath.infj`` and
+  ``cmath.nanj`` to match the format used by complex repr.
+
 - Issue #27861: Fixed a crash in sqlite3.Connection.cursor() when a factory
   creates not a cursor.  Patch by Xiang Zhang.
 
diff --git a/Modules/cmathmodule.c b/Modules/cmathmodule.c
--- a/Modules/cmathmodule.c
+++ b/Modules/cmathmodule.c
@@ -81,6 +81,54 @@
 #endif
 #define CM_SCALE_DOWN (-(CM_SCALE_UP+1)/2)
 
+/* Constants cmath.inf, cmath.infj, cmath.nan, cmath.nanj.
+   cmath.nan and cmath.nanj are defined only when either
+   PY_NO_SHORT_FLOAT_REPR is *not* defined (which should be
+   the most common situation on machines using an IEEE 754
+   representation), or Py_NAN is defined. */
+
+static double
+m_inf(void)
+{
+#ifndef PY_NO_SHORT_FLOAT_REPR
+    return _Py_dg_infinity(0);
+#else
+    return Py_HUGE_VAL;
+#endif
+}
+
+static Py_complex
+c_infj(void)
+{
+    Py_complex r;
+    r.real = 0.0;
+    r.imag = m_inf();
+    return r;
+}
+
+#if !defined(PY_NO_SHORT_FLOAT_REPR) || defined(Py_NAN)
+
+static double
+m_nan(void)
+{
+#ifndef PY_NO_SHORT_FLOAT_REPR
+    return _Py_dg_stdnan(0);
+#else
+    return Py_NAN;
+#endif
+}
+
+static Py_complex
+c_nanj(void)
+{
+    Py_complex r;
+    r.real = 0.0;
+    r.imag = m_nan();
+    return r;
+}
+
+#endif
+
 /* forward declarations */
 static Py_complex cmath_asinh_impl(PyObject *, Py_complex);
 static Py_complex cmath_atanh_impl(PyObject *, Py_complex);
@@ -1240,6 +1288,12 @@
                        PyFloat_FromDouble(Py_MATH_PI));
     PyModule_AddObject(m, "e", PyFloat_FromDouble(Py_MATH_E));
     PyModule_AddObject(m, "tau", PyFloat_FromDouble(Py_MATH_TAU)); /* 2pi */
+    PyModule_AddObject(m, "inf", PyFloat_FromDouble(m_inf()));
+    PyModule_AddObject(m, "infj", PyComplex_FromCComplex(c_infj()));
+#if !defined(PY_NO_SHORT_FLOAT_REPR) || defined(Py_NAN)
+    PyModule_AddObject(m, "nan", PyFloat_FromDouble(m_nan()));
+    PyModule_AddObject(m, "nanj", PyComplex_FromCComplex(c_nanj()));
+#endif
 
     /* initialize special value tables */
 

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Mon Aug 29 10:21:14 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Mon, 29 Aug 2016 15:21:14 +0100
Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7
 2016-08-29
Message-ID: <3b262eec-6d71-4839-9c38-1e3292350629@irsmsx151.ger.corp.intel.com>

Results for project Python 2.7, build date 2016-08-29 02:47:41 +0000
commit:		55ec5fdc3099
previous commit:	24e580e667fb
revision date:	2016-08-27 02:54:43 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.14%          0.81%          5.04%             4.48%
:-)             pybench      0.22%          0.05%          6.20%             4.31%
:-(            regex_v8      0.59%         -0.12%         -2.08%            10.78%
:-)               nbody      0.10%         -0.03%          7.68%             1.67%
:-)        json_dump_v2      0.31%          0.27%          3.53%            10.08%
:-|      normal_startup      1.05%          0.29%         -0.58%             2.17%
:-)             ssbench      0.19%         -0.30%          2.71%             1.85%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-08-29/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Mon Aug 29 10:47:54 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Mon, 29 Aug 2016 15:47:54 +0100
Subject: [Python-checkins] GOOD Benchmark Results for Python Default
 2016-08-29
Message-ID: 

Results for project Python default, build date 2016-08-29 02:00:44 +0000
commit:		8ec5a00e5d75
previous commit:	10a73648c92c
revision date:	2016-08-27 18:26:35 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.32%          2.35%         10.42%            15.89%
:-)             pybench      2.99%          0.26%          6.84%             6.29%
:-(            regex_v8      2.61%          0.56%         -3.97%            10.16%
:-|               nbody      0.15%          1.56%          1.59%             8.26%
:-|        json_dump_v2      0.31%          0.10%         -1.43%            11.10%
:-)      normal_startup      1.22%         -0.08%          2.83%             6.15%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-08-29/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Mon Aug 29 11:40:40 2016
From: python-checkins at python.org (mark.dickinson)
Date: Mon, 29 Aug 2016 15:40:40 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327214=3A_Fix_pote?=
 =?utf-8?q?ntial_bug_and_remove_useless_optimization_in_long=5Finvert=2E?=
Message-ID: <20160829154040.63860.11097.3A8D2488@psf.io>

https://hg.python.org/cpython/rev/6e1d38674b17
changeset:   102938:6e1d38674b17
user:        Mark Dickinson 
date:        Mon Aug 29 16:40:29 2016 +0100
summary:
  Issue #27214: Fix potential bug and remove useless optimization in long_invert. Thanks Oren Milman.

files:
  Misc/NEWS            |  4 ++++
  Objects/longobject.c |  6 ++++--
  2 files changed, 8 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,10 @@
 Core and Builtins
 -----------------
 
+- Issue #27214: In long_invert, be more careful about modifying object
+  returned by long_add, and remove an unnecessary check for small longs.
+  Thanks Oren Milman for analysis and patch.
+
 - Issue #27506: Support passing the bytes/bytearray.translate() "delete"
   argument by keyword.
 
diff --git a/Objects/longobject.c b/Objects/longobject.c
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -4170,8 +4170,10 @@
     Py_DECREF(w);
     if (x == NULL)
         return NULL;
-    Py_SIZE(x) = -(Py_SIZE(x));
-    return (PyObject *)maybe_small_long(x);
+    _PyLong_Negate(&x);
+    /* No need for maybe_small_long here, since any small
+       longs will have been caught in the Py_SIZE <= 1 fast path. */
+    return (PyObject *)x;
 }
 
 static PyObject *

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Aug 29 12:27:05 2016
From: python-checkins at python.org (mark.dickinson)
Date: Mon, 29 Aug 2016 16:27:05 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325402=3A_in_int-t?=
 =?utf-8?q?o-decimal-string_conversion=2C_reduce_intermediate_storage?=
Message-ID: <20160829162654.82639.59135.2C802371@psf.io>

https://hg.python.org/cpython/rev/1902e1d79e25
changeset:   102939:1902e1d79e25
user:        Mark Dickinson 
date:        Mon Aug 29 17:26:43 2016 +0100
summary:
  Issue #25402: in int-to-decimal-string conversion, reduce intermediate storage requirements and relax restriction on converting large integers. Patch by Serhiy Storchaka.

files:
  Misc/NEWS            |   4 ++++
  Objects/longobject.c |  19 +++++++++++--------
  2 files changed, 15 insertions(+), 8 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,10 @@
 Core and Builtins
 -----------------
 
+- Issue #25402: In int-to-decimal-string conversion, improve the estimate
+  of the intermediate memory required, and remove an unnecessarily strict
+  overflow check. Patch by Serhiy Storchaka.
+
 - Issue #27214: In long_invert, be more careful about modifying object
   returned by long_add, and remove an unnecessary check for small longs.
   Thanks Oren Milman for analysis and patch.
diff --git a/Objects/longobject.c b/Objects/longobject.c
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -1591,6 +1591,7 @@
     Py_ssize_t size, strlen, size_a, i, j;
     digit *pout, *pin, rem, tenpow;
     int negative;
+    int d;
     enum PyUnicode_Kind kind;
 
     a = (PyLongObject *)aa;
@@ -1608,15 +1609,17 @@
 
        But log2(a) < size_a * PyLong_SHIFT, and
        log2(_PyLong_DECIMAL_BASE) = log2(10) * _PyLong_DECIMAL_SHIFT
-                                  > 3 * _PyLong_DECIMAL_SHIFT
+                                  > 3.3 * _PyLong_DECIMAL_SHIFT
+
+         size_a * PyLong_SHIFT / (3.3 * _PyLong_DECIMAL_SHIFT) =
+             size_a + size_a / d < size_a + size_a / floor(d),
+       where d = (3.3 * _PyLong_DECIMAL_SHIFT) /
+                 (PyLong_SHIFT - 3.3 * _PyLong_DECIMAL_SHIFT)
     */
-    if (size_a > PY_SSIZE_T_MAX / PyLong_SHIFT) {
-        PyErr_SetString(PyExc_OverflowError,
-                        "int too large to format");
-        return -1;
-    }
-    /* the expression size_a * PyLong_SHIFT is now safe from overflow */
-    size = 1 + size_a * PyLong_SHIFT / (3 * _PyLong_DECIMAL_SHIFT);
+    d = (33 * _PyLong_DECIMAL_SHIFT) /
+        (10 * PyLong_SHIFT - 33 * _PyLong_DECIMAL_SHIFT);
+    assert(size_a < PY_SSIZE_T_MAX/2);
+    size = 1 + size_a + size_a / d;
     scratch = _PyLong_New(size);
     if (scratch == NULL)
         return -1;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Aug 29 14:03:32 2016
From: python-checkins at python.org (matthias.klose)
Date: Mon, 29 Aug 2016 18:03:32 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A__-_Issue_=2323968=2C_keep_?=
 =?utf-8?q?platform=5Ftriplet_and_multiarch_macros_in_sync?=
Message-ID: <20160829180332.24216.1106.2BA617A1@psf.io>

https://hg.python.org/cpython/rev/5c77488830bc
changeset:   102940:5c77488830bc
user:        doko at ubuntu.com
date:        Mon Aug 29 20:03:25 2016 +0200
summary:
   - Issue #23968, keep platform_triplet and multiarch macros in sync

files:
  configure    |  2 ++
  configure.ac |  2 ++
  2 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -5395,6 +5395,8 @@
   if test x$PLATFORM_TRIPLET != x$MULTIARCH; then
     as_fn_error $? "internal configure error for the platform triplet, please file a bug report" "$LINENO" 5
   fi
+elif test x$PLATFORM_TRIPLET != x && test x$MULTIARCH = x; then
+  MULTIARCH=$PLATFORM_TRIPLET
 fi
 if test x$PLATFORM_TRIPLET = x; then
   PLATDIR=plat-$MACHDEP
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -882,6 +882,8 @@
   if test x$PLATFORM_TRIPLET != x$MULTIARCH; then
     AC_MSG_ERROR([internal configure error for the platform triplet, please file a bug report])
   fi
+elif test x$PLATFORM_TRIPLET != x && test x$MULTIARCH = x; then
+  MULTIARCH=$PLATFORM_TRIPLET
 fi
 if test x$PLATFORM_TRIPLET = x; then
   PLATDIR=plat-$MACHDEP

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Aug 29 14:27:54 2016
From: python-checkins at python.org (mark.dickinson)
Date: Mon, 29 Aug 2016 18:27:54 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327870=3A_A_left_s?=
 =?utf-8?q?hift_of_zero_by_a_large_integer_no_longer_attempts_to?=
Message-ID: <20160829182754.24301.38192.CAD7F802@psf.io>

https://hg.python.org/cpython/rev/09fa42818cf8
changeset:   102941:09fa42818cf8
parent:      102939:1902e1d79e25
user:        Mark Dickinson 
date:        Mon Aug 29 19:27:06 2016 +0100
summary:
  Issue #27870: A left shift of zero by a large integer no longer attempts to allocate large amounts of memory.

files:
  Lib/test/test_long.py |  15 +++++++++++++++
  Misc/NEWS             |   3 +++
  Objects/longobject.c  |   5 +++++
  3 files changed, 23 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_long.py b/Lib/test/test_long.py
--- a/Lib/test/test_long.py
+++ b/Lib/test/test_long.py
@@ -878,6 +878,21 @@
             self.check_truediv(-x, y)
             self.check_truediv(-x, -y)
 
+    def test_lshift_of_zero(self):
+        self.assertEqual(0 << 0, 0)
+        self.assertEqual(0 << 10, 0)
+        with self.assertRaises(ValueError):
+            0 << -1
+
+    @support.cpython_only
+    def test_huge_lshift_of_zero(self):
+        # Shouldn't try to allocate memory for a huge shift. See issue #27870.
+        # Other implementations may have a different boundary for overflow,
+        # or not raise at all.
+        self.assertEqual(0 << sys.maxsize, 0)
+        with self.assertRaises(OverflowError):
+            0 << (sys.maxsize + 1)
+
     def test_small_ints(self):
         for i in range(-5, 257):
             self.assertIs(i, i + 0)
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #27870: A left shift of zero by a large integer no longer attempts
+  to allocate large amounts of memory.
+
 - Issue #25402: In int-to-decimal-string conversion, improve the estimate
   of the intermediate memory required, and remove an unnecessarily strict
   overflow check. Patch by Serhiy Storchaka.
diff --git a/Objects/longobject.c b/Objects/longobject.c
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -4281,6 +4281,11 @@
         PyErr_SetString(PyExc_ValueError, "negative shift count");
         return NULL;
     }
+
+    if (Py_SIZE(a) == 0) {
+        return PyLong_FromLong(0);
+    }
+
     /* wordshift, remshift = divmod(shiftby, PyLong_SHIFT) */
     wordshift = shiftby / PyLong_SHIFT;
     remshift  = shiftby - wordshift * PyLong_SHIFT;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Aug 29 14:27:54 2016
From: python-checkins at python.org (mark.dickinson)
Date: Mon, 29 Aug 2016 18:27:54 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_heads?=
Message-ID: <20160829182754.82639.20905.3457DC98@psf.io>

https://hg.python.org/cpython/rev/7d73555c83ff
changeset:   102942:7d73555c83ff
parent:      102941:09fa42818cf8
parent:      102940:5c77488830bc
user:        Mark Dickinson 
date:        Mon Aug 29 19:27:43 2016 +0100
summary:
  Merge heads

files:
  configure    |  2 ++
  configure.ac |  2 ++
  2 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -5395,6 +5395,8 @@
   if test x$PLATFORM_TRIPLET != x$MULTIARCH; then
     as_fn_error $? "internal configure error for the platform triplet, please file a bug report" "$LINENO" 5
   fi
+elif test x$PLATFORM_TRIPLET != x && test x$MULTIARCH = x; then
+  MULTIARCH=$PLATFORM_TRIPLET
 fi
 if test x$PLATFORM_TRIPLET = x; then
   PLATDIR=plat-$MACHDEP
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -882,6 +882,8 @@
   if test x$PLATFORM_TRIPLET != x$MULTIARCH; then
     AC_MSG_ERROR([internal configure error for the platform triplet, please file a bug report])
   fi
+elif test x$PLATFORM_TRIPLET != x && test x$MULTIARCH = x; then
+  MULTIARCH=$PLATFORM_TRIPLET
 fi
 if test x$PLATFORM_TRIPLET = x; then
   PLATDIR=plat-$MACHDEP

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Aug 29 14:38:21 2016
From: python-checkins at python.org (mark.dickinson)
Date: Mon, 29 Aug 2016 18:38:21 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3ODcw?=
 =?utf-8?q?=3A_A_left_shift_of_zero_by_a_large_integer_no_longer_attempts_?=
 =?utf-8?q?to?=
Message-ID: <20160829183821.1752.31521.A0321AF8@psf.io>

https://hg.python.org/cpython/rev/58ea646ef657
changeset:   102943:58ea646ef657
branch:      2.7
parent:      102933:97dbba8a6d4a
user:        Mark Dickinson 
date:        Mon Aug 29 19:38:12 2016 +0100
summary:
  Issue #27870: A left shift of zero by a large integer no longer attempts to allocate large amounts of memory.

files:
  Lib/test/test_long.py |  15 +++++++++++++++
  Misc/NEWS             |   3 +++
  Objects/longobject.c  |   5 +++++
  3 files changed, 23 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_long.py b/Lib/test/test_long.py
--- a/Lib/test/test_long.py
+++ b/Lib/test/test_long.py
@@ -202,6 +202,21 @@
                 self.assertEqual(x, y,
                     Frm("bad result for a*b: a=%r, b=%r, x=%r, y=%r", a, b, x, y))
 
+    def test_lshift_of_zero(self):
+        self.assertEqual(0L << 0, 0)
+        self.assertEqual(0L << 10, 0)
+        with self.assertRaises(ValueError):
+            0L << -1
+
+    @test_support.cpython_only
+    def test_huge_lshift_of_zero(self):
+        # Shouldn't try to allocate memory for a huge shift. See issue #27870.
+        # Other implementations may have a different boundary for overflow,
+        # or not raise at all.
+        self.assertEqual(0L << sys.maxsize, 0)
+        with self.assertRaises(OverflowError):
+            0L << (sys.maxsize + 1)
+
     def check_bitop_identities_1(self, x):
         eq = self.assertEqual
         eq(x & 0, 0, Frm("x & 0 != 0 for x=%r", x))
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #27870: A left shift of zero by a large integer no longer attempts
+  to allocate large amounts of memory.
+
 - Issue #25604: Fix a minor bug in integer true division; this bug could
   potentially have caused off-by-one-ulp results on platforms with
   unreliable ldexp implementations.
diff --git a/Objects/longobject.c b/Objects/longobject.c
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -3715,6 +3715,11 @@
         PyErr_SetString(PyExc_ValueError, "negative shift count");
         goto lshift_error;
     }
+
+    if (Py_SIZE(a) == 0) {
+        return PyLong_FromLong(0);
+    }
+
     /* wordshift, remshift = divmod(shiftby, PyLong_SHIFT) */
     wordshift = shiftby / PyLong_SHIFT;
     remshift  = shiftby - wordshift * PyLong_SHIFT;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Aug 30 01:24:16 2016
From: python-checkins at python.org (zach.ware)
Date: Tue, 30 Aug 2016 05:24:16 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_with_3=2E5?=
Message-ID: <20160830052413.22114.79637.1B782BEE@psf.io>

https://hg.python.org/cpython/rev/053431aacef6
changeset:   102945:053431aacef6
parent:      102942:7d73555c83ff
parent:      102944:8191730a2911
user:        Zachary Ware 
date:        Tue Aug 30 00:23:37 2016 -0500
summary:
  Merge with 3.5

files:
  Doc/library/unittest.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst
--- a/Doc/library/unittest.rst
+++ b/Doc/library/unittest.rst
@@ -2053,7 +2053,7 @@
    The *failfast*, *catchbreak* and *buffer* parameters have the same
    effect as the same-name `command-line options`_.
 
-   The *warning* argument specifies the :ref:`warning filter `
+   The *warnings* argument specifies the :ref:`warning filter `
    that should be used while running the tests.  If it's not specified, it will
    remain ``None`` if a :option:`-W` option is passed to :program:`python`,
    otherwise it will be set to ``'default'``.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Aug 30 01:24:16 2016
From: python-checkins at python.org (zach.ware)
Date: Tue, 30 Aug 2016 05:24:16 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_typo?=
Message-ID: <20160830052413.2496.18519.97EE1A79@psf.io>

https://hg.python.org/cpython/rev/8191730a2911
changeset:   102944:8191730a2911
branch:      3.5
parent:      102932:c046cbb24f98
user:        Zachary Ware 
date:        Tue Aug 30 00:16:13 2016 -0500
summary:
  Fix typo

files:
  Doc/library/unittest.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst
--- a/Doc/library/unittest.rst
+++ b/Doc/library/unittest.rst
@@ -2053,7 +2053,7 @@
    The *failfast*, *catchbreak* and *buffer* parameters have the same
    effect as the same-name `command-line options`_.
 
-   The *warning* argument specifies the :ref:`warning filter `
+   The *warnings* argument specifies the :ref:`warning filter `
    that should be used while running the tests.  If it's not specified, it will
    remain ``None`` if a :option:`-W` option is passed to :program:`python`,
    otherwise it will be set to ``'default'``.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Aug 30 02:41:21 2016
From: python-checkins at python.org (zach.ware)
Date: Tue, 30 Aug 2016 06:41:21 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Include_test?=
 =?utf-8?q?=5Ftools_and_test=5Fwarnings_in_install?=
Message-ID: <20160830064121.98042.84267.85BB082B@psf.io>

https://hg.python.org/cpython/rev/cbdf907a75c7
changeset:   102946:cbdf907a75c7
branch:      3.5
parent:      102944:8191730a2911
user:        Zachary Ware 
date:        Tue Aug 30 00:28:40 2016 -0500
summary:
  Include test_tools and test_warnings in install

test_tools is pretty much useless unless somebody copies the scripts
it tests into the proper location, but we might as well install it
anyway to make sure it skips properly.  test_warnings just didn't get
included when it was turned into a package in 94966dfd3bd3.

files:
  Makefile.pre.in |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Makefile.pre.in b/Makefile.pre.in
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -1221,6 +1221,7 @@
 		importlib test/test_importlib test/test_importlib/builtin \
 		test/test_importlib/extension test/test_importlib/frozen \
 		test/test_importlib/import_ test/test_importlib/source \
+		test/test_tools test/test_warnings \
 		turtledemo \
 		multiprocessing multiprocessing/dummy \
 		unittest unittest/test unittest/test/testmock \

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Aug 30 02:41:42 2016
From: python-checkins at python.org (zach.ware)
Date: Tue, 30 Aug 2016 06:41:42 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_with_3=2E5?=
Message-ID: <20160830064142.2663.40275.69CA0781@psf.io>

https://hg.python.org/cpython/rev/dab6a7c34e1f
changeset:   102947:dab6a7c34e1f
parent:      102945:053431aacef6
parent:      102946:cbdf907a75c7
user:        Zachary Ware 
date:        Tue Aug 30 01:41:35 2016 -0500
summary:
  Merge with 3.5

files:
  Makefile.pre.in |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Makefile.pre.in b/Makefile.pre.in
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -1202,6 +1202,7 @@
 		importlib test/test_importlib test/test_importlib/builtin \
 		test/test_importlib/extension test/test_importlib/frozen \
 		test/test_importlib/import_ test/test_importlib/source \
+		test/test_tools test/test_warnings \
 		turtledemo \
 		multiprocessing multiprocessing/dummy \
 		unittest unittest/test unittest/test/testmock \

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Aug 30 03:12:52 2016
From: python-checkins at python.org (zach.ware)
Date: Tue, 30 Aug 2016 07:12:52 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Include_test/t?=
 =?utf-8?q?est=5Fwarnings/data_as_well?=
Message-ID: <20160830071252.14052.46914.BF2BD24C@psf.io>

https://hg.python.org/cpython/rev/ad9c2492556b
changeset:   102948:ad9c2492556b
branch:      3.5
parent:      102946:cbdf907a75c7
user:        Zachary Ware 
date:        Tue Aug 30 02:12:27 2016 -0500
summary:
  Include test/test_warnings/data as well

files:
  Makefile.pre.in |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Makefile.pre.in b/Makefile.pre.in
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -1221,7 +1221,7 @@
 		importlib test/test_importlib test/test_importlib/builtin \
 		test/test_importlib/extension test/test_importlib/frozen \
 		test/test_importlib/import_ test/test_importlib/source \
-		test/test_tools test/test_warnings \
+		test/test_tools test/test_warnings test/test_warnings/data \
 		turtledemo \
 		multiprocessing multiprocessing/dummy \
 		unittest unittest/test unittest/test/testmock \

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Aug 30 03:12:53 2016
From: python-checkins at python.org (zach.ware)
Date: Tue, 30 Aug 2016 07:12:53 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_with_3=2E5?=
Message-ID: <20160830071252.16660.5599.A8973A77@psf.io>

https://hg.python.org/cpython/rev/14f042958c32
changeset:   102949:14f042958c32
parent:      102947:dab6a7c34e1f
parent:      102948:ad9c2492556b
user:        Zachary Ware 
date:        Tue Aug 30 02:12:43 2016 -0500
summary:
  Merge with 3.5

files:
  Makefile.pre.in |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Makefile.pre.in b/Makefile.pre.in
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -1202,7 +1202,7 @@
 		importlib test/test_importlib test/test_importlib/builtin \
 		test/test_importlib/extension test/test_importlib/frozen \
 		test/test_importlib/import_ test/test_importlib/source \
-		test/test_tools test/test_warnings \
+		test/test_tools test/test_warnings test/test_warnings/data \
 		turtledemo \
 		multiprocessing multiprocessing/dummy \
 		unittest unittest/test unittest/test/testmock \

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Tue Aug 30 08:56:28 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Tue, 30 Aug 2016 13:56:28 +0100
Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-08-30
Message-ID: 

Results for project Python 2.7, build date 2016-08-30 02:47:19 +0000
commit:		58ea646ef657
previous commit:	55ec5fdc3099
revision date:	2016-08-29 18:38:12 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.12%         -0.21%          4.84%             5.69%
:-)             pybench      0.21%          0.17%          6.36%             4.26%
:-(            regex_v8      0.60%         -0.27%         -2.35%            11.28%
:-)               nbody      0.08%          0.52%          8.17%             3.09%
:-|        json_dump_v2      0.64%         -1.94%          1.66%            10.51%
:-|      normal_startup      0.61%         -0.22%         -0.80%             2.00%
:-)             ssbench      0.16%          0.16%          2.88%             0.88%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-08-30/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Tue Aug 30 08:58:16 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Tue, 30 Aug 2016 13:58:16 +0100
Subject: [Python-checkins] BAD Benchmark Results for Python Default
 2016-08-30
Message-ID: 

Results for project Python default, build date 2016-08-30 02:00:36 +0000
commit:		7d73555c83ff
previous commit:	8ec5a00e5d75
revision date:	2016-08-29 18:27:43 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.19%         -1.23%          9.32%            16.73%
:-)             pybench      3.12%         -0.60%          6.27%             7.82%
:-(            regex_v8      2.77%          0.13%         -3.83%             5.25%
:-(               nbody      0.09%         -2.93%         -1.30%            12.07%
:-(        json_dump_v2      0.36%         -1.03%         -2.47%            12.49%
:-)      normal_startup      0.76%         -0.49%          2.05%             6.15%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-default-2016-08-30/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Tue Aug 30 11:03:46 2016
From: python-checkins at python.org (zach.ware)
Date: Tue, 30 Aug 2016 15:03:46 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogU2tpcCB0ZXN0X3Rv?=
 =?utf-8?q?ols=2Etest=5Fi18n_when_pygettext=2Epy_is_missing?=
Message-ID: <20160830150346.14072.3832.7AD8747A@psf.io>

https://hg.python.org/cpython/rev/53f2cde033ac
changeset:   102950:53f2cde033ac
branch:      3.5
parent:      102948:ad9c2492556b
user:        Zachary Ware 
date:        Tue Aug 30 10:00:26 2016 -0500
summary:
  Skip test_tools.test_i18n when pygettext.py is missing

files:
  Lib/test/test_tools/test_i18n.py |  6 +++++-
  1 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_tools/test_i18n.py b/Lib/test/test_tools/test_i18n.py
--- a/Lib/test/test_tools/test_i18n.py
+++ b/Lib/test/test_tools/test_i18n.py
@@ -4,9 +4,13 @@
 import unittest
 
 from test.support.script_helper import assert_python_ok
-from test.test_tools import toolsdir
+from test.test_tools import skip_if_missing, toolsdir
 from test.support import temp_cwd
 
+
+skip_if_missing()
+
+
 class Test_pygettext(unittest.TestCase):
     """Tests for the pygettext.py tool"""
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Aug 30 11:03:46 2016
From: python-checkins at python.org (zach.ware)
Date: Tue, 30 Aug 2016 15:03:46 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_with_3=2E5?=
Message-ID: <20160830150346.12898.65226.9BCCD6E2@psf.io>

https://hg.python.org/cpython/rev/552547e92707
changeset:   102951:552547e92707
parent:      102949:14f042958c32
parent:      102950:53f2cde033ac
user:        Zachary Ware 
date:        Tue Aug 30 10:03:32 2016 -0500
summary:
  Merge with 3.5

files:
  Lib/test/test_tools/test_i18n.py |  6 +++++-
  1 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_tools/test_i18n.py b/Lib/test/test_tools/test_i18n.py
--- a/Lib/test/test_tools/test_i18n.py
+++ b/Lib/test/test_tools/test_i18n.py
@@ -4,9 +4,13 @@
 import unittest
 
 from test.support.script_helper import assert_python_ok
-from test.test_tools import toolsdir
+from test.test_tools import skip_if_missing, toolsdir
 from test.support import temp_cwd
 
+
+skip_if_missing()
+
+
 class Test_pygettext(unittest.TestCase):
     """Tests for the pygettext.py tool"""
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Aug 30 13:48:02 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 30 Aug 2016 17:48:02 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327895=3A__Spellin?=
 =?utf-8?q?g_fixes_=28Contributed_by_Ville_Skytt=C3=A4=29=2E?=
Message-ID: <20160830174759.16660.76177.32971B99@psf.io>

https://hg.python.org/cpython/rev/75d6d5d9b674
changeset:   102952:75d6d5d9b674
user:        Raymond Hettinger 
date:        Tue Aug 30 10:47:49 2016 -0700
summary:
  Issue #27895:  Spelling fixes (Contributed by Ville Skytt?).

files:
  Doc/library/datetime.rst                   |   2 +-
  Doc/library/email.contentmanager.rst       |   2 +-
  Doc/library/idle.rst                       |   2 +-
  Doc/library/smtpd.rst                      |   4 +-
  Doc/whatsnew/3.3.rst                       |   2 +-
  Include/abstract.h                         |   2 +-
  Include/bytesobject.h                      |   2 +-
  Include/pymath.h                           |   2 +-
  Lib/asyncio/streams.py                     |   2 +-
  Lib/concurrent/futures/process.py          |   2 +-
  Lib/concurrent/futures/thread.py           |   2 +-
  Lib/distutils/tests/test_msvc9compiler.py  |   2 +-
  Lib/email/contentmanager.py                |   6 +-
  Lib/email/generator.py                     |   2 +-
  Lib/email/header.py                        |   4 +-
  Lib/email/message.py                       |   2 +-
  Lib/http/client.py                         |   2 +-
  Lib/idlelib/README.txt                     |   2 +-
  Lib/idlelib/help.html                      |   2 +-
  Lib/idlelib/idle_test/test_paragraph.py    |   2 +-
  Lib/shutil.py                              |   2 +-
  Lib/statistics.py                          |   2 +-
  Lib/test/_test_multiprocessing.py          |   2 +-
  Lib/test/datetimetester.py                 |   2 +-
  Lib/test/test_asyncio/test_locks.py        |   4 +-
  Lib/test/test_concurrent_futures.py        |   2 +-
  Lib/test/test_descr.py                     |   2 +-
  Lib/test/test_difflib.py                   |  12 ++--
  Lib/test/test_difflib_expect.html          |  12 ++--
  Lib/test/test_email/test_email.py          |   4 +-
  Lib/test/test_email/test_generator.py      |   2 +-
  Lib/test/test_importlib/test_util.py       |   2 +-
  Lib/test/test_ipaddress.py                 |   6 +-
  Lib/test/test_pep247.py                    |   2 +-
  Lib/test/test_shutil.py                    |   4 +-
  Lib/test/test_subprocess.py                |   2 +-
  Lib/test/test_urllib.py                    |   2 +-
  Lib/test/test_winreg.py                    |   2 +-
  Lib/tkinter/__init__.py                    |   2 +-
  Lib/unittest/test/test_discovery.py        |   2 +-
  Lib/unittest/test/testmock/testcallable.py |   2 +-
  Lib/venv/scripts/posix/activate            |   2 +-
  Lib/venv/scripts/posix/activate.csh        |   2 +-
  Lib/venv/scripts/posix/activate.fish       |   2 +-
  Mac/PythonLauncher/MyAppDelegate.m         |   2 +-
  Misc/HISTORY                               |  22 +++++-----
  Misc/NEWS                                  |  12 ++--
  Modules/_ctypes/ctypes.h                   |   2 +-
  Modules/_hashopenssl.c                     |   2 +-
  Modules/_io/iobase.c                       |   2 +-
  Modules/_pickle.c                          |   4 +-
  Modules/_testcapimodule.c                  |   2 +-
  Modules/_threadmodule.c                    |   2 +-
  Modules/_tracemalloc.c                     |   2 +-
  Modules/binascii.c                         |   4 +-
  Modules/mathmodule.c                       |   2 +-
  Modules/socketmodule.c                     |   2 +-
  Modules/zipimport.c                        |   2 +-
  Objects/bytearrayobject.c                  |   2 +-
  Objects/bytesobject.c                      |   4 +-
  Objects/codeobject.c                       |   2 +-
  Objects/listsort.txt                       |   2 +-
  Objects/longobject.c                       |   2 +-
  Objects/stringlib/codecs.h                 |   6 +-
  Objects/typeobject.c                       |   2 +-
  Objects/unicodeobject.c                    |  14 +++---
  Python/ceval.c                             |  12 ++--
  Python/condvar.h                           |   2 +-
  Python/formatter_unicode.c                 |   2 +-
  README                                     |   2 +-
  configure                                  |   2 +-
  configure.ac                               |   2 +-
  72 files changed, 122 insertions(+), 122 deletions(-)


diff --git a/Doc/library/datetime.rst b/Doc/library/datetime.rst
--- a/Doc/library/datetime.rst
+++ b/Doc/library/datetime.rst
@@ -1842,7 +1842,7 @@
 :attr:`~datetime.fold` attribute are considered equal in comparisons.
 
 Applications that can't bear wall-time ambiguities should explicitly check the
-value of the :attr:`~datetime.fold` atribute or avoid using hybrid
+value of the :attr:`~datetime.fold` attribute or avoid using hybrid
 :class:`tzinfo` subclasses; there are no ambiguities when using :class:`timezone`,
 or any other fixed-offset :class:`tzinfo` subclass (such as a class representing
 only EST (fixed offset -5 hours), or only EDT (fixed offset -4 hours)).
diff --git a/Doc/library/email.contentmanager.rst b/Doc/library/email.contentmanager.rst
--- a/Doc/library/email.contentmanager.rst
+++ b/Doc/library/email.contentmanager.rst
@@ -433,5 +433,5 @@
 
        If *headers* is specified and is a list of strings of the form
        ``headername: headervalue`` or a list of ``header`` objects
-       (distinguised from strings by having a ``name`` attribute), add the
+       (distinguished from strings by having a ``name`` attribute), add the
        headers to *msg*.
diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst
--- a/Doc/library/idle.rst
+++ b/Doc/library/idle.rst
@@ -531,7 +531,7 @@
    -c command  run command in the shell window
    -d          enable debugger and open shell window
    -e          open editor window
-   -h          print help message with legal combinatios and exit
+   -h          print help message with legal combinations and exit
    -i          open shell window
    -r file     run file in shell window
    -s          run $IDLESTARTUP or $PYTHONSTARTUP first, in shell window
diff --git a/Doc/library/smtpd.rst b/Doc/library/smtpd.rst
--- a/Doc/library/smtpd.rst
+++ b/Doc/library/smtpd.rst
@@ -44,7 +44,7 @@
    dictionary is a suitable value).  If not specified the :mod:`asyncore`
    global socket map is used.
 
-   *enable_SMTPUTF8* determins whether the ``SMTPUTF8`` extension (as defined
+   *enable_SMTPUTF8* determines whether the ``SMTPUTF8`` extension (as defined
    in :RFC:`6531`) should be enabled.  The default is ``False``.
    When ``True``, ``SMTPUTF8`` is accepted as a parameter to the ``MAIL``
    command and when present is passed to :meth:`process_message` in the
@@ -162,7 +162,7 @@
    accepted in a ``DATA`` command.  A value of ``None`` or ``0`` means no
    limit.
 
-   *enable_SMTPUTF8* determins whether the ``SMTPUTF8`` extension (as defined
+   *enable_SMTPUTF8* determines whether the ``SMTPUTF8`` extension (as defined
    in :RFC:`6531`) should be enabled.  The default is ``False``.
    *decode_data* and *enable_SMTPUTF8* cannot be set to ``True`` at the same
    time.
diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst
--- a/Doc/whatsnew/3.3.rst
+++ b/Doc/whatsnew/3.3.rst
@@ -1954,7 +1954,7 @@
   :attr:`~ssl.OP_NO_COMPRESSION` can be used to disable compression.
   (Contributed by Antoine Pitrou in :issue:`13634`.)
 
-* Support has been added for the Next Procotol Negotiation extension using
+* Support has been added for the Next Protocol Negotiation extension using
   the :meth:`ssl.SSLContext.set_npn_protocols` method.
   (Contributed by Colin Marc in :issue:`14204`.)
 
diff --git a/Include/abstract.h b/Include/abstract.h
--- a/Include/abstract.h
+++ b/Include/abstract.h
@@ -487,7 +487,7 @@
     /* old buffer API
        FIXME:  usage of these should all be replaced in Python itself
        but for backwards compatibility we will implement them.
-       Their usage without a corresponding "unlock" mechansim
+       Their usage without a corresponding "unlock" mechanism
        may create issues (but they would already be there). */
 
      PyAPI_FUNC(int) PyObject_AsCharBuffer(PyObject *obj,
diff --git a/Include/bytesobject.h b/Include/bytesobject.h
--- a/Include/bytesobject.h
+++ b/Include/bytesobject.h
@@ -131,7 +131,7 @@
 #define F_ZERO	(1<<4)
 
 #ifndef Py_LIMITED_API
-/* The _PyBytesWriter structure is big: it contains an embeded "stack buffer".
+/* The _PyBytesWriter structure is big: it contains an embedded "stack buffer".
    A _PyBytesWriter variable must be declared at the end of variables in a
    function to optimize the memory allocation on the stack. */
 typedef struct {
diff --git a/Include/pymath.h b/Include/pymath.h
--- a/Include/pymath.h
+++ b/Include/pymath.h
@@ -37,7 +37,7 @@
 #endif /* __STDC__ */
 #endif /* _MSC_VER */
 
-/* High precision defintion of pi and e (Euler)
+/* High precision definition of pi and e (Euler)
  * The values are taken from libc6's math.h.
  */
 #ifndef Py_MATH_PIl
diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py
--- a/Lib/asyncio/streams.py
+++ b/Lib/asyncio/streams.py
@@ -590,7 +590,7 @@
         bytes. If the EOF was received and the internal buffer is empty, return
         an empty bytes object.
 
-        If n is zero, return empty bytes object immediatelly.
+        If n is zero, return empty bytes object immediately.
 
         If n is positive, this function try to read `n` bytes, and may return
         less or equal bytes than requested, but at least one byte. If EOF was
diff --git a/Lib/concurrent/futures/process.py b/Lib/concurrent/futures/process.py
--- a/Lib/concurrent/futures/process.py
+++ b/Lib/concurrent/futures/process.py
@@ -63,7 +63,7 @@
 # interpreter to exit when there are still idle processes in a
 # ProcessPoolExecutor's process pool (i.e. shutdown() was not called). However,
 # allowing workers to die with the interpreter has two undesirable properties:
-#   - The workers would still be running during interpretor shutdown,
+#   - The workers would still be running during interpreter shutdown,
 #     meaning that they would fail in unpredictable ways.
 #   - The workers could be killed while evaluating a work item, which could
 #     be bad if the callable being evaluated has external side-effects e.g.
diff --git a/Lib/concurrent/futures/thread.py b/Lib/concurrent/futures/thread.py
--- a/Lib/concurrent/futures/thread.py
+++ b/Lib/concurrent/futures/thread.py
@@ -16,7 +16,7 @@
 # to exit when there are still idle threads in a ThreadPoolExecutor's thread
 # pool (i.e. shutdown() was not called). However, allowing workers to die with
 # the interpreter has two undesirable properties:
-#   - The workers would still be running during interpretor shutdown,
+#   - The workers would still be running during interpreter shutdown,
 #     meaning that they would fail in unpredictable ways.
 #   - The workers could be killed while evaluating a work item, which could
 #     be bad if the callable being evaluated has external side-effects e.g.
diff --git a/Lib/distutils/tests/test_msvc9compiler.py b/Lib/distutils/tests/test_msvc9compiler.py
--- a/Lib/distutils/tests/test_msvc9compiler.py
+++ b/Lib/distutils/tests/test_msvc9compiler.py
@@ -125,7 +125,7 @@
         self.assertRaises(KeyError, Reg.get_value, 'xxx', 'xxx')
 
         # looking for values that should exist on all
-        # windows registeries versions.
+        # windows registry versions.
         path = r'Control Panel\Desktop'
         v = Reg.get_value(path, 'dragfullwindows')
         self.assertIn(v, ('0', '1', '2'))
diff --git a/Lib/email/contentmanager.py b/Lib/email/contentmanager.py
--- a/Lib/email/contentmanager.py
+++ b/Lib/email/contentmanager.py
@@ -141,7 +141,7 @@
 def _encode_text(string, charset, cte, policy):
     lines = string.encode(charset).splitlines()
     linesep = policy.linesep.encode('ascii')
-    def embeded_body(lines): return linesep.join(lines) + linesep
+    def embedded_body(lines): return linesep.join(lines) + linesep
     def normal_body(lines): return b'\n'.join(lines) + b'\n'
     if cte==None:
         # Use heuristics to decide on the "best" encoding.
@@ -152,7 +152,7 @@
         if (policy.cte_type == '8bit' and
                 max(len(x) for x in lines) <= policy.max_line_length):
             return '8bit', normal_body(lines).decode('ascii', 'surrogateescape')
-        sniff = embeded_body(lines[:10])
+        sniff = embedded_body(lines[:10])
         sniff_qp = quoprimime.body_encode(sniff.decode('latin-1'),
                                           policy.max_line_length)
         sniff_base64 = binascii.b2a_base64(sniff)
@@ -171,7 +171,7 @@
         data = quoprimime.body_encode(normal_body(lines).decode('latin-1'),
                                       policy.max_line_length)
     elif cte == 'base64':
-        data = _encode_base64(embeded_body(lines), policy.max_line_length)
+        data = _encode_base64(embedded_body(lines), policy.max_line_length)
     else:
         raise ValueError("Unknown content transfer encoding {}".format(cte))
     return cte, data
diff --git a/Lib/email/generator.py b/Lib/email/generator.py
--- a/Lib/email/generator.py
+++ b/Lib/email/generator.py
@@ -97,7 +97,7 @@
         self._NL = policy.linesep
         self._encoded_NL = self._encode(self._NL)
         self._EMPTY = ''
-        self._encoded_EMTPY = self._encode('')
+        self._encoded_EMPTY = self._encode('')
         # Because we use clone (below) when we recursively process message
         # subparts, and because clone uses the computed policy (not None),
         # submessages will automatically get set to the computed policy when
diff --git a/Lib/email/header.py b/Lib/email/header.py
--- a/Lib/email/header.py
+++ b/Lib/email/header.py
@@ -49,7 +49,7 @@
 
 # Find a header embedded in a putative header value.  Used to check for
 # header injection attack.
-_embeded_header = re.compile(r'\n[^ \t]+:')
+_embedded_header = re.compile(r'\n[^ \t]+:')
 
 
 

@@ -385,7 +385,7 @@
         if self._chunks:
             formatter.add_transition()
         value = formatter._str(linesep)
-        if _embeded_header.search(value):
+        if _embedded_header.search(value):
             raise HeaderParseError("header value appears to contain "
                 "an embedded header: {!r}".format(value))
         return value
diff --git a/Lib/email/message.py b/Lib/email/message.py
--- a/Lib/email/message.py
+++ b/Lib/email/message.py
@@ -1043,7 +1043,7 @@
             yield from parts
             return
         # Otherwise we more or less invert the remaining logic in get_body.
-        # This only really works in edge cases (ex: non-text relateds or
+        # This only really works in edge cases (ex: non-text related or
         # alternatives) if the sending agent sets content-disposition.
         seen = []   # Only skip the first example of each candidate type.
         for part in parts:
diff --git a/Lib/http/client.py b/Lib/http/client.py
--- a/Lib/http/client.py
+++ b/Lib/http/client.py
@@ -136,7 +136,7 @@
 #
 # VCHAR defined in http://tools.ietf.org/html/rfc5234#appendix-B.1
 
-# the patterns for both name and value are more leniant than RFC
+# the patterns for both name and value are more lenient than RFC
 # definitions to allow for backwards compatibility
 _is_legal_header_name = re.compile(rb'[^:\s][^:\r\n]*').fullmatch
 _is_illegal_header_value = re.compile(rb'\n(?![ \t])|\r(?![ \t\n])').search
diff --git a/Lib/idlelib/README.txt b/Lib/idlelib/README.txt
--- a/Lib/idlelib/README.txt
+++ b/Lib/idlelib/README.txt
@@ -65,7 +65,7 @@
 percolator.py     # Manage delegator stack (nim).
 pyparse.py        # Give information on code indentation
 pyshell.py        # Start IDLE, manage shell, complete editor window
-query.py          # Query user for informtion
+query.py          # Query user for information
 redirector.py     # Intercept widget subcommands (for percolator) (nim).
 replace.py        # Search and replace pattern in text.
 rpc.py            # Commuicate between idle and user processes (nim).
diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html
--- a/Lib/idlelib/help.html
+++ b/Lib/idlelib/help.html
@@ -497,7 +497,7 @@
 -c command  run command in the shell window
 -d          enable debugger and open shell window
 -e          open editor window
--h          print help message with legal combinatios and exit
+-h          print help message with legal combinations and exit
 -i          open shell window
 -r file     run file in shell window
 -s          run $IDLESTARTUP or $PYTHONSTARTUP first, in shell window
diff --git a/Lib/idlelib/idle_test/test_paragraph.py b/Lib/idlelib/idle_test/test_paragraph.py
--- a/Lib/idlelib/idle_test/test_paragraph.py
+++ b/Lib/idlelib/idle_test/test_paragraph.py
@@ -159,7 +159,7 @@
 class ReformatFunctionTest(unittest.TestCase):
     """Test the reformat_paragraph function without the editor window."""
 
-    def test_reformat_paragrah(self):
+    def test_reformat_paragraph(self):
         Equal = self.assertEqual
         reform = fp.reformat_paragraph
         hw = "O hello world"
diff --git a/Lib/shutil.py b/Lib/shutil.py
--- a/Lib/shutil.py
+++ b/Lib/shutil.py
@@ -64,7 +64,7 @@
 
 class RegistryError(Exception):
     """Raised when a registry operation with the archiving
-    and unpacking registeries fails"""
+    and unpacking registries fails"""
 
 
 def copyfileobj(fsrc, fdst, length=16*1024):
diff --git a/Lib/statistics.py b/Lib/statistics.py
--- a/Lib/statistics.py
+++ b/Lib/statistics.py
@@ -454,7 +454,7 @@
         """Return the nth root of a positive huge number."""
         assert x > 0
         # I state without proof that ??x ? ??2???(x//2)
-        # and that for sufficiently big x the error is acceptible.
+        # and that for sufficiently big x the error is acceptable.
         # We now halve x until it is small enough to get the root.
         m = 0
         while True:
diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py
--- a/Lib/test/_test_multiprocessing.py
+++ b/Lib/test/_test_multiprocessing.py
@@ -26,7 +26,7 @@
 _multiprocessing = test.support.import_module('_multiprocessing')
 # Skip tests if sem_open implementation is broken.
 test.support.import_module('multiprocessing.synchronize')
-# import threading after _multiprocessing to raise a more revelant error
+# import threading after _multiprocessing to raise a more relevant error
 # message: "No module named _multiprocessing". _multiprocessing is not compiled
 # without thread support.
 import threading
diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py
--- a/Lib/test/datetimetester.py
+++ b/Lib/test/datetimetester.py
@@ -3958,7 +3958,7 @@
         self.assertRaises(TypeError, lambda: as_date >= as_datetime)
         self.assertRaises(TypeError, lambda: as_datetime >= as_date)
 
-        # Neverthelss, comparison should work with the base-class (date)
+        # Nevertheless, comparison should work with the base-class (date)
         # projection if use of a date method is forced.
         self.assertEqual(as_date.__eq__(as_datetime), True)
         different_day = (as_date.day + 1) % 20 + 1
diff --git a/Lib/test/test_asyncio/test_locks.py b/Lib/test/test_asyncio/test_locks.py
--- a/Lib/test/test_asyncio/test_locks.py
+++ b/Lib/test/test_asyncio/test_locks.py
@@ -130,8 +130,8 @@
     def test_cancel_race(self):
         # Several tasks:
         # - A acquires the lock
-        # - B is blocked in aqcuire()
-        # - C is blocked in aqcuire()
+        # - B is blocked in acquire()
+        # - C is blocked in acquire()
         #
         # Now, concurrently:
         # - B is cancelled
diff --git a/Lib/test/test_concurrent_futures.py b/Lib/test/test_concurrent_futures.py
--- a/Lib/test/test_concurrent_futures.py
+++ b/Lib/test/test_concurrent_futures.py
@@ -4,7 +4,7 @@
 test.support.import_module('_multiprocessing')
 # Skip tests if sem_open implementation is broken.
 test.support.import_module('multiprocessing.synchronize')
-# import threading after _multiprocessing to raise a more revelant error
+# import threading after _multiprocessing to raise a more relevant error
 # message: "No module named _multiprocessing". _multiprocessing is not compiled
 # without thread support.
 test.support.import_module('threading')
diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py
--- a/Lib/test/test_descr.py
+++ b/Lib/test/test_descr.py
@@ -876,7 +876,7 @@
         self.assertEqual(Frag().__int__(), 42)
         self.assertEqual(int(Frag()), 42)
 
-    def test_diamond_inheritence(self):
+    def test_diamond_inheritance(self):
         # Testing multiple inheritance special cases...
         class A(object):
             def spam(self): return "A"
diff --git a/Lib/test/test_difflib.py b/Lib/test/test_difflib.py
--- a/Lib/test/test_difflib.py
+++ b/Lib/test/test_difflib.py
@@ -122,17 +122,17 @@
 """
 
 patch914575_from2 = """
-\t\tLine 1: preceeded by from:[tt] to:[ssss]
-  \t\tLine 2: preceeded by from:[sstt] to:[sssst]
-  \t \tLine 3: preceeded by from:[sstst] to:[ssssss]
+\t\tLine 1: preceded by from:[tt] to:[ssss]
+  \t\tLine 2: preceded by from:[sstt] to:[sssst]
+  \t \tLine 3: preceded by from:[sstst] to:[ssssss]
 Line 4:  \thas from:[sst] to:[sss] after :
 Line 5: has from:[t] to:[ss] at end\t
 """
 
 patch914575_to2 = """
-    Line 1: preceeded by from:[tt] to:[ssss]
-    \tLine 2: preceeded by from:[sstt] to:[sssst]
-      Line 3: preceeded by from:[sstst] to:[ssssss]
+    Line 1: preceded by from:[tt] to:[ssss]
+    \tLine 2: preceded by from:[sstt] to:[sssst]
+      Line 3: preceded by from:[sstst] to:[ssssss]
 Line 4:   has from:[sst] to:[sss] after :
 Line 5: has from:[t] to:[ss] at end
 """
diff --git a/Lib/test/test_difflib_expect.html b/Lib/test/test_difflib_expect.html
--- a/Lib/test/test_difflib_expect.html
+++ b/Lib/test/test_difflib_expect.html
@@ -387,9 +387,9 @@
         
         
             f1f1
-            t2    Line 1: preceeded by from:[tt] to:[ssss]t2    Line 1: preceeded by from:[tt] to:[ssss]
-            3      Line 2: preceeded by from:[sstt] to:[sssst]3      Line 2: preceeded by from:[sstt] to:[sssst]
-            4      Line 3: preceeded by from:[sstst] to:[ssssss]4      Line 3: preceeded by from:[sstst] to:[ssssss]
+            t2    Line 1: preceded by from:[tt] to:[ssss]t2    Line 1: preceded by from:[tt] to:[ssss]
+            3      Line 2: preceded by from:[sstt] to:[sssst]3      Line 2: preceded by from:[sstt] to:[sssst]
+            4      Line 3: preceded by from:[sstst] to:[ssssss]4      Line 3: preceded by from:[sstst] to:[ssssss]
             5Line 4:   has from:[sst] to:[sss] after :5Line 4:   has from:[sst] to:[sss] after :
             6Line 5: has from:[t] to:[ss] at end 6Line 5: has from:[t] to:[ss] at end
         
@@ -403,9 +403,9 @@
         
         
             f1f1
-            t2                Line 1: preceeded by from:[tt] to:[ssss]t2    Line 1: preceeded by from:[tt] to:[ssss]
-            3                Line 2: preceeded by from:[sstt] to:[sssst]3        Line 2: preceeded by from:[sstt] to:[sssst]
-            4                Line 3: preceeded by from:[sstst] to:[ssssss]4      Line 3: preceeded by from:[sstst] to:[ssssss]
+            t2                Line 1: preceded by from:[tt] to:[ssss]t2    Line 1: preceded by from:[tt] to:[ssss]
+            3                Line 2: preceded by from:[sstt] to:[sssst]3        Line 2: preceded by from:[sstt] to:[sssst]
+            4                Line 3: preceded by from:[sstst] to:[ssssss]4      Line 3: preceded by from:[sstst] to:[ssssss]
             5Line 4:         has from:[sst] to:[sss] after :5Line 4:   has from:[sst] to:[sss] after :
             6Line 5: has from:[t] to:[ss] at end     6Line 5: has from:[t] to:[ss] at end
         
diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py
--- a/Lib/test/test_email/test_email.py
+++ b/Lib/test/test_email/test_email.py
@@ -723,12 +723,12 @@
 
     # Issue 5871: reject an attempt to embed a header inside a header value
     # (header injection attack).
-    def test_embeded_header_via_Header_rejected(self):
+    def test_embedded_header_via_Header_rejected(self):
         msg = Message()
         msg['Dummy'] = Header('dummy\nX-Injected-Header: test')
         self.assertRaises(errors.HeaderParseError, msg.as_string)
 
-    def test_embeded_header_via_string_rejected(self):
+    def test_embedded_header_via_string_rejected(self):
         msg = Message()
         msg['Dummy'] = 'dummy\nX-Injected-Header: test'
         self.assertRaises(errors.HeaderParseError, msg.as_string)
diff --git a/Lib/test/test_email/test_generator.py b/Lib/test/test_email/test_generator.py
--- a/Lib/test/test_email/test_generator.py
+++ b/Lib/test/test_email/test_generator.py
@@ -143,7 +143,7 @@
     def test_set_mangle_from_via_policy(self):
         source = textwrap.dedent("""\
             Subject: test that
-             from is mangeld in the body!
+             from is mangled in the body!
 
             From time to time I write a rhyme.
             """)
diff --git a/Lib/test/test_importlib/test_util.py b/Lib/test/test_importlib/test_util.py
--- a/Lib/test/test_importlib/test_util.py
+++ b/Lib/test/test_importlib/test_util.py
@@ -372,7 +372,7 @@
         # bacon
         self.assertEqual('bacon', self.util.resolve_name('bacon', None))
 
-    def test_aboslute_within_package(self):
+    def test_absolute_within_package(self):
         # bacon in spam
         self.assertEqual('bacon', self.util.resolve_name('bacon', 'spam'))
 
diff --git a/Lib/test/test_ipaddress.py b/Lib/test/test_ipaddress.py
--- a/Lib/test/test_ipaddress.py
+++ b/Lib/test/test_ipaddress.py
@@ -1263,7 +1263,7 @@
         ip4 = ipaddress.IPv4Address('1.1.1.3')
         ip5 = ipaddress.IPv4Address('1.1.1.4')
         ip6 = ipaddress.IPv4Address('1.1.1.0')
-        # check that addreses are subsumed properly.
+        # check that addresses are subsumed properly.
         collapsed = ipaddress.collapse_addresses(
             [ip1, ip2, ip3, ip4, ip5, ip6])
         self.assertEqual(list(collapsed),
@@ -1277,7 +1277,7 @@
         ip4 = ipaddress.IPv4Address('1.1.1.3')
         #ip5 = ipaddress.IPv4Interface('1.1.1.4/30')
         #ip6 = ipaddress.IPv4Interface('1.1.1.4/30')
-        # check that addreses are subsumed properly.
+        # check that addresses are subsumed properly.
         collapsed = ipaddress.collapse_addresses([ip1, ip2, ip3, ip4])
         self.assertEqual(list(collapsed),
                          [ipaddress.IPv4Network('1.1.1.0/30')])
@@ -1291,7 +1291,7 @@
         # stored in no particular order b/c we want CollapseAddr to call
         # [].sort
         ip6 = ipaddress.IPv4Network('1.1.0.0/22')
-        # check that addreses are subsumed properly.
+        # check that addresses are subsumed properly.
         collapsed = ipaddress.collapse_addresses([ip1, ip2, ip3, ip4, ip5,
                                                      ip6])
         self.assertEqual(list(collapsed),
diff --git a/Lib/test/test_pep247.py b/Lib/test/test_pep247.py
--- a/Lib/test/test_pep247.py
+++ b/Lib/test/test_pep247.py
@@ -1,5 +1,5 @@
 """
-Test suite to check compilance with PEP 247, the standard API
+Test suite to check compliance with PEP 247, the standard API
 for hashing algorithms
 """
 
diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py
--- a/Lib/test/test_shutil.py
+++ b/Lib/test/test_shutil.py
@@ -1306,10 +1306,10 @@
             shutil.chown(filename)
 
         with self.assertRaises(LookupError):
-            shutil.chown(filename, user='non-exising username')
+            shutil.chown(filename, user='non-existing username')
 
         with self.assertRaises(LookupError):
-            shutil.chown(filename, group='non-exising groupname')
+            shutil.chown(filename, group='non-existing groupname')
 
         with self.assertRaises(TypeError):
             shutil.chown(filename, b'spam')
diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -682,7 +682,7 @@
         self.assertEqual(stdout, "banana")
         self.assertStderrEqual(stderr.encode(), b"pineapple\npear\n")
 
-    def test_communicate_timeout_large_ouput(self):
+    def test_communicate_timeout_large_output(self):
         # Test an expiring timeout while the child is outputting lots of data.
         p = subprocess.Popen([sys.executable, "-c",
                               'import sys,os,time;'
diff --git a/Lib/test/test_urllib.py b/Lib/test/test_urllib.py
--- a/Lib/test/test_urllib.py
+++ b/Lib/test/test_urllib.py
@@ -1,4 +1,4 @@
-"""Regresssion tests for what was in Python 2's "urllib" module"""
+"""Regression tests for what was in Python 2's "urllib" module"""
 
 import urllib.parse
 import urllib.request
diff --git a/Lib/test/test_winreg.py b/Lib/test/test_winreg.py
--- a/Lib/test/test_winreg.py
+++ b/Lib/test/test_winreg.py
@@ -169,7 +169,7 @@
         DeleteKey(key, subkeystr)
 
         try:
-            # Shouldnt be able to delete it twice!
+            # Shouldn't be able to delete it twice!
             DeleteKey(key, subkeystr)
             self.fail("Deleting the key twice succeeded")
         except OSError:
diff --git a/Lib/tkinter/__init__.py b/Lib/tkinter/__init__.py
--- a/Lib/tkinter/__init__.py
+++ b/Lib/tkinter/__init__.py
@@ -245,7 +245,7 @@
         if self.delta == 0:
             del attrs['delta']
         # widget usually is known
-        # serial and time are not very interesing
+        # serial and time are not very interesting
         # keysym_num duplicates keysym
         # x_root and y_root mostly duplicate x and y
         keys = ('send_event',
diff --git a/Lib/unittest/test/test_discovery.py b/Lib/unittest/test/test_discovery.py
--- a/Lib/unittest/test/test_discovery.py
+++ b/Lib/unittest/test/test_discovery.py
@@ -349,7 +349,7 @@
         suite = list(loader._find_tests(abspath('/foo'), 'test*.py'))
 
         # We should have loaded tests from both my_package and
-        # my_pacakge.test_module, and also run the load_tests hook in both.
+        # my_package.test_module, and also run the load_tests hook in both.
         # (normally this would be nested TestSuites.)
         self.assertEqual(suite,
                          [['my_package load_tests', [],
diff --git a/Lib/unittest/test/testmock/testcallable.py b/Lib/unittest/test/testmock/testcallable.py
--- a/Lib/unittest/test/testmock/testcallable.py
+++ b/Lib/unittest/test/testmock/testcallable.py
@@ -27,7 +27,7 @@
             self.assertIn(mock.__class__.__name__, repr(mock))
 
 
-    def test_heirarchy(self):
+    def test_hierarchy(self):
         self.assertTrue(issubclass(MagicMock, Mock))
         self.assertTrue(issubclass(NonCallableMagicMock, NonCallableMock))
 
diff --git a/Lib/venv/scripts/posix/activate b/Lib/venv/scripts/posix/activate
--- a/Lib/venv/scripts/posix/activate
+++ b/Lib/venv/scripts/posix/activate
@@ -34,7 +34,7 @@
     fi
 }
 
-# unset irrelavent variables
+# unset irrelevant variables
 deactivate nondestructive
 
 VIRTUAL_ENV="__VENV_DIR__"
diff --git a/Lib/venv/scripts/posix/activate.csh b/Lib/venv/scripts/posix/activate.csh
--- a/Lib/venv/scripts/posix/activate.csh
+++ b/Lib/venv/scripts/posix/activate.csh
@@ -5,7 +5,7 @@
 
 alias deactivate 'test $?_OLD_VIRTUAL_PATH != 0 && setenv PATH "$_OLD_VIRTUAL_PATH" && unset _OLD_VIRTUAL_PATH; rehash; test $?_OLD_VIRTUAL_PROMPT != 0 && set prompt="$_OLD_VIRTUAL_PROMPT" && unset _OLD_VIRTUAL_PROMPT; unsetenv VIRTUAL_ENV; test "\!:*" != "nondestructive" && unalias deactivate'
 
-# Unset irrelavent variables.
+# Unset irrelevant variables.
 deactivate nondestructive
 
 setenv VIRTUAL_ENV "__VENV_DIR__"
diff --git a/Lib/venv/scripts/posix/activate.fish b/Lib/venv/scripts/posix/activate.fish
--- a/Lib/venv/scripts/posix/activate.fish
+++ b/Lib/venv/scripts/posix/activate.fish
@@ -29,7 +29,7 @@
     end
 end
 
-# unset irrelavent variables
+# unset irrelevant variables
 deactivate nondestructive
 
 set -gx VIRTUAL_ENV "__VENV_DIR__"
diff --git a/Mac/PythonLauncher/MyAppDelegate.m b/Mac/PythonLauncher/MyAppDelegate.m
--- a/Mac/PythonLauncher/MyAppDelegate.m
+++ b/Mac/PythonLauncher/MyAppDelegate.m
@@ -34,7 +34,7 @@
 - (BOOL)shouldShowUI
 {
     // if this call comes before applicationDidFinishLaunching: we
-    // should terminate immedeately after starting the script.
+    // should terminate immediately after starting the script.
     if (!initial_action_done)
         should_terminate = YES;
     initial_action_done = YES;
diff --git a/Misc/HISTORY b/Misc/HISTORY
--- a/Misc/HISTORY
+++ b/Misc/HISTORY
@@ -1131,7 +1131,7 @@
   and http.client. Patch by EungJun Yi.
 
 - Issue #14777: tkinter may return undecoded UTF-8 bytes as a string when
-  accessing the Tk clipboard.  Modify clipboad_get() to first request type
+  accessing the Tk clipboard.  Modify clipboard_get() to first request type
   UTF8_STRING when no specific type is requested in an X11 windowing
   environment, falling back to the current default type STRING if that fails.
   Original patch by Thomas Kluyver.
@@ -5693,7 +5693,7 @@
   for reading).
 
 - hashlib has two new constant attributes: algorithms_guaranteed and
-  algorithms_avaiable that respectively list the names of hash algorithms
+  algorithms_available that respectively list the names of hash algorithms
   guaranteed to exist in all Python implementations and the names of hash
   algorithms available in the current process.
 
@@ -7344,7 +7344,7 @@
 - Issue #2846: Add support for gzip.GzipFile reading zero-padded files.  Patch
   by Brian Curtin.
 
-- Issue #7681: Use floor division in appropiate places in the wave module.
+- Issue #7681: Use floor division in appropriate places in the wave module.
 
 - Issue #5372: Drop the reuse of .o files in Distutils' ccompiler (since
   Extension extra options may change the output without changing the .c
@@ -10921,7 +10921,7 @@
 
 - Support for BeOS and AtheOS was removed (according to PEP 11).
 
-- Support for RiscOS, Irix, Tru64 was removed (alledgedly).
+- Support for RiscOS, Irix, Tru64 was removed (allegedly).
 
 Tools/Demos
 -----------
@@ -12912,7 +12912,7 @@
 - Bug #947906: An object oriented interface has been added to the calendar
   module. It's possible to generate HTML calendar now and the module can be
   called as a script (e.g. via ``python -mcalendar``). Localized month and
-  weekday names can be ouput (even if an exotic encoding is used) using
+  weekday names can be output (even if an exotic encoding is used) using
   special classes that use unicode.
 
 Build
@@ -13295,7 +13295,7 @@
   ``True`` for ``!=``, and raises ``TypeError`` for other comparison
   operators.  Because datetime is a subclass of date, comparing only the
   base class (date) members can still be done, if that's desired, by
-  forcing using of the approprate date method; e.g.,
+  forcing using of the appropriate date method; e.g.,
   ``a_date.__eq__(a_datetime)`` is true if and only if the year, month
   and day members of ``a_date`` and ``a_datetime`` are equal.
 
@@ -23770,7 +23770,7 @@
 
 - copy.py: Make sure the objects returned by __getinitargs__() are
 kept alive (in the memo) to avoid a certain kind of nasty crash.  (Not
-easily reproducable because it requires a later call to
+easily reproducible because it requires a later call to
 __getinitargs__() to return a tuple that happens to be allocated at
 the same address.)
 
@@ -27402,7 +27402,7 @@
 There is now a script to patch Makefile and config.c to add a new
 optional built-in module: Addmodule.sh.  Read the script before using!
 
-Useing Addmodule.sh, all optional modules can now be configured at
+Using Addmodule.sh, all optional modules can now be configured at
 compile time using Configure.py, so there are no modules left that
 require dynamic loading.
 
@@ -27833,9 +27833,9 @@
 
 SUNAUDIODEV: symbolic constant definitions for sunaudiodef (sun only)
 
-SV: symbolic constat definitions for sv (sgi only)
-
-CD: symbolic constat definitions for cd (sgi only)
+SV: symbolic constant definitions for sv (sgi only)
+
+CD: symbolic constant definitions for cd (sgi only)
 
 
 New demos
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -425,7 +425,7 @@
 - Issue #27079: Fixed curses.ascii functions isblank(), iscntrl() and ispunct().
 
 - Issue #27294: Numerical state in the repr for Tkinter event objects is now
-  represented as a compination of known flags.
+  represented as a combination of known flags.
 
 - Issue #27177: Match objects in the re module now support index-like objects
   as group indices.  Based on patches by Jeroen Demeyer and Xiang Zhang.
@@ -5662,7 +5662,7 @@
 
 - Issue #22120: For functions using an unsigned integer return converter,
   Argument Clinic now generates a cast to that type for the comparison
-  to -1 in the generated code.  (This supresses a compilation warning.)
+  to -1 in the generated code.  (This suppresses a compilation warning.)
 
 - Issue #18974: Tools/scripts/diff.py now uses argparse instead of optparse.
 
@@ -6762,7 +6762,7 @@
 
 - Issue #19466: Clear the frames of daemon threads earlier during the
   Python shutdown to call objects destructors. So "unclosed file" resource
-  warnings are now corretly emitted for daemon threads.
+  warnings are now correctly emitted for daemon threads.
 
 - Issue #19514: Deduplicate some _Py_IDENTIFIER declarations.
   Patch by Andrei Dorian Duma.
@@ -7692,7 +7692,7 @@
 
 - Issue #18709: Fix CVE-2013-4238. The SSL module now handles NULL bytes
   inside subjectAltName correctly. Formerly the module has used OpenSSL's
-  GENERAL_NAME_print() function to get the string represention of ASN.1
+  GENERAL_NAME_print() function to get the string representation of ASN.1
   strings for ``rfc822Name`` (email), ``dNSName`` (DNS) and
   ``uniformResourceIdentifier`` (URI).
 
@@ -7785,7 +7785,7 @@
 Documentation
 -------------
 
-- Issue #18743: Fix references to non-existant "StringIO" module.
+- Issue #18743: Fix references to non-existent "StringIO" module.
 
 - Issue #18783: Removed existing mentions of Python long type in docstrings,
   error messages and comments.
@@ -8724,7 +8724,7 @@
   specifically addresses a stack misalignment issue on x86 and issues on
   some more recent platforms.
 
-- Issue #8862: Fixed curses cleanup when getkey is interrputed by a signal.
+- Issue #8862: Fixed curses cleanup when getkey is interrupted by a signal.
 
 - Issue #17443: imaplib.IMAP4_stream was using the default unbuffered IO
   in subprocess, but the imap code assumes buffered IO.  In Python2 this
diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h
--- a/Modules/_ctypes/ctypes.h
+++ b/Modules/_ctypes/ctypes.h
@@ -238,7 +238,7 @@
  StgDictObject function to a generic one.
 
  Currently, PyCFuncPtr types have 'converters' and 'checker' entries in their
- type dict.  They are only used to cache attributes from other entries, whihc
+ type dict.  They are only used to cache attributes from other entries, which
  is wrong.
 
  One use case is the .value attribute that all simple types have.  But some
diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c
--- a/Modules/_hashopenssl.c
+++ b/Modules/_hashopenssl.c
@@ -724,7 +724,7 @@
 /*
  *  This macro generates constructor function definitions for specific
  *  hash algorithms.  These constructors are much faster than calling
- *  the generic one passing it a python string and are noticably
+ *  the generic one passing it a python string and are noticeably
  *  faster than calling a python new() wrapper.  Thats important for
  *  code that wants to make hashes of a bunch of small strings.
  */
diff --git a/Modules/_io/iobase.c b/Modules/_io/iobase.c
--- a/Modules/_io/iobase.c
+++ b/Modules/_io/iobase.c
@@ -90,7 +90,7 @@
     return NULL;
 }
 
-/* Positionning */
+/* Positioning */
 
 PyDoc_STRVAR(iobase_seek_doc,
     "Change stream position.\n"
diff --git a/Modules/_pickle.c b/Modules/_pickle.c
--- a/Modules/_pickle.c
+++ b/Modules/_pickle.c
@@ -2131,7 +2131,7 @@
         Py_UCS4 ch = PyUnicode_READ(kind, data, i);
         /* Map 32-bit characters to '\Uxxxxxxxx' */
         if (ch >= 0x10000) {
-            /* -1: substract 1 preallocated byte */
+            /* -1: subtract 1 preallocated byte */
             p = _PyBytesWriter_Prepare(&writer, p, 10-1);
             if (p == NULL)
                 goto error;
@@ -2149,7 +2149,7 @@
         }
         /* Map 16-bit characters, '\\' and '\n' to '\uxxxx' */
         else if (ch >= 256 || ch == '\\' || ch == '\n') {
-            /* -1: substract 1 preallocated byte */
+            /* -1: subtract 1 preallocated byte */
             p = _PyBytesWriter_Prepare(&writer, p, 6-1);
             if (p == NULL)
                 goto error;
diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c
--- a/Modules/_testcapimodule.c
+++ b/Modules/_testcapimodule.c
@@ -3798,7 +3798,7 @@
 {
     PyThreadState *tstate = PyThreadState_GET();
 
-    /* substract one to ignore the frame of the get_recursion_depth() call */
+    /* subtract one to ignore the frame of the get_recursion_depth() call */
     return PyLong_FromLong(tstate->recursion_depth - 1);
 }
 
diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c
--- a/Modules/_threadmodule.c
+++ b/Modules/_threadmodule.c
@@ -45,7 +45,7 @@
 /* Helper to acquire an interruptible lock with a timeout.  If the lock acquire
  * is interrupted, signal handlers are run, and if they raise an exception,
  * PY_LOCK_INTR is returned.  Otherwise, PY_LOCK_ACQUIRED or PY_LOCK_FAILURE
- * are returned, depending on whether the lock can be acquired withing the
+ * are returned, depending on whether the lock can be acquired within the
  * timeout.
  */
 static PyLockStatus
diff --git a/Modules/_tracemalloc.c b/Modules/_tracemalloc.c
--- a/Modules/_tracemalloc.c
+++ b/Modules/_tracemalloc.c
@@ -716,7 +716,7 @@
 
         if (ADD_TRACE(ptr2, new_size) < 0) {
             /* Memory allocation failed. The error cannot be reported to
-               the caller, because realloc() may already have shrinked the
+               the caller, because realloc() may already have shrunk the
                memory block and so removed bytes.
 
                This case is very unlikely: a hash entry has just been
diff --git a/Modules/binascii.c b/Modules/binascii.c
--- a/Modules/binascii.c
+++ b/Modules/binascii.c
@@ -837,7 +837,7 @@
     if (in_byte == RUNCHAR) {
         INBYTE(in_repeat);
         /* only 1 byte will be written, but 2 bytes were preallocated:
-           substract 1 byte to prevent overallocation */
+           subtract 1 byte to prevent overallocation */
         writer.min_size--;
 
         if (in_repeat != 0) {
@@ -858,7 +858,7 @@
         if (in_byte == RUNCHAR) {
             INBYTE(in_repeat);
             /* only 1 byte will be written, but 2 bytes were preallocated:
-               substract 1 byte to prevent overallocation */
+               subtract 1 byte to prevent overallocation */
             writer.min_size--;
 
             if ( in_repeat == 0 ) {
diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c
--- a/Modules/mathmodule.c
+++ b/Modules/mathmodule.c
@@ -1274,7 +1274,7 @@
 
 /* Divide-and-conquer factorial algorithm
  *
- * Based on the formula and psuedo-code provided at:
+ * Based on the formula and pseudo-code provided at:
  * http://www.luschny.de/math/factorial/binarysplitfact.html
  *
  * Faster algorithms exist, but they're more complicated and depend on
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -6611,7 +6611,7 @@
     PyModule_AddIntConstant(m, "SOMAXCONN", 5); /* Common value */
 #endif
 
-    /* Ancilliary message types */
+    /* Ancillary message types */
 #ifdef  SCM_RIGHTS
     PyModule_AddIntMacro(m, SCM_RIGHTS);
 #endif
diff --git a/Modules/zipimport.c b/Modules/zipimport.c
--- a/Modules/zipimport.c
+++ b/Modules/zipimport.c
@@ -1315,7 +1315,7 @@
     return code;
 }
 
-/* Replace any occurances of "\r\n?" in the input string with "\n".
+/* Replace any occurrences of "\r\n?" in the input string with "\n".
    This converts DOS and Mac line endings to Unix line endings.
    Also append a trailing "\n" to be compatible with
    PyParser_SimpleParseFile(). Returns a new reference. */
diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c
--- a/Objects/bytearrayobject.c
+++ b/Objects/bytearrayobject.c
@@ -481,7 +481,7 @@
 
                If growth < 0 and lo != 0, the operation is completed, but a
                MemoryError is still raised and the memory block is not
-               shrinked. Otherwise, the bytearray is restored in its previous
+               shrunk. Otherwise, the bytearray is restored in its previous
                state and a MemoryError is raised. */
             if (lo == 0) {
                 self->ob_start += growth;
diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c
--- a/Objects/bytesobject.c
+++ b/Objects/bytesobject.c
@@ -247,7 +247,7 @@
             ++f;
         }
 
-        /* substract bytes preallocated for the format string
+        /* subtract bytes preallocated for the format string
            (ex: 2 for "%s") */
         writer.min_size -= (f - p + 1);
 
@@ -1093,7 +1093,7 @@
     assert(PyBytes_Check(w));
 
     /* Append bytes to output buffer. */
-    writer->min_size--;   /* substract 1 preallocated byte */
+    writer->min_size--;   /* subtract 1 preallocated byte */
     p = _PyBytesWriter_WriteBytes(writer, p,
                                   PyBytes_AS_STRING(w),
                                   PyBytes_GET_SIZE(w));
diff --git a/Objects/codeobject.c b/Objects/codeobject.c
--- a/Objects/codeobject.c
+++ b/Objects/codeobject.c
@@ -719,7 +719,7 @@
     /* possible optimization: if f->f_lasti == instr_ub
        (likely to be a common case) then we already know
        instr_lb -- if we stored the matching value of p
-       somwhere we could skip the first while loop. */
+       somewhere we could skip the first while loop. */
 
     /* See lnotab_notes.txt for the description of
        co_lnotab.  A point to remember: increments to p
diff --git a/Objects/listsort.txt b/Objects/listsort.txt
--- a/Objects/listsort.txt
+++ b/Objects/listsort.txt
@@ -694,7 +694,7 @@
 
 But in CPython's case, comparisons are extraordinarily expensive compared to
 moving data, and the details matter.  Moving objects is just copying
-pointers.  Comparisons can be arbitrarily expensive (can invoke arbitary
+pointers.  Comparisons can be arbitrarily expensive (can invoke arbitrary
 user-supplied Python code), but even in simple cases (like 3 < 4) _all_
 decisions are made at runtime:  what's the type of the left comparand?  the
 type of the right?  do they need to be coerced to a common type?  where's the
diff --git a/Objects/longobject.c b/Objects/longobject.c
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -368,7 +368,7 @@
 /* Checking for overflow in PyLong_AsLong is a PITA since C doesn't define
  * anything about what happens when a signed integer operation overflows,
  * and some compilers think they're doing you a favor by being "clever"
- * then.  The bit pattern for the largest postive signed long is
+ * then.  The bit pattern for the largest positive signed long is
  * (unsigned long)LONG_MAX, and for the smallest negative signed long
  * it is abs(LONG_MIN), which we could write -(unsigned long)LONG_MIN.
  * However, some other compilers warn about applying unary minus to an
diff --git a/Objects/stringlib/codecs.h b/Objects/stringlib/codecs.h
--- a/Objects/stringlib/codecs.h
+++ b/Objects/stringlib/codecs.h
@@ -347,7 +347,7 @@
                 break;
 
             case _Py_ERROR_BACKSLASHREPLACE:
-                /* substract preallocated bytes */
+                /* subtract preallocated bytes */
                 writer.min_size -= max_char_size * (endpos - startpos);
                 p = backslashreplace(&writer, p,
                                      unicode, startpos, endpos);
@@ -357,7 +357,7 @@
                 break;
 
             case _Py_ERROR_XMLCHARREFREPLACE:
-                /* substract preallocated bytes */
+                /* subtract preallocated bytes */
                 writer.min_size -= max_char_size * (endpos - startpos);
                 p = xmlcharrefreplace(&writer, p,
                                       unicode, startpos, endpos);
@@ -387,7 +387,7 @@
                 if (!rep)
                     goto error;
 
-                /* substract preallocated bytes */
+                /* subtract preallocated bytes */
                 writer.min_size -= max_char_size;
 
                 if (PyBytes_Check(rep)) {
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -3792,7 +3792,7 @@
     /* Try to fetch cached copy of copyreg from sys.modules first in an
        attempt to avoid the import overhead. Previously this was implemented
        by storing a reference to the cached module in a static variable, but
-       this broke when multiple embeded interpreters were in use (see issue
+       this broke when multiple embedded interpreters were in use (see issue
        #17408 and #19088). */
     copyreg_module = PyDict_GetItemWithError(interp->modules, copyreg_str);
     if (copyreg_module != NULL) {
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -6110,7 +6110,7 @@
 
         /* Escape backslashes */
         if (ch == '\\') {
-            /* -1: substract 1 preallocated byte */
+            /* -1: subtract 1 preallocated byte */
             p = _PyBytesWriter_Prepare(&writer, p, 2-1);
             if (p == NULL)
                 goto error;
@@ -6183,7 +6183,7 @@
 
         /* Map non-printable US ASCII to '\xhh' */
         else if (ch < ' ' || ch >= 0x7F) {
-            /* -1: substract 1 preallocated byte */
+            /* -1: subtract 1 preallocated byte */
             p = _PyBytesWriter_Prepare(&writer, p, 4-1);
             if (p == NULL)
                 goto error;
@@ -6363,7 +6363,7 @@
         if (ch >= 0x10000) {
             assert(ch <= MAX_UNICODE);
 
-            /* -1: substract 1 preallocated byte */
+            /* -1: subtract 1 preallocated byte */
             p = _PyBytesWriter_Prepare(&writer, p, 10-1);
             if (p == NULL)
                 goto error;
@@ -6381,7 +6381,7 @@
         }
         /* Map 16-bit characters to '\uxxxx' */
         else if (ch >= 256) {
-            /* -1: substract 1 preallocated byte */
+            /* -1: subtract 1 preallocated byte */
             p = _PyBytesWriter_Prepare(&writer, p, 6-1);
             if (p == NULL)
                 goto error;
@@ -6705,7 +6705,7 @@
                 break;
 
             case _Py_ERROR_BACKSLASHREPLACE:
-                /* substract preallocated bytes */
+                /* subtract preallocated bytes */
                 writer.min_size -= (collend - collstart);
                 str = backslashreplace(&writer, str,
                                        unicode, collstart, collend);
@@ -6715,7 +6715,7 @@
                 break;
 
             case _Py_ERROR_XMLCHARREFREPLACE:
-                /* substract preallocated bytes */
+                /* subtract preallocated bytes */
                 writer.min_size -= (collend - collstart);
                 str = xmlcharrefreplace(&writer, str,
                                         unicode, collstart, collend);
@@ -6747,7 +6747,7 @@
                 if (rep == NULL)
                     goto onError;
 
-                /* substract preallocated bytes */
+                /* subtract preallocated bytes */
                 writer.min_size -= 1;
 
                 if (PyBytes_Check(rep)) {
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -2090,16 +2090,16 @@
 
         TARGET(YIELD_FROM) {
             PyObject *v = POP();
-            PyObject *reciever = TOP();
+            PyObject *receiver = TOP();
             int err;
-            if (PyGen_CheckExact(reciever) || PyCoro_CheckExact(reciever)) {
-                retval = _PyGen_Send((PyGenObject *)reciever, v);
+            if (PyGen_CheckExact(receiver) || PyCoro_CheckExact(receiver)) {
+                retval = _PyGen_Send((PyGenObject *)receiver, v);
             } else {
                 _Py_IDENTIFIER(send);
                 if (v == Py_None)
-                    retval = Py_TYPE(reciever)->tp_iternext(reciever);
+                    retval = Py_TYPE(receiver)->tp_iternext(receiver);
                 else
-                    retval = _PyObject_CallMethodIdObjArgs(reciever, &PyId_send, v, NULL);
+                    retval = _PyObject_CallMethodIdObjArgs(receiver, &PyId_send, v, NULL);
             }
             Py_DECREF(v);
             if (retval == NULL) {
@@ -2110,7 +2110,7 @@
                 err = _PyGen_FetchStopIterationValue(&val);
                 if (err < 0)
                     goto error;
-                Py_DECREF(reciever);
+                Py_DECREF(receiver);
                 SET_TOP(val);
                 DISPATCH();
             }
diff --git a/Python/condvar.h b/Python/condvar.h
--- a/Python/condvar.h
+++ b/Python/condvar.h
@@ -238,7 +238,7 @@
     cv->waiting++;
     PyMUTEX_UNLOCK(cs);
     /* "lost wakeup bug" would occur if the caller were interrupted here,
-     * but we are safe because we are using a semaphore wich has an internal
+     * but we are safe because we are using a semaphore which has an internal
      * count.
      */
     wait = WaitForSingleObjectEx(cv->sem, ms, FALSE);
diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c
--- a/Python/formatter_unicode.c
+++ b/Python/formatter_unicode.c
@@ -121,7 +121,7 @@
 } InternalFormatSpec;
 
 #if 0
-/* Occassionally useful for debugging. Should normally be commented out. */
+/* Occasionally useful for debugging. Should normally be commented out. */
 static void
 DEBUG_PRINT_FORMAT_SPEC(InternalFormatSpec *format)
 {
diff --git a/README b/README
--- a/README
+++ b/README
@@ -68,7 +68,7 @@
 After this instrumented version of the interpreter is built, the Makefile
 will automatically run a training workload. This is necessary in order to
 profile the interpreter execution. Note also that any output, both stdout
-and stderr, that may appear at this step is supressed.
+and stderr, that may appear at this step is suppressed.
 
 Finally, the last step is to rebuild the interpreter, using the information
 collected in the previous one. The end result will be a Python binary
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -7112,7 +7112,7 @@
         # Calculate an appropriate deployment target for this build:
         # The deployment target value is used explicitly to enable certain
         # features are enabled (such as builtin libedit support for readline)
-        # through the use of Apple's Availabiliy Macros and is used as a
+        # through the use of Apple's Availability Macros and is used as a
         # component of the string returned by distutils.get_platform().
         #
         # Use the value from:
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -1639,7 +1639,7 @@
         # Calculate an appropriate deployment target for this build:
         # The deployment target value is used explicitly to enable certain
         # features are enabled (such as builtin libedit support for readline)
-        # through the use of Apple's Availabiliy Macros and is used as a
+        # through the use of Apple's Availability Macros and is used as a
         # component of the string returned by distutils.get_platform().
         #
         # Use the value from:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Aug 30 15:35:56 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 30 Aug 2016 19:35:56 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327842=3A_The_csv?=
 =?utf-8?q?=2EDictReader_now_returns_rows_of_type_OrderedDict=2E?=
Message-ID: <20160830193556.22134.55256.389C9DEE@psf.io>

https://hg.python.org/cpython/rev/bb3e2a5be31b
changeset:   102953:bb3e2a5be31b
user:        Raymond Hettinger 
date:        Tue Aug 30 12:35:50 2016 -0700
summary:
  Issue #27842: The csv.DictReader now returns rows of type OrderedDict.

files:
  Doc/library/csv.rst  |  39 +++++++++++++++++++------------
  Lib/csv.py           |   3 +-
  Lib/test/test_csv.py |  16 +++++++++++++
  Misc/NEWS            |   3 ++
  4 files changed, 45 insertions(+), 16 deletions(-)


diff --git a/Doc/library/csv.rst b/Doc/library/csv.rst
--- a/Doc/library/csv.rst
+++ b/Doc/library/csv.rst
@@ -149,18 +149,25 @@
 .. class:: DictReader(csvfile, fieldnames=None, restkey=None, restval=None, \
                       dialect='excel', *args, **kwds)
 
-   Create an object which operates like a regular reader but maps the
-   information read into a dict whose keys are given by the optional
-   *fieldnames* parameter.  The *fieldnames* parameter is a :mod:`sequence
-   ` whose elements are associated with the fields of the
-   input data in order. These elements become the keys of the resulting
-   dictionary.  If the *fieldnames* parameter is omitted, the values in the
-   first row of the *csvfile* will be used as the fieldnames.  If the row read
-   has more fields than the fieldnames sequence, the remaining data is added as
-   a sequence keyed by the value of *restkey*.  If the row read has fewer
-   fields than the fieldnames sequence, the remaining keys take the value of
-   the optional *restval* parameter.  Any other optional or keyword arguments
-   are passed to the underlying :class:`reader` instance.
+   Create an object that operates like a regular reader but maps the
+   information in each row to an :mod:`OrderedDict `
+   whose keys are given by the optional *fieldnames* parameter.
+
+   The *fieldnames* parameter is a :term:`sequence`.  If *fieldnames* is
+   omitted, the values in the first row of the *csvfile* will be used as the
+   fieldnames.  Regardless of how the fieldnames are determined, the ordered
+   dictionary preserves their original ordering.
+
+   If a row has more fields than fieldnames, the remaining data is put in a
+   list and stored with the fieldname specified by *restkey* (which defaults
+   to ``None``).  If a non-blank row has fewer fields than fieldnames, the
+   missing values are filled-in with ``None``.
+
+   All other optional or keyword arguments are passed to the underlying
+   :class:`reader` instance.
+
+   .. versionchanged:: 3.6
+      Returned rows are now of type :class:`OrderedDict`.
 
    A short usage example::
 
@@ -170,9 +177,11 @@
        ...     for row in reader:
        ...         print(row['first_name'], row['last_name'])
        ...
-       Baked Beans
-       Lovely Spam
-       Wonderful Spam
+       Eric Idle
+       John Cleese
+
+       >>> print(row)
+       OrderedDict([('first_name', 'John'), ('last_name', 'Cleese')])
 
 
 .. class:: DictWriter(csvfile, fieldnames, restval='', extrasaction='raise', \
diff --git a/Lib/csv.py b/Lib/csv.py
--- a/Lib/csv.py
+++ b/Lib/csv.py
@@ -11,6 +11,7 @@
                  __doc__
 from _csv import Dialect as _Dialect
 
+from collections import OrderedDict
 from io import StringIO
 
 __all__ = ["QUOTE_MINIMAL", "QUOTE_ALL", "QUOTE_NONNUMERIC", "QUOTE_NONE",
@@ -116,7 +117,7 @@
         # values
         while row == []:
             row = next(self.reader)
-        d = dict(zip(self.fieldnames, row))
+        d = OrderedDict(zip(self.fieldnames, row))
         lf = len(self.fieldnames)
         lr = len(row)
         if lf < lr:
diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py
--- a/Lib/test/test_csv.py
+++ b/Lib/test/test_csv.py
@@ -10,6 +10,7 @@
 import gc
 import pickle
 from test import support
+from itertools import permutations
 
 class Test_Csv(unittest.TestCase):
     """
@@ -1092,6 +1093,21 @@
             fileobj.seek(0)
             self.assertEqual(fileobj.read(), expected)
 
+class KeyOrderingTest(unittest.TestCase):
+
+    def test_ordering_for_the_dict_reader_and_writer(self):
+        resultset = set()
+        for keys in permutations("abcde"):
+            with TemporaryFile('w+', newline='', encoding="utf-8") as fileobject:
+                dw = csv.DictWriter(fileobject, keys)
+                dw.writeheader()
+                fileobject.seek(0)
+                dr = csv.DictReader(fileobject)
+                kt = tuple(dr.fieldnames)
+                self.assertEqual(keys, kt)
+                resultset.add(kt)
+        # Final sanity check: were all permutations unique?
+        self.assertEqual(len(resultset), 120, "Key ordering: some key permutations not collected (expected 120)")
 
 class MiscTestCase(unittest.TestCase):
     def test__all__(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -64,6 +64,9 @@
   match ``math.inf`` and ``math.nan``, and also ``cmath.infj`` and
   ``cmath.nanj`` to match the format used by complex repr.
 
+- Issue #27842: The csv.DictReader now returns rows of type OrderedDict.
+  (Contributed by Steve Holden.)
+
 - Issue #27861: Fixed a crash in sqlite3.Connection.cursor() when a factory
   creates not a cursor.  Patch by Xiang Zhang.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Aug 30 15:57:32 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 30 Aug 2016 19:57:32 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327895=3A_Strength?=
 =?utf-8?q?en_the_dict_reader_tests=2E?=
Message-ID: <20160830195731.8555.48210.8A367CFC@psf.io>

https://hg.python.org/cpython/rev/89ce78ce1bfa
changeset:   102954:89ce78ce1bfa
user:        Raymond Hettinger 
date:        Tue Aug 30 12:57:26 2016 -0700
summary:
  Issue #27895: Strengthen the dict reader tests.

files:
  Lib/test/test_csv.py |  38 ++++++++++++++++++++++++++++++++
  1 files changed, 38 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py
--- a/Lib/test/test_csv.py
+++ b/Lib/test/test_csv.py
@@ -11,6 +11,8 @@
 import pickle
 from test import support
 from itertools import permutations
+from textwrap import dedent
+from collections import OrderedDict
 
 class Test_Csv(unittest.TestCase):
     """
@@ -1109,6 +1111,42 @@
         # Final sanity check: were all permutations unique?
         self.assertEqual(len(resultset), 120, "Key ordering: some key permutations not collected (expected 120)")
 
+    def test_ordered_dict_reader(self):
+        data = dedent('''\
+            FirstName,LastName
+            Eric,Idle
+            Graham,Chapman,Over1,Over2
+
+            Under1
+            John,Cleese
+        ''').splitlines()
+
+        self.assertEqual(list(csv.DictReader(data)),
+            [OrderedDict([('FirstName', 'Eric'), ('LastName', 'Idle')]),
+             OrderedDict([('FirstName', 'Graham'), ('LastName', 'Chapman'),
+                          (None, ['Over1', 'Over2'])]),
+             OrderedDict([('FirstName', 'Under1'), ('LastName', None)]),
+             OrderedDict([('FirstName', 'John'), ('LastName', 'Cleese')]),
+            ])
+
+        self.assertEqual(list(csv.DictReader(data, restkey='OtherInfo')),
+            [OrderedDict([('FirstName', 'Eric'), ('LastName', 'Idle')]),
+             OrderedDict([('FirstName', 'Graham'), ('LastName', 'Chapman'),
+                          ('OtherInfo', ['Over1', 'Over2'])]),
+             OrderedDict([('FirstName', 'Under1'), ('LastName', None)]),
+             OrderedDict([('FirstName', 'John'), ('LastName', 'Cleese')]),
+            ])
+
+        del data[0]            # Remove the header row
+        self.assertEqual(list(csv.DictReader(data, fieldnames=['fname', 'lname'])),
+            [OrderedDict([('fname', 'Eric'), ('lname', 'Idle')]),
+             OrderedDict([('fname', 'Graham'), ('lname', 'Chapman'),
+                          (None, ['Over1', 'Over2'])]),
+             OrderedDict([('fname', 'Under1'), ('lname', None)]),
+             OrderedDict([('fname', 'John'), ('lname', 'Cleese')]),
+            ])
+
+
 class MiscTestCase(unittest.TestCase):
     def test__all__(self):
         extra = {'__doc__', '__version__'}

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Aug 30 16:44:09 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 30 Aug 2016 20:44:09 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328894=3A_Fix_to?=
 =?utf-8?q?=5Faddrs_refs_in_smtplib_docs?=
Message-ID: <20160830204409.31391.32966.0AAE1F10@psf.io>

https://hg.python.org/cpython/rev/f673a60bdc1a
changeset:   102956:f673a60bdc1a
parent:      102954:89ce78ce1bfa
user:        Raymond Hettinger 
date:        Tue Aug 30 13:43:53 2016 -0700
summary:
  Issue #28894: Fix to_addrs refs in smtplib docs

files:
  Doc/library/smtplib.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
--- a/Doc/library/smtplib.rst
+++ b/Doc/library/smtplib.rst
@@ -486,7 +486,7 @@
    those arguments with addresses extracted from the headers of *msg* as
    specified in :rfc:`5322`\: *from_addr* is set to the :mailheader:`Sender`
    field if it is present, and otherwise to the :mailheader:`From` field.
-   *to_adresses* combines the values (if any) of the :mailheader:`To`,
+   *to_addrs* combines the values (if any) of the :mailheader:`To`,
    :mailheader:`Cc`, and :mailheader:`Bcc` fields from *msg*.  If exactly one
    set of :mailheader:`Resent-*` headers appear in the message, the regular
    headers are ignored and the :mailheader:`Resent-*` headers are used instead.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Aug 30 16:44:12 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 30 Aug 2016 20:44:12 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_typos?=
Message-ID: <20160830204408.52691.65790.482255C8@psf.io>

https://hg.python.org/cpython/rev/8d9d90ef80b0
changeset:   102955:8d9d90ef80b0
branch:      3.5
parent:      102950:53f2cde033ac
user:        Raymond Hettinger 
date:        Tue Aug 30 13:25:06 2016 -0700
summary:
  Fix typos

files:
  Doc/library/smtplib.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
--- a/Doc/library/smtplib.rst
+++ b/Doc/library/smtplib.rst
@@ -439,7 +439,7 @@
    and the accompanying error message sent by the server.
 
    If ``SMTPUTF8`` is included in *mail_options*, and the server supports it,
-   *from_addr* and *to_addr* may contain non-ASCII characters.
+   *from_addr* and *to_addrs* may contain non-ASCII characters.
 
    This method may raise the following exceptions:
 
@@ -486,7 +486,7 @@
    those arguments with addresses extracted from the headers of *msg* as
    specified in :rfc:`5322`\: *from_addr* is set to the :mailheader:`Sender`
    field if it is present, and otherwise to the :mailheader:`From` field.
-   *to_adresses* combines the values (if any) of the :mailheader:`To`,
+   *to_addrs* combines the values (if any) of the :mailheader:`To`,
    :mailheader:`Cc`, and :mailheader:`Bcc` fields from *msg*.  If exactly one
    set of :mailheader:`Resent-*` headers appear in the message, the regular
    headers are ignored and the :mailheader:`Resent-*` headers are used instead.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Aug 30 16:46:39 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 30 Aug 2016 20:46:39 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
 =?utf-8?q?=29=3A_Merge?=
Message-ID: <20160830204634.8301.54164.D4F0C59B@psf.io>

https://hg.python.org/cpython/rev/4340be056a7c
changeset:   102957:4340be056a7c
parent:      102956:f673a60bdc1a
parent:      102955:8d9d90ef80b0
user:        Raymond Hettinger 
date:        Tue Aug 30 13:46:29 2016 -0700
summary:
  Merge

files:
  Doc/library/smtplib.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
--- a/Doc/library/smtplib.rst
+++ b/Doc/library/smtplib.rst
@@ -439,7 +439,7 @@
    and the accompanying error message sent by the server.
 
    If ``SMTPUTF8`` is included in *mail_options*, and the server supports it,
-   *from_addr* and *to_addr* may contain non-ASCII characters.
+   *from_addr* and *to_addrs* may contain non-ASCII characters.
 
    This method may raise the following exceptions:
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Aug 30 16:58:52 2016
From: python-checkins at python.org (terry.reedy)
Date: Tue, 30 Aug 2016 20:58:52 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogPyBXb3JraW5nIERp?=
 =?utf-8?q?rectory_=3F?=
Message-ID: <20160830205852.2523.25247.52E8D7E1@psf.io>

https://hg.python.org/cpython/rev/d048df7efe0b
changeset:   102959:d048df7efe0b
branch:      3.5
parent:      102955:8d9d90ef80b0
user:        Terry Jan Reedy 
date:        Tue Aug 30 16:58:01 2016 -0400
summary:
  ? Working Directory ?

files:
  Doc/library/idle.rst  |   2 +-
  Lib/idlelib/help.html |  31 +++++++++++++++++--------------
  2 files changed, 18 insertions(+), 15 deletions(-)


diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst
--- a/Doc/library/idle.rst
+++ b/Doc/library/idle.rst
@@ -531,7 +531,7 @@
    -c command  run command in the shell window
    -d          enable debugger and open shell window
    -e          open editor window
-   -h          print help message with legal combinatios and exit
+   -h          print help message with legal combinations and exit
    -i          open shell window
    -r file     run file in shell window
    -s          run $IDLESTARTUP or $PYTHONSTARTUP first, in shell window
diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html
--- a/Lib/idlelib/help.html
+++ b/Lib/idlelib/help.html
@@ -6,7 +6,7 @@
   
     
 
-    25.5. IDLE — Python 3.5.2 documentation
+    25.5. IDLE — Python 3.5.2 documentation
 
     
     
@@ -58,13 +58,13 @@
              accesskey="P">previous |
         
    • 
-        
  • Python »
    • 
+        
  • Python »
    • 
         
  • 
-          3.5.2 Documentation »
+          3.5.2 Documentation »
         
    • 
 
-          
  • The Python Standard Library »
    • 
-          
  • 25. Graphical User Interfaces with Tk »
    • 
+          
  • The Python Standard Library »
    • 
+          
  • 25. Graphical User Interfaces with Tk »
    •
    @@ -497,7 +497,7 @@ -c command run command in the shell window -d enable debugger and open shell window -e open editor window --h print help message with legal combinatios and exit +-h print help message with legal combinations and exit -i open shell window -r file run file in shell window -s run $IDLESTARTUP or $PYTHONSTARTUP first, in shell window @@ -659,11 +659,14 @@
    @@ -687,13 +690,13 @@ >previous |
    • -
  • Python »
    • +
  • Python »
  • - 3.5.2 Documentation » + 3.5.2 Documentation »
    • -
  • The Python Standard Library »
    • -
  • 25. Graphical User Interfaces with Tk »
    • +
  • The Python Standard Library »
    • +
  • 25. Graphical User Interfaces with Tk »
    • @@ -702,10 +705,10 @@ The Python Software Foundation is a non-profit corporation. Please donate.
    - Last updated on Aug 25, 2016. + Last updated on Aug 30, 2016. Found a bug?
    - Created using Sphinx 1.4.6. + Created using Sphinx 1.3.6.
    -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 30 16:58:52 2016 From: python-checkins at python.org (terry.reedy) Date: Tue, 30 Aug 2016 20:58:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Backport_spell?= =?utf-8?q?ing_fix=2E?= Message-ID: <20160830205852.77456.69928.9A4C8899@psf.io> https://hg.python.org/cpython/rev/a5c43e660ae9 changeset: 102958:a5c43e660ae9 branch: 2.7 parent: 102943:58ea646ef657 user: Terry Jan Reedy date: Tue Aug 30 16:57:53 2016 -0400 summary: Backport spelling fix. files: Doc/library/idle.rst | 2 +- Lib/idlelib/help.html | 31 +++++++++++++++++-------------- 2 files changed, 18 insertions(+), 15 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -527,7 +527,7 @@ -c command run command in the shell window -d enable debugger and open shell window -e open editor window - -h print help message with legal combinatios and exit + -h print help message with legal combinations and exit -i open shell window -r file run file in shell window -s run $IDLESTARTUP or $PYTHONSTARTUP first, in shell window diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -6,7 +6,7 @@ - 24.6. IDLE — Python 2.7.12 documentation + 24.6. IDLE — Python 2.7.12 documentation @@ -58,13 +58,13 @@ accesskey="P">previous |
    • -
  • Python »
    • +
  • Python »
  • - Python 2.7.12 documentation » + Python 2.7.12 documentation »
    • -
  • The Python Standard Library »
    • -
  • 24. Graphical User Interfaces with Tk »
    • +
  • The Python Standard Library »
    • +
  • 24. Graphical User Interfaces with Tk »
    • @@ -495,7 +495,7 @@ -c command run command in the shell window -d enable debugger and open shell window -e open editor window --h print help message with legal combinatios and exit +-h print help message with legal combinations and exit -i open shell window -r file run file in shell window -s run $IDLESTARTUP or $PYTHONSTARTUP first, in shell window @@ -657,11 +657,14 @@
    @@ -685,13 +688,13 @@ >previous |
    • -
  • Python »
    • +
  • Python »
  • - Python 2.7.12 documentation » + Python 2.7.12 documentation »
    • -
  • The Python Standard Library »
    • -
  • 24. Graphical User Interfaces with Tk »
    • +
  • The Python Standard Library »
    • +
  • 24. Graphical User Interfaces with Tk »
    • @@ -700,10 +703,10 @@ The Python Software Foundation is a non-profit corporation. Please donate.
    - Last updated on Aug 25, 2016. + Last updated on Aug 30, 2016. Found a bug?
    - Created using Sphinx 1.4.6. + Created using Sphinx 1.3.6.
    -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 30 16:58:52 2016 From: python-checkins at python.org (terry.reedy) Date: Tue, 30 Aug 2016 20:58:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20160830205852.22277.24383.962D3ADD@psf.io> https://hg.python.org/cpython/rev/3b59ca37b4a1 changeset: 102960:3b59ca37b4a1 parent: 102957:4340be056a7c parent: 102959:d048df7efe0b user: Terry Jan Reedy date: Tue Aug 30 16:58:27 2016 -0400 summary: Merge with 3.5 files: Lib/idlelib/help.html | 29 ++++++++++++++++------------- 1 files changed, 16 insertions(+), 13 deletions(-) diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -6,7 +6,7 @@ - 25.5. IDLE — Python 3.5.2 documentation + 25.5. IDLE — Python 3.5.2 documentation @@ -58,13 +58,13 @@ accesskey="P">previous |
    • -
  • Python »
    • +
  • Python »
  • - 3.5.2 Documentation » + 3.5.2 Documentation »
    • -
  • The Python Standard Library »
    • -
  • 25. Graphical User Interfaces with Tk »
    • +
  • The Python Standard Library »
    • +
  • 25. Graphical User Interfaces with Tk »
    • @@ -659,11 +659,14 @@ @@ -687,13 +690,13 @@ >previous |
    • -
  • Python »
    • +
  • Python »
  • - 3.5.2 Documentation » + 3.5.2 Documentation »
    • -
  • The Python Standard Library »
    • -
  • 25. Graphical User Interfaces with Tk »
    • +
  • The Python Standard Library »
    • +
  • 25. Graphical User Interfaces with Tk »
    • @@ -702,10 +705,10 @@ The Python Software Foundation is a non-profit corporation. Please donate.
    - Last updated on Aug 25, 2016. + Last updated on Aug 30, 2016. Found a bug?
    - Created using Sphinx 1.4.6. + Created using Sphinx 1.3.6.
    -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 30 20:19:29 2016 From: python-checkins at python.org (terry.reedy) Date: Wed, 31 Aug 2016 00:19:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317642=3A_add_larg?= =?utf-8?q?er_font_sizes_for_classroom_projection=2E?= Message-ID: <20160831001929.14257.13329.74D6FC49@psf.io> https://hg.python.org/cpython/rev/c9d59e6cc1e4 changeset: 102962:c9d59e6cc1e4 parent: 102960:3b59ca37b4a1 user: Terry Jan Reedy date: Tue Aug 30 20:19:13 2016 -0400 summary: Issue #17642: add larger font sizes for classroom projection. files: Lib/idlelib/configdialog.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/configdialog.py b/Lib/idlelib/configdialog.py --- a/Lib/idlelib/configdialog.py +++ b/Lib/idlelib/configdialog.py @@ -1018,7 +1018,8 @@ pass ##font size dropdown self.optMenuFontSize.SetMenu(('7', '8', '9', '10', '11', '12', '13', - '14', '16', '18', '20', '22'), fontSize ) + '14', '16', '18', '20', '22', + '25', '29', '34', '40'), fontSize ) ##fontWeight self.fontBold.set(fontBold) ##font sample -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 30 20:19:29 2016 From: python-checkins at python.org (terry.reedy) Date: Wed, 31 Aug 2016 00:19:29 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE3NjQy?= =?utf-8?q?=3A_add_larger_font_sizes_for_classroom_projection=2E?= Message-ID: <20160831001929.129819.37656.4CA367CF@psf.io> https://hg.python.org/cpython/rev/f478f9b88319 changeset: 102961:f478f9b88319 branch: 2.7 parent: 102958:a5c43e660ae9 user: Terry Jan Reedy date: Tue Aug 30 20:19:07 2016 -0400 summary: Issue #17642: add larger font sizes for classroom projection. files: Lib/idlelib/configDialog.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/configDialog.py b/Lib/idlelib/configDialog.py --- a/Lib/idlelib/configDialog.py +++ b/Lib/idlelib/configDialog.py @@ -1012,7 +1012,8 @@ pass ##font size dropdown self.optMenuFontSize.SetMenu(('7', '8', '9', '10', '11', '12', '13', - '14', '16', '18', '20', '22'), fontSize ) + '14', '16', '18', '20', '22', + '25', '29', '34', '40'), fontSize ) ##fontWeight self.fontBold.set(fontBold) ##font sample -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 30 21:17:49 2016 From: python-checkins at python.org (r.david.murray) Date: Wed, 31 Aug 2016 01:17:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_=2327893=3A_arg_name_and_bytes_references_in_email?= =?utf-8?q?=2Eparser_docs=2E?= Message-ID: <20160831011749.8501.11508.22F3C0E7@psf.io> https://hg.python.org/cpython/rev/059f9f518834 changeset: 102964:059f9f518834 parent: 102962:c9d59e6cc1e4 parent: 102963:b8dd9ae08a91 user: R David Murray date: Tue Aug 30 21:17:25 2016 -0400 summary: Merge #27893: arg name and bytes references in email.parser docs. files: Doc/library/email.parser.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/library/email.parser.rst b/Doc/library/email.parser.rst --- a/Doc/library/email.parser.rst +++ b/Doc/library/email.parser.rst @@ -202,12 +202,12 @@ reading the headers or not. The default is ``False``, meaning it parses the entire contents of the file. - .. method:: parsebytes(bytes, headersonly=False) + .. method:: parsebytes(text, headersonly=False) - Similar to the :meth:`parse` method, except it takes a byte string object - instead of a file-like object. Calling this method on a byte string is - exactly equivalent to wrapping *text* in a :class:`~io.BytesIO` instance - first and calling :meth:`parse`. + Similar to the :meth:`parse` method, except it takes a :term:`bytes-like + object` instead of a file-like object. Calling this method is equivalent + to wrapping *text* in a :class:`~io.BytesIO` instance first and calling + :meth:`parse`. Optional *headersonly* is as with the :meth:`parse` method. @@ -233,7 +233,7 @@ .. function:: message_from_bytes(s, _class=email.message.Message, *, \ policy=policy.compat32) - Return a message object structure from a byte string. This is exactly + Return a message object structure from a :term:`bytes-like object`. This is exactly equivalent to ``BytesParser().parsebytes(s)``. Optional *_class* and *strict* are interpreted as with the :class:`~email.parser.Parser` class constructor. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Aug 30 21:17:49 2016 From: python-checkins at python.org (r.david.murray) Date: Wed, 31 Aug 2016 01:17:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzI3ODkzOiBhcmcg?= =?utf-8?q?name_and_bytes_references_in_email=2Eparser_docs=2E?= Message-ID: <20160831011749.16660.72559.7C99F8DF@psf.io> https://hg.python.org/cpython/rev/b8dd9ae08a91 changeset: 102963:b8dd9ae08a91 branch: 3.5 parent: 102959:d048df7efe0b user: R David Murray date: Tue Aug 30 21:17:02 2016 -0400 summary: #27893: arg name and bytes references in email.parser docs. Perhaps the BytesParser 'text' argument should really be bytes, but it hasn't been, it has been text, so for backward compatibility and for consistency with the regular Parser class, I'm keeping it as 'text'. files: Doc/library/email.parser.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/library/email.parser.rst b/Doc/library/email.parser.rst --- a/Doc/library/email.parser.rst +++ b/Doc/library/email.parser.rst @@ -202,12 +202,12 @@ reading the headers or not. The default is ``False``, meaning it parses the entire contents of the file. - .. method:: parsebytes(bytes, headersonly=False) + .. method:: parsebytes(text, headersonly=False) - Similar to the :meth:`parse` method, except it takes a byte string object - instead of a file-like object. Calling this method on a byte string is - exactly equivalent to wrapping *text* in a :class:`~io.BytesIO` instance - first and calling :meth:`parse`. + Similar to the :meth:`parse` method, except it takes a :term:`bytes-like + object` instead of a file-like object. Calling this method is equivalent + to wrapping *text* in a :class:`~io.BytesIO` instance first and calling + :meth:`parse`. Optional *headersonly* is as with the :meth:`parse` method. @@ -233,7 +233,7 @@ .. function:: message_from_bytes(s, _class=email.message.Message, *, \ policy=policy.compat32) - Return a message object structure from a byte string. This is exactly + Return a message object structure from a :term:`bytes-like object`. This is exactly equivalent to ``BytesParser().parsebytes(s)``. Optional *_class* and *strict* are interpreted as with the :class:`~email.parser.Parser` class constructor. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 00:51:10 2016 From: python-checkins at python.org (terry.reedy) Date: Wed, 31 Aug 2016 04:51:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327891=3A_Consiste?= =?utf-8?q?ntly_group_and_sort_imports_within_idlelib_modules=2E?= Message-ID: <20160831045110.129968.71641.79677EBE@psf.io> https://hg.python.org/cpython/rev/96ac4cd43a20 changeset: 102965:96ac4cd43a20 user: Terry Jan Reedy date: Wed Aug 31 00:50:55 2016 -0400 summary: Issue #27891: Consistently group and sort imports within idlelib modules. files: Lib/idlelib/README.txt | 21 +++++++- Lib/idlelib/autocomplete.py | 17 +++--- Lib/idlelib/autocomplete_w.py | 3 +- Lib/idlelib/autoexpand.py | 3 +- Lib/idlelib/browser.py | 6 +- Lib/idlelib/calltips.py | 2 +- Lib/idlelib/codecontext.py | 6 +- Lib/idlelib/colorizer.py | 9 +- Lib/idlelib/config.py | 2 +- Lib/idlelib/configdialog.py | 6 +- Lib/idlelib/debugger.py | 8 +- Lib/idlelib/debugobj.py | 3 +- Lib/idlelib/dynoption.py | 1 + Lib/idlelib/editor.py | 41 ++++++++------- Lib/idlelib/filelist.py | 1 + Lib/idlelib/grep.py | 11 ++- Lib/idlelib/help.py | 2 + Lib/idlelib/help_about.py | 5 +- Lib/idlelib/history.py | 4 +- Lib/idlelib/hyperparser.py | 5 +- Lib/idlelib/idle_test/test_iomenu.py | 5 +- Lib/idlelib/macosx.py | 3 +- Lib/idlelib/multicall.py | 4 +- Lib/idlelib/outwin.py | 7 +- Lib/idlelib/paragraph.py | 4 +- Lib/idlelib/parenmatch.py | 1 - Lib/idlelib/pathbrowser.py | 7 +- Lib/idlelib/percolator.py | 2 +- Lib/idlelib/pyparse.py | 2 +- Lib/idlelib/pyshell.py | 28 +++++----- Lib/idlelib/query.py | 3 +- Lib/idlelib/replace.py | 6 +- Lib/idlelib/rpc.py | 26 ++++++--- Lib/idlelib/run.py | 24 ++++---- Lib/idlelib/runscript.py | 3 +- Lib/idlelib/scrolledlist.py | 4 +- Lib/idlelib/search.py | 1 + Lib/idlelib/searchbase.py | 1 + Lib/idlelib/searchengine.py | 3 + Lib/idlelib/stackviewer.py | 11 +++- Lib/idlelib/statusbar.py | 2 + Lib/idlelib/tabbedpages.py | 3 + Lib/idlelib/textview.py | 2 +- Lib/idlelib/tree.py | 4 +- Lib/idlelib/undo.py | 7 +- Lib/idlelib/windows.py | 2 + Lib/idlelib/zoomheight.py | 2 + 47 files changed, 199 insertions(+), 124 deletions(-) diff --git a/Lib/idlelib/README.txt b/Lib/idlelib/README.txt --- a/Lib/idlelib/README.txt +++ b/Lib/idlelib/README.txt @@ -228,4 +228,23 @@ Center Insert # eEW.center_insert_event - + + +CODE STYLE -- Generally PEP 8. + +import +------ +Put import at the top, unless there is a good reason otherwise. +PEP 8 says to group stdlib, 3rd-party dependencies, and package imports. +For idlelib, the groups are general stdlib, tkinter, and idlelib. +Sort modules within each group, except that tkinter.ttk follows tkinter. +Sort 'from idlelib import mod1' and 'from idlelib.mod2 import object' +together by module, ignoring within module objects. +Put 'import __main__' after other idlelib imports. + +Imports only needed for testing are not at the top but are put in the +htest function def or the "if __name__ == '__main__'" clause. + +Within module imports like "from idlelib.mod import class" may cause +circular imports to deadlock. Even without this, circular imports may +require at least one of the imports to be delayed until a function call. diff --git a/Lib/idlelib/autocomplete.py b/Lib/idlelib/autocomplete.py --- a/Lib/idlelib/autocomplete.py +++ b/Lib/idlelib/autocomplete.py @@ -4,26 +4,27 @@ a window with all available names, for the user to select from. """ import os +import string import sys -import string -from idlelib.config import idleConf - -# This string includes all chars that may be in an identifier -ID_CHARS = string.ascii_letters + string.digits + "_" - -# These constants represent the two different types of completions +# These constants represent the two different types of completions. +# They must be defined here so autocomple_w can import them. COMPLETE_ATTRIBUTES, COMPLETE_FILES = range(1, 2+1) from idlelib import autocomplete_w +from idlelib.config import idleConf from idlelib.hyperparser import HyperParser +import __main__ -import __main__ +# This string includes all chars that may be in an identifier. +# TODO Update this here and elsewhere. +ID_CHARS = string.ascii_letters + string.digits + "_" SEPS = os.sep if os.altsep: # e.g. '/' on Windows... SEPS += os.altsep + class AutoComplete: menudefs = [ diff --git a/Lib/idlelib/autocomplete_w.py b/Lib/idlelib/autocomplete_w.py --- a/Lib/idlelib/autocomplete_w.py +++ b/Lib/idlelib/autocomplete_w.py @@ -3,8 +3,9 @@ """ from tkinter import * from tkinter.ttk import Scrollbar + +from idlelib.autocomplete import COMPLETE_FILES, COMPLETE_ATTRIBUTES from idlelib.multicall import MC_SHIFT -from idlelib.autocomplete import COMPLETE_FILES, COMPLETE_ATTRIBUTES HIDE_VIRTUAL_EVENT_NAME = "<>" HIDE_SEQUENCES = ("", "") diff --git a/Lib/idlelib/autoexpand.py b/Lib/idlelib/autoexpand.py --- a/Lib/idlelib/autoexpand.py +++ b/Lib/idlelib/autoexpand.py @@ -12,8 +12,8 @@ This is an extension file and there is only one instance of AutoExpand. ''' +import re import string -import re ###$ event <> ###$ win @@ -100,7 +100,6 @@ i = i-1 return line[i:] - if __name__ == '__main__': import unittest unittest.main('idlelib.idle_test.test_autoexpand', verbosity=2) diff --git a/Lib/idlelib/browser.py b/Lib/idlelib/browser.py --- a/Lib/idlelib/browser.py +++ b/Lib/idlelib/browser.py @@ -11,13 +11,13 @@ """ import os +import pyclbr import sys -import pyclbr +from idlelib.config import idleConf from idlelib import pyshell +from idlelib.tree import TreeNode, TreeItem, ScrolledCanvas from idlelib.windows import ListedToplevel -from idlelib.tree import TreeNode, TreeItem, ScrolledCanvas -from idlelib.config import idleConf file_open = None # Method...Item and Class...Item use this. # Normally pyshell.flist.open, but there is no pyshell.flist for htest. diff --git a/Lib/idlelib/calltips.py b/Lib/idlelib/calltips.py --- a/Lib/idlelib/calltips.py +++ b/Lib/idlelib/calltips.py @@ -5,7 +5,6 @@ which disappear when you type a closing parenthesis. """ -import __main__ import inspect import re import sys @@ -14,6 +13,7 @@ from idlelib import calltip_w from idlelib.hyperparser import HyperParser +import __main__ class CallTips: diff --git a/Lib/idlelib/codecontext.py b/Lib/idlelib/codecontext.py --- a/Lib/idlelib/codecontext.py +++ b/Lib/idlelib/codecontext.py @@ -9,10 +9,12 @@ not open blocks are not shown in the context hints pane. """ +import re +from sys import maxsize as INFINITY + import tkinter from tkinter.constants import TOP, LEFT, X, W, SUNKEN -import re -from sys import maxsize as INFINITY + from idlelib.config import idleConf BLOCKOPENERS = {"class", "def", "elif", "else", "except", "finally", "for", diff --git a/Lib/idlelib/colorizer.py b/Lib/idlelib/colorizer.py --- a/Lib/idlelib/colorizer.py +++ b/Lib/idlelib/colorizer.py @@ -1,9 +1,10 @@ +import builtins +import keyword +import re import time -import re -import keyword -import builtins + +from idlelib.config import idleConf from idlelib.delegator import Delegator -from idlelib.config import idleConf DEBUG = False diff --git a/Lib/idlelib/config.py b/Lib/idlelib/config.py --- a/Lib/idlelib/config.py +++ b/Lib/idlelib/config.py @@ -18,10 +18,10 @@ """ # TODOs added Oct 2014, tjr +from configparser import ConfigParser import os import sys -from configparser import ConfigParser from tkinter.font import Font, nametofont class InvalidConfigType(Exception): pass diff --git a/Lib/idlelib/configdialog.py b/Lib/idlelib/configdialog.py --- a/Lib/idlelib/configdialog.py +++ b/Lib/idlelib/configdialog.py @@ -11,17 +11,17 @@ """ from tkinter import * from tkinter.ttk import Scrollbar -import tkinter.messagebox as tkMessageBox import tkinter.colorchooser as tkColorChooser import tkinter.font as tkFont +import tkinter.messagebox as tkMessageBox from idlelib.config import idleConf +from idlelib.config_key import GetKeysDialog from idlelib.dynoption import DynOptionMenu -from idlelib.config_key import GetKeysDialog +from idlelib import macosx from idlelib.query import SectionName, HelpSource from idlelib.tabbedpages import TabbedPageSet from idlelib.textview import view_text -from idlelib import macosx class ConfigDialog(Toplevel): diff --git a/Lib/idlelib/debugger.py b/Lib/idlelib/debugger.py --- a/Lib/idlelib/debugger.py +++ b/Lib/idlelib/debugger.py @@ -1,10 +1,12 @@ +import bdb import os -import bdb + from tkinter import * from tkinter.ttk import Scrollbar + +from idlelib import macosx +from idlelib.scrolledlist import ScrolledList from idlelib.windows import ListedToplevel -from idlelib.scrolledlist import ScrolledList -from idlelib import macosx class Idb(bdb.Bdb): diff --git a/Lib/idlelib/debugobj.py b/Lib/idlelib/debugobj.py --- a/Lib/idlelib/debugobj.py +++ b/Lib/idlelib/debugobj.py @@ -8,11 +8,10 @@ # XXX TO DO: # - for classes/modules, add "open source" to object browser +from reprlib import Repr from idlelib.tree import TreeItem, TreeNode, ScrolledCanvas -from reprlib import Repr - myrepr = Repr() myrepr.maxstring = 100 myrepr.maxother = 100 diff --git a/Lib/idlelib/dynoption.py b/Lib/idlelib/dynoption.py --- a/Lib/idlelib/dynoption.py +++ b/Lib/idlelib/dynoption.py @@ -3,6 +3,7 @@ and setting of highlightthickness """ import copy + from tkinter import OptionMenu, _setit, StringVar, Button class DynOptionMenu(OptionMenu): diff --git a/Lib/idlelib/editor.py b/Lib/idlelib/editor.py --- a/Lib/idlelib/editor.py +++ b/Lib/idlelib/editor.py @@ -6,24 +6,28 @@ import re import string import sys +import tokenize +import traceback +import webbrowser + from tkinter import * from tkinter.ttk import Scrollbar import tkinter.simpledialog as tkSimpleDialog import tkinter.messagebox as tkMessageBox -import traceback -import webbrowser +from idlelib.config import idleConf +from idlelib import configdialog +from idlelib import grep +from idlelib import help +from idlelib import help_about +from idlelib import macosx from idlelib.multicall import MultiCallCreator +from idlelib import pyparse from idlelib import query +from idlelib import replace +from idlelib import search +from idlelib import textview from idlelib import windows -from idlelib import search -from idlelib import grep -from idlelib import replace -from idlelib import pyparse -from idlelib.config import idleConf -from idlelib import help_about, textview, configdialog -from idlelib import macosx -from idlelib import help # The default tab setting for a Text widget, in average-width characters. TK_TABWIDTH_DEFAULT = 8 @@ -1515,9 +1519,6 @@ break return raw, effective -import tokenize -_tokenize = tokenize -del tokenize class IndentSearcher(object): @@ -1542,8 +1543,8 @@ return self.text.get(mark, mark + " lineend+1c") def tokeneater(self, type, token, start, end, line, - INDENT=_tokenize.INDENT, - NAME=_tokenize.NAME, + INDENT=tokenize.INDENT, + NAME=tokenize.NAME, OPENERS=('class', 'def', 'for', 'if', 'try', 'while')): if self.finished: pass @@ -1554,19 +1555,19 @@ self.finished = 1 def run(self): - save_tabsize = _tokenize.tabsize - _tokenize.tabsize = self.tabwidth + save_tabsize = tokenize.tabsize + tokenize.tabsize = self.tabwidth try: try: - tokens = _tokenize.generate_tokens(self.readline) + tokens = tokenize.generate_tokens(self.readline) for token in tokens: self.tokeneater(*token) - except (_tokenize.TokenError, SyntaxError): + except (tokenize.TokenError, SyntaxError): # since we cut off the tokenizer early, we can trigger # spurious errors pass finally: - _tokenize.tabsize = save_tabsize + tokenize.tabsize = save_tabsize return self.blkopenline, self.indentedline ### end autoindent code ### diff --git a/Lib/idlelib/filelist.py b/Lib/idlelib/filelist.py --- a/Lib/idlelib/filelist.py +++ b/Lib/idlelib/filelist.py @@ -1,4 +1,5 @@ import os + from tkinter import * import tkinter.messagebox as tkMessageBox diff --git a/Lib/idlelib/grep.py b/Lib/idlelib/grep.py --- a/Lib/idlelib/grep.py +++ b/Lib/idlelib/grep.py @@ -1,11 +1,14 @@ +import fnmatch import os -import fnmatch import sys + from tkinter import StringVar, BooleanVar from tkinter.ttk import Checkbutton + +from idlelib.searchbase import SearchDialogBase from idlelib import searchengine -from idlelib.searchbase import SearchDialogBase -# Importing OutputWindow fails due to import loop + +# Importing OutputWindow here fails due to import loop # EditorWindow -> GrepDialop -> OutputWindow -> EditorWindow def grep(text, io=None, flist=None): @@ -127,9 +130,9 @@ def _grep_dialog(parent): # htest # - from idlelib.pyshell import PyShellFileList from tkinter import Toplevel, Text, SEL, END from tkinter.ttk import Button + from idlelib.pyshell import PyShellFileList top = Toplevel(parent) top.title("Test GrepDialog") x, y = map(int, parent.geometry().split('+')[1:]) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -27,9 +27,11 @@ from html.parser import HTMLParser from os.path import abspath, dirname, isfile, join from platform import python_version + from tkinter import Toplevel, Frame, Text, Menu from tkinter.ttk import Menubutton, Scrollbar from tkinter import font as tkfont + from idlelib.config import idleConf ## About IDLE ## diff --git a/Lib/idlelib/help_about.py b/Lib/idlelib/help_about.py --- a/Lib/idlelib/help_about.py +++ b/Lib/idlelib/help_about.py @@ -1,12 +1,14 @@ """About Dialog for IDLE """ - import os from sys import version + from tkinter import * + from idlelib import textview + class AboutDialog(Toplevel): """Modal about dialog for idle @@ -144,6 +146,7 @@ def Ok(self, event=None): self.destroy() + if __name__ == '__main__': import unittest unittest.main('idlelib.idle_test.test_help_about', verbosity=2, exit=False) diff --git a/Lib/idlelib/history.py b/Lib/idlelib/history.py --- a/Lib/idlelib/history.py +++ b/Lib/idlelib/history.py @@ -2,6 +2,7 @@ from idlelib.config import idleConf + class History: ''' Implement Idle Shell history mechanism. @@ -99,6 +100,7 @@ self.pointer = None self.prefix = None + if __name__ == "__main__": from unittest import main - main('idlelib.idle_test.test_idlehistory', verbosity=2, exit=False) + main('idlelib.idle_test.test_history', verbosity=2, exit=False) diff --git a/Lib/idlelib/hyperparser.py b/Lib/idlelib/hyperparser.py --- a/Lib/idlelib/hyperparser.py +++ b/Lib/idlelib/hyperparser.py @@ -4,12 +4,11 @@ proper indentation of code. HyperParser gives additional information on the structure of code. """ +from keyword import iskeyword +import string -import string -from keyword import iskeyword from idlelib import pyparse - # all ASCII chars that may be in an identifier _ASCII_ID_CHARS = frozenset(string.ascii_letters + string.digits + "_") # all ASCII chars that may be the first char of an identifier diff --git a/Lib/idlelib/idle_test/test_iomenu.py b/Lib/idlelib/idle_test/test_iomenu.py --- a/Lib/idlelib/idle_test/test_iomenu.py +++ b/Lib/idlelib/idle_test/test_iomenu.py @@ -1,6 +1,7 @@ import unittest import io -from idlelib.pyshell import PseudoInputFile, PseudoOutputFile + +from idlelib.run import PseudoInputFile, PseudoOutputFile class S(str): @@ -230,4 +231,4 @@ if __name__ == '__main__': - unittest.main() + unittest.main(verbosity=2) diff --git a/Lib/idlelib/macosx.py b/Lib/idlelib/macosx.py --- a/Lib/idlelib/macosx.py +++ b/Lib/idlelib/macosx.py @@ -2,8 +2,9 @@ A number of functions that enhance IDLE on Mac OSX. """ from sys import platform # Used in _init_tk_type, changed by test. +import warnings + import tkinter -import warnings ## Define functions that query the Mac graphics type. diff --git a/Lib/idlelib/multicall.py b/Lib/idlelib/multicall.py --- a/Lib/idlelib/multicall.py +++ b/Lib/idlelib/multicall.py @@ -28,9 +28,9 @@ unless this conflicts with the first rule. Each function will be called at most once for each event. """ +import re +import sys -import sys -import re import tkinter # the event type constants, which define the meaning of mc_type diff --git a/Lib/idlelib/outwin.py b/Lib/idlelib/outwin.py --- a/Lib/idlelib/outwin.py +++ b/Lib/idlelib/outwin.py @@ -1,9 +1,12 @@ +import re + from tkinter import * +import tkinter.messagebox as tkMessageBox + from idlelib.editor import EditorWindow -import re -import tkinter.messagebox as tkMessageBox from idlelib import iomenu + class OutputWindow(EditorWindow): """An editor window that can serve as an output file. diff --git a/Lib/idlelib/paragraph.py b/Lib/idlelib/paragraph.py --- a/Lib/idlelib/paragraph.py +++ b/Lib/idlelib/paragraph.py @@ -14,10 +14,11 @@ spaces, they will not be considered part of the same block. * Fancy comments, like this bulleted list, aren't handled :-) """ +import re -import re from idlelib.config import idleConf + class FormatParagraph: menudefs = [ @@ -189,6 +190,7 @@ if m is None: return "" return m.group(1) + if __name__ == "__main__": import unittest unittest.main('idlelib.idle_test.test_paragraph', diff --git a/Lib/idlelib/parenmatch.py b/Lib/idlelib/parenmatch.py --- a/Lib/idlelib/parenmatch.py +++ b/Lib/idlelib/parenmatch.py @@ -4,7 +4,6 @@ paren. Paren here is used generically; the matching applies to parentheses, square brackets, and curly braces. """ - from idlelib.hyperparser import HyperParser from idlelib.config import idleConf diff --git a/Lib/idlelib/pathbrowser.py b/Lib/idlelib/pathbrowser.py --- a/Lib/idlelib/pathbrowser.py +++ b/Lib/idlelib/pathbrowser.py @@ -1,10 +1,10 @@ +import importlib.machinery import os import sys -import importlib.machinery -from idlelib.tree import TreeItem from idlelib.browser import ClassBrowser, ModuleBrowserTreeItem from idlelib.pyshell import PyShellFileList +from idlelib.tree import TreeItem class PathBrowser(ClassBrowser): @@ -24,6 +24,7 @@ def rootnode(self): return PathBrowserTreeItem() + class PathBrowserTreeItem(TreeItem): def GetText(self): @@ -36,6 +37,7 @@ sublist.append(item) return sublist + class DirBrowserTreeItem(TreeItem): def __init__(self, dir, packages=[]): @@ -95,6 +97,7 @@ sorted.sort() return sorted + def _path_browser(parent): # htest # flist = PyShellFileList(parent) PathBrowser(flist, _htest=True) diff --git a/Lib/idlelib/percolator.py b/Lib/idlelib/percolator.py --- a/Lib/idlelib/percolator.py +++ b/Lib/idlelib/percolator.py @@ -1,5 +1,5 @@ +from idlelib.delegator import Delegator from idlelib.redirector import WidgetRedirector -from idlelib.delegator import Delegator class Percolator: diff --git a/Lib/idlelib/pyparse.py b/Lib/idlelib/pyparse.py --- a/Lib/idlelib/pyparse.py +++ b/Lib/idlelib/pyparse.py @@ -1,6 +1,6 @@ +from collections import Mapping import re import sys -from collections import Mapping # Reason last stmt is continued (or C_NONE if it's not). (C_NONE, C_BACKSLASH, C_STRING_FIRST_LINE, diff --git a/Lib/idlelib/pyshell.py b/Lib/idlelib/pyshell.py --- a/Lib/idlelib/pyshell.py +++ b/Lib/idlelib/pyshell.py @@ -15,9 +15,13 @@ parent=root) sys.exit(1) +from code import InteractiveInterpreter import getopt +import io +import linecache import os import os.path +from platform import python_version, system import re import socket import subprocess @@ -25,23 +29,20 @@ import threading import time import tokenize +import warnings -import linecache -from code import InteractiveInterpreter -from platform import python_version, system - -from idlelib import testing +from idlelib import testing # bool value +from idlelib.colorizer import ColorDelegator +from idlelib.config import idleConf +from idlelib import debugger +from idlelib import debugger_r from idlelib.editor import EditorWindow, fixwordbreaks from idlelib.filelist import FileList -from idlelib.colorizer import ColorDelegator +from idlelib import macosx +from idlelib.outwin import OutputWindow +from idlelib import rpc +from idlelib.run import idle_formatwarning, PseudoInputFile, PseudoOutputFile from idlelib.undo import UndoDelegator -from idlelib.outwin import OutputWindow -from idlelib.config import idleConf -from idlelib.run import idle_formatwarning, PseudoInputFile, PseudoOutputFile -from idlelib import rpc -from idlelib import debugger -from idlelib import debugger_r -from idlelib import macosx HOST = '127.0.0.1' # python execution server on localhost loopback PORT = 0 # someday pass in host, port for remote debug capability @@ -51,7 +52,6 @@ # temporarily redirect the stream to the shell window to display warnings when # checking user's code. warning_stream = sys.__stderr__ # None, at least on Windows, if no console. -import warnings def idle_showwarning( message, category, filename, lineno, file=None, line=None): diff --git a/Lib/idlelib/query.py b/Lib/idlelib/query.py --- a/Lib/idlelib/query.py +++ b/Lib/idlelib/query.py @@ -23,9 +23,10 @@ import importlib import os from sys import executable, platform # Platform is set for one test. + from tkinter import Toplevel, StringVar, W, E, N, S +from tkinter.ttk import Frame, Button, Entry, Label from tkinter import filedialog -from tkinter.ttk import Frame, Button, Entry, Label from tkinter.font import Font class Query(Toplevel): diff --git a/Lib/idlelib/replace.py b/Lib/idlelib/replace.py --- a/Lib/idlelib/replace.py +++ b/Lib/idlelib/replace.py @@ -3,12 +3,12 @@ Defines various replace related functions like replace, replace all, replace+find. """ +import re + from tkinter import StringVar, TclError +from idlelib.searchbase import SearchDialogBase from idlelib import searchengine -from idlelib.searchbase import SearchDialogBase -import re - def replace(text): """Returns a singleton ReplaceDialog instance.The single dialog diff --git a/Lib/idlelib/rpc.py b/Lib/idlelib/rpc.py --- a/Lib/idlelib/rpc.py +++ b/Lib/idlelib/rpc.py @@ -26,23 +26,21 @@ accomplished in Idle. """ - -import sys +import builtins +import copyreg +import io +import marshal import os -import io +import pickle +import queue +import select import socket -import select import socketserver import struct -import pickle +import sys import threading -import queue import traceback -import copyreg import types -import marshal -import builtins - def unpickle_code(ms): co = marshal.loads(ms) @@ -60,10 +58,12 @@ p.dump(obj) return f.getvalue() + class CodePickler(pickle.Pickler): dispatch_table = {types.CodeType: pickle_code} dispatch_table.update(copyreg.dispatch_table) + BUFSIZE = 8*1024 LOCALHOST = '127.0.0.1' @@ -487,16 +487,19 @@ # Token mix-in class pass + def remoteref(obj): oid = id(obj) objecttable[oid] = obj return RemoteProxy(oid) + class RemoteProxy(object): def __init__(self, oid): self.oid = oid + class RPCHandler(socketserver.BaseRequestHandler, SocketIO): debugging = False @@ -514,6 +517,7 @@ def get_remote_proxy(self, oid): return RPCProxy(self, oid) + class RPCClient(SocketIO): debugging = False @@ -539,6 +543,7 @@ def get_remote_proxy(self, oid): return RPCProxy(self, oid) + class RPCProxy(object): __methods = None @@ -587,6 +592,7 @@ if not callable(attr): attributes[name] = 1 + class MethodProxy(object): def __init__(self, sockio, oid, name): diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py --- a/Lib/idlelib/run.py +++ b/Lib/idlelib/run.py @@ -2,21 +2,21 @@ import linecache import queue import sys +import time +import traceback import _thread as thread import threading -import time -import traceback -import tkinter +import warnings -from idlelib import calltips -from idlelib import autocomplete +import tkinter # Tcl, deletions, messagebox if startup fails -from idlelib import debugger_r -from idlelib import debugobj_r -from idlelib import stackviewer -from idlelib import rpc -from idlelib import iomenu - +from idlelib import autocomplete # AutoComplete, fetch_encodings +from idlelib import calltips # CallTips +from idlelib import debugger_r # start_debugger +from idlelib import debugobj_r # remote_object_tree_item +from idlelib import iomenu # encoding +from idlelib import rpc # multiple objects +from idlelib import stackviewer # StackTreeItem import __main__ for mod in ('simpledialog', 'messagebox', 'font', @@ -27,7 +27,6 @@ LOCALHOST = '127.0.0.1' -import warnings def idle_formatwarning(message, category, filename, lineno, line=None): """Format warnings the IDLE way.""" @@ -280,6 +279,7 @@ capture_warnings(False) sys.exit(0) + class MyRPCServer(rpc.RPCServer): def handle_error(self, request, client_address): diff --git a/Lib/idlelib/runscript.py b/Lib/idlelib/runscript.py --- a/Lib/idlelib/runscript.py +++ b/Lib/idlelib/runscript.py @@ -20,11 +20,12 @@ import os import tabnanny import tokenize + import tkinter.messagebox as tkMessageBox -from idlelib import pyshell from idlelib.config import idleConf from idlelib import macosx +from idlelib import pyshell indent_message = """Error: Inconsistent indentation detected! diff --git a/Lib/idlelib/scrolledlist.py b/Lib/idlelib/scrolledlist.py --- a/Lib/idlelib/scrolledlist.py +++ b/Lib/idlelib/scrolledlist.py @@ -1,6 +1,8 @@ from tkinter import * +from tkinter.ttk import Scrollbar + from idlelib import macosx -from tkinter.ttk import Scrollbar + class ScrolledList: diff --git a/Lib/idlelib/search.py b/Lib/idlelib/search.py --- a/Lib/idlelib/search.py +++ b/Lib/idlelib/search.py @@ -24,6 +24,7 @@ "Handle the editor edit menu item and corresponding event." return _setup(text).find_selection(text) + class SearchDialog(SearchDialogBase): def create_widgets(self): diff --git a/Lib/idlelib/searchbase.py b/Lib/idlelib/searchbase.py --- a/Lib/idlelib/searchbase.py +++ b/Lib/idlelib/searchbase.py @@ -3,6 +3,7 @@ from tkinter import Toplevel, Frame from tkinter.ttk import Entry, Label, Button, Checkbutton, Radiobutton + class SearchDialogBase: '''Create most of a 3 or 4 row, 3 column search dialog. diff --git a/Lib/idlelib/searchengine.py b/Lib/idlelib/searchengine.py --- a/Lib/idlelib/searchengine.py +++ b/Lib/idlelib/searchengine.py @@ -1,5 +1,6 @@ '''Define SearchEngine for search dialogs.''' import re + from tkinter import StringVar, BooleanVar, TclError import tkinter.messagebox as tkMessageBox @@ -14,6 +15,7 @@ # This creates a cycle that persists until root is deleted. return root._searchengine + class SearchEngine: """Handles searching a text widget for Find, Replace, and Grep.""" @@ -186,6 +188,7 @@ col = len(chars) - 1 return None + def search_reverse(prog, chars, col): '''Search backwards and return an re match object or None. diff --git a/Lib/idlelib/stackviewer.py b/Lib/idlelib/stackviewer.py --- a/Lib/idlelib/stackviewer.py +++ b/Lib/idlelib/stackviewer.py @@ -1,11 +1,12 @@ +import linecache import os +import re import sys -import linecache -import re + import tkinter as tk +from idlelib.debugobj import ObjectTreeItem, make_objecttreeitem from idlelib.tree import TreeNode, TreeItem, ScrolledCanvas -from idlelib.debugobj import ObjectTreeItem, make_objecttreeitem def StackBrowser(root, flist=None, tb=None, top=None): if top is None: @@ -16,6 +17,7 @@ node = TreeNode(sc.canvas, None, item) node.expand() + class StackTreeItem(TreeItem): def __init__(self, flist=None, tb=None): @@ -54,6 +56,7 @@ sublist.append(item) return sublist + class FrameTreeItem(TreeItem): def __init__(self, info, flist): @@ -95,6 +98,7 @@ if os.path.isfile(filename): self.flist.gotofileline(filename, lineno) + class VariablesTreeItem(ObjectTreeItem): def GetText(self): @@ -119,6 +123,7 @@ sublist.append(item) return sublist + def _stack_viewer(parent): # htest # from idlelib.pyshell import PyShellFileList top = tk.Toplevel(parent) diff --git a/Lib/idlelib/statusbar.py b/Lib/idlelib/statusbar.py --- a/Lib/idlelib/statusbar.py +++ b/Lib/idlelib/statusbar.py @@ -1,5 +1,6 @@ from tkinter import Frame, Label + class MultiStatusBar(Frame): def __init__(self, master, **kw): @@ -17,6 +18,7 @@ label.config(width=width) label.config(text=text) + def _multistatus_bar(parent): # htest # from tkinter import Toplevel, Frame, Text, Button top = Toplevel(parent) diff --git a/Lib/idlelib/tabbedpages.py b/Lib/idlelib/tabbedpages.py --- a/Lib/idlelib/tabbedpages.py +++ b/Lib/idlelib/tabbedpages.py @@ -285,6 +285,7 @@ # placed hide it self.tab_set.lower() + class TabbedPageSet(Frame): """A Tkinter tabbed-pane widget. @@ -302,6 +303,7 @@ remove_page() methods. """ + class Page(object): """Abstract base class for TabbedPageSet's pages. @@ -467,6 +469,7 @@ self._tab_set.set_selected_tab(page_name) + def _tabbed_pages(parent): # htest # top=Toplevel(parent) x, y = map(int, parent.geometry().split('+')[1:]) diff --git a/Lib/idlelib/textview.py b/Lib/idlelib/textview.py --- a/Lib/idlelib/textview.py +++ b/Lib/idlelib/textview.py @@ -1,11 +1,11 @@ """Simple text browser for IDLE """ - from tkinter import * from tkinter.ttk import Scrollbar from tkinter.messagebox import showerror + class TextViewer(Toplevel): """A simple text viewer dialog for IDLE diff --git a/Lib/idlelib/tree.py b/Lib/idlelib/tree.py --- a/Lib/idlelib/tree.py +++ b/Lib/idlelib/tree.py @@ -15,10 +15,12 @@ # - optimize tree redraw after expand of subnode import os + from tkinter import * from tkinter.ttk import Scrollbar + +from idlelib.config import idleConf from idlelib import zoomheight -from idlelib.config import idleConf ICONDIR = "Icons" diff --git a/Lib/idlelib/undo.py b/Lib/idlelib/undo.py --- a/Lib/idlelib/undo.py +++ b/Lib/idlelib/undo.py @@ -1,5 +1,7 @@ import string + from idlelib.delegator import Delegator + # tkintter import not needed because module does not create widgets, # although many methods operate on text widget arguments. @@ -158,7 +160,6 @@ class Command: - # Base class for Undoable commands tags = None @@ -204,7 +205,6 @@ class InsertCommand(Command): - # Undoable insert command def __init__(self, index1, chars, tags=None): @@ -262,7 +262,6 @@ class DeleteCommand(Command): - # Undoable delete command def __init__(self, index1, index2=None): @@ -297,8 +296,8 @@ text.see('insert') ##sys.__stderr__.write("undo: %s\n" % self) + class CommandSequence(Command): - # Wrapper for a sequence of undoable cmds to be undone/redone # as a unit diff --git a/Lib/idlelib/windows.py b/Lib/idlelib/windows.py --- a/Lib/idlelib/windows.py +++ b/Lib/idlelib/windows.py @@ -1,5 +1,6 @@ from tkinter import * + class WindowList: def __init__(self): @@ -48,6 +49,7 @@ t, v, tb = sys.exc_info() print("warning: callback failed in WindowList", t, ":", v) + registry = WindowList() add_windows_to_menu = registry.add_windows_to_menu diff --git a/Lib/idlelib/zoomheight.py b/Lib/idlelib/zoomheight.py --- a/Lib/idlelib/zoomheight.py +++ b/Lib/idlelib/zoomheight.py @@ -5,6 +5,7 @@ from idlelib import macosx + class ZoomHeight: menudefs = [ @@ -20,6 +21,7 @@ top = self.editwin.top zoom_height(top) + def zoom_height(top): geom = top.wm_geometry() m = re.match(r"(\d+)x(\d+)\+(-?\d+)\+(-?\d+)", geom) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 03:12:34 2016 From: python-checkins at python.org (ethan.furman) Date: Wed, 31 Aug 2016 07:12:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_issue23591=3A_add_Flags=2C?= =?utf-8?q?_IntFlags=2C_and_tests?= Message-ID: <20160831071233.16351.5170.29438874@psf.io> https://hg.python.org/cpython/rev/39661e2ff030 changeset: 102966:39661e2ff030 user: Ethan Furman date: Wed Aug 31 00:12:15 2016 -0700 summary: issue23591: add Flags, IntFlags, and tests files: Lib/enum.py | 227 ++++++++++++++- Lib/test/test_enum.py | 452 +++++++++++++++++++++++++++++- 2 files changed, 668 insertions(+), 11 deletions(-) diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -1,5 +1,7 @@ import sys from types import MappingProxyType, DynamicClassAttribute +from functools import reduce +from operator import or_ as _or_ # try _collections first to reduce startup cost try: @@ -8,7 +10,7 @@ from collections import OrderedDict -__all__ = ['EnumMeta', 'Enum', 'IntEnum', 'unique'] +__all__ = ['EnumMeta', 'Enum', 'IntEnum', 'Flags', 'IntFlags', 'unique'] def _is_descriptor(obj): @@ -64,7 +66,10 @@ """ if _is_sunder(key): - if key not in ('_order_', ): + if key not in ( + '_order_', '_create_pseudo_member_', '_decompose_', + '_generate_next_value_', '_missing_', + ): raise ValueError('_names_ are reserved for future Enum use') elif _is_dunder(key): if key == '__order__': @@ -75,7 +80,7 @@ elif not _is_descriptor(value): if key in self: # enum overwriting a descriptor? - raise TypeError('Key already defined as: %r' % self[key]) + raise TypeError('%r already defined as: %r' % (key, self[key])) self._member_names.append(key) super().__setitem__(key, value) @@ -91,9 +96,15 @@ """Metaclass for Enum""" @classmethod def __prepare__(metacls, cls, bases): - return _EnumDict() + # create the namespace dict + enum_dict = _EnumDict() + # inherit previous flags and _generate_next_value_ function + member_type, first_enum = metacls._get_mixins_(bases) + if first_enum is not None: + enum_dict['_generate_next_value_'] = getattr(first_enum, '_generate_next_value_', None) + return enum_dict - def __new__(metacls, cls, bases, classdict): + def __new__(metacls, cls, bases, classdict, **kwds): # an Enum class is final once enumeration items have been defined; it # cannot be mixed with other types (int, float, etc.) if it has an # inherited __new__ unless a new __new__ is defined (or the resulting @@ -104,7 +115,7 @@ # save enum items into separate mapping so they don't get baked into # the new class - members = {k: classdict[k] for k in classdict._member_names} + enum_members = {k: classdict[k] for k in classdict._member_names} for name in classdict._member_names: del classdict[name] @@ -112,7 +123,7 @@ _order_ = classdict.pop('_order_', None) # check for illegal enum names (any others?) - invalid_names = set(members) & {'mro', } + invalid_names = set(enum_members) & {'mro', } if invalid_names: raise ValueError('Invalid enum member name: {0}'.format( ','.join(invalid_names))) @@ -156,7 +167,7 @@ # a custom __new__ is doing something funky with the values -- such as # auto-numbering ;) for member_name in classdict._member_names: - value = members[member_name] + value = enum_members[member_name] if not isinstance(value, tuple): args = (value, ) else: @@ -170,7 +181,10 @@ else: enum_member = __new__(enum_class, *args) if not hasattr(enum_member, '_value_'): - enum_member._value_ = member_type(*args) + if member_type is object: + enum_member._value_ = value + else: + enum_member._value_ = member_type(*args) value = enum_member._value_ enum_member._name_ = member_name enum_member.__objclass__ = enum_class @@ -344,13 +358,18 @@ """ metacls = cls.__class__ bases = (cls, ) if type is None else (type, cls) + _, first_enum = cls._get_mixins_(bases) classdict = metacls.__prepare__(class_name, bases) # special processing needed for names? if isinstance(names, str): names = names.replace(',', ' ').split() if isinstance(names, (tuple, list)) and isinstance(names[0], str): - names = [(e, i) for (i, e) in enumerate(names, start)] + original_names, names = names, [] + last_value = None + for count, name in enumerate(original_names): + last_value = first_enum._generate_next_value_(name, start, count, last_value) + names.append((name, last_value)) # Here, names is either an iterable of (name, value) or a mapping. for item in names: @@ -492,6 +511,16 @@ for member in cls._member_map_.values(): if member._value_ == value: return member + # still not found -- try _missing_ hook + return cls._missing_(value) + + @staticmethod + def _generate_next_value_(name, start, count, last_value): + if not count: + return start + return last_value + 1 + @classmethod + def _missing_(cls, value): raise ValueError("%r is not a valid %s" % (value, cls.__name__)) def __repr__(self): @@ -585,6 +614,184 @@ def _reduce_ex_by_name(self, proto): return self.name +class Flags(Enum): + """Support for flags""" + @staticmethod + def _generate_next_value_(name, start, count, last_value): + """ + Generate the next value when not given. + + name: the name of the member + start: the initital start value or None + count: the number of existing members + last_value: the last value assigned or None + """ + if not count: + return start if start is not None else 1 + high_bit = _high_bit(last_value) + return 2 ** (high_bit+1) + + @classmethod + def _missing_(cls, value): + original_value = value + if value < 0: + value = ~value + possible_member = cls._create_pseudo_member_(value) + for member in possible_member._decompose_(): + if member._name_ is None and member._value_ != 0: + raise ValueError('%r is not a valid %s' % (original_value, cls.__name__)) + if original_value < 0: + possible_member = ~possible_member + return possible_member + + @classmethod + def _create_pseudo_member_(cls, value): + pseudo_member = cls._value2member_map_.get(value, None) + if pseudo_member is None: + # construct a non-singleton enum pseudo-member + pseudo_member = object.__new__(cls) + pseudo_member._name_ = None + pseudo_member._value_ = value + cls._value2member_map_[value] = pseudo_member + return pseudo_member + + def _decompose_(self): + """Extract all members from the value.""" + value = self._value_ + members = [] + cls = self.__class__ + for member in sorted(cls, key=lambda m: m._value_, reverse=True): + while _high_bit(value) > _high_bit(member._value_): + unknown = self._create_pseudo_member_(2 ** _high_bit(value)) + members.append(unknown) + value &= ~unknown._value_ + if ( + (value & member._value_ == member._value_) + and (member._value_ or not members) + ): + value &= ~member._value_ + members.append(member) + if not members or value: + members.append(self._create_pseudo_member_(value)) + members = list(members) + return members + + def __contains__(self, other): + if not isinstance(other, self.__class__): + return NotImplemented + return other._value_ & self._value_ == other._value_ + + def __iter__(self): + if self.value == 0: + return iter([]) + else: + return iter(self._decompose_()) + + def __repr__(self): + cls = self.__class__ + if self._name_ is not None: + return '<%s.%s: %r>' % (cls.__name__, self._name_, self._value_) + members = self._decompose_() + if len(members) == 1 and members[0]._name_ is None: + return '<%s: %r>' % (cls.__name__, members[0]._value_) + else: + return '<%s.%s: %r>' % ( + cls.__name__, + '|'.join([str(m._name_ or m._value_) for m in members]), + self._value_, + ) + + def __str__(self): + cls = self.__class__ + if self._name_ is not None: + return '%s.%s' % (cls.__name__, self._name_) + members = self._decompose_() + if len(members) == 1 and members[0]._name_ is None: + return '%s.%r' % (cls.__name__, members[0]._value_) + else: + return '%s.%s' % ( + cls.__name__, + '|'.join([str(m._name_ or m._value_) for m in members]), + ) + + def __or__(self, other): + if not isinstance(other, self.__class__): + return NotImplemented + return self.__class__(self._value_ | other._value_) + + def __and__(self, other): + if not isinstance(other, self.__class__): + return NotImplemented + return self.__class__(self._value_ & other._value_) + + def __xor__(self, other): + if not isinstance(other, self.__class__): + return NotImplemented + return self.__class__(self._value_ ^ other._value_) + + def __invert__(self): + members = self._decompose_() + inverted_members = [m for m in self.__class__ if m not in members and not m._value_ & self._value_] + inverted = reduce(_or_, inverted_members, self.__class__(0)) + return self.__class__(inverted) + + +class IntFlags(int, Flags): + """Support for integer-based Flags""" + + @classmethod + def _create_pseudo_member_(cls, value): + pseudo_member = cls._value2member_map_.get(value, None) + if pseudo_member is None: + # construct a non-singleton enum pseudo-member + pseudo_member = int.__new__(cls, value) + pseudo_member._name_ = None + pseudo_member._value_ = value + cls._value2member_map_[value] = pseudo_member + return pseudo_member + + @classmethod + def _missing_(cls, value): + possible_member = cls._create_pseudo_member_(value) + return possible_member + + def __or__(self, other): + if not isinstance(other, (self.__class__, int)): + return NotImplemented + return self.__class__(self._value_ | self.__class__(other)._value_) + + def __and__(self, other): + if not isinstance(other, (self.__class__, int)): + return NotImplemented + return self.__class__(self._value_ & self.__class__(other)._value_) + + def __xor__(self, other): + if not isinstance(other, (self.__class__, int)): + return NotImplemented + return self.__class__(self._value_ ^ self.__class__(other)._value_) + + __ror__ = __or__ + __rand__ = __and__ + __rxor__ = __xor__ + + def __invert__(self): + # members = self._decompose_() + # inverted_members = [m for m in self.__class__ if m not in members and not m._value_ & self._value_] + # inverted = reduce(_or_, inverted_members, self.__class__(0)) + return self.__class__(~self._value_) + + + + +def _high_bit(value): + """return the highest bit set in value""" + bit = 0 + while 'looking for the highest bit': + limit = 2 ** bit + if limit > value: + return bit - 1 + bit += 1 + def unique(enumeration): """Class decorator for enumerations ensuring unique member values.""" duplicates = [] diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -3,7 +3,7 @@ import pydoc import unittest from collections import OrderedDict -from enum import Enum, IntEnum, EnumMeta, unique +from enum import Enum, IntEnum, EnumMeta, Flags, IntFlags, unique from io import StringIO from pickle import dumps, loads, PicklingError, HIGHEST_PROTOCOL from test import support @@ -1633,6 +1633,456 @@ verde = green +class TestFlags(unittest.TestCase): + """Tests of the Flags.""" + + class Perm(Flags): + R, W, X = 4, 2, 1 + + class Open(Flags): + RO = 0 + WO = 1 + RW = 2 + AC = 3 + CE = 1<<19 + + def test_str(self): + Perm = self.Perm + self.assertEqual(str(Perm.R), 'Perm.R') + self.assertEqual(str(Perm.W), 'Perm.W') + self.assertEqual(str(Perm.X), 'Perm.X') + self.assertEqual(str(Perm.R | Perm.W), 'Perm.R|W') + self.assertEqual(str(Perm.R | Perm.W | Perm.X), 'Perm.R|W|X') + self.assertEqual(str(Perm(0)), 'Perm.0') + self.assertEqual(str(~Perm.R), 'Perm.W|X') + self.assertEqual(str(~Perm.W), 'Perm.R|X') + self.assertEqual(str(~Perm.X), 'Perm.R|W') + self.assertEqual(str(~(Perm.R | Perm.W)), 'Perm.X') + self.assertEqual(str(~(Perm.R | Perm.W | Perm.X)), 'Perm.0') + self.assertEqual(str(Perm(~0)), 'Perm.R|W|X') + + Open = self.Open + self.assertEqual(str(Open.RO), 'Open.RO') + self.assertEqual(str(Open.WO), 'Open.WO') + self.assertEqual(str(Open.AC), 'Open.AC') + self.assertEqual(str(Open.RO | Open.CE), 'Open.CE') + self.assertEqual(str(Open.WO | Open.CE), 'Open.CE|WO') + self.assertEqual(str(~Open.RO), 'Open.CE|AC') + self.assertEqual(str(~Open.WO), 'Open.CE|RW') + self.assertEqual(str(~Open.AC), 'Open.CE') + self.assertEqual(str(~(Open.RO | Open.CE)), 'Open.AC') + self.assertEqual(str(~(Open.WO | Open.CE)), 'Open.RW') + + def test_repr(self): + Perm = self.Perm + self.assertEqual(repr(Perm.R), '') + self.assertEqual(repr(Perm.W), '') + self.assertEqual(repr(Perm.X), '') + self.assertEqual(repr(Perm.R | Perm.W), '') + self.assertEqual(repr(Perm.R | Perm.W | Perm.X), '') + self.assertEqual(repr(Perm(0)), '') + self.assertEqual(repr(~Perm.R), '') + self.assertEqual(repr(~Perm.W), '') + self.assertEqual(repr(~Perm.X), '') + self.assertEqual(repr(~(Perm.R | Perm.W)), '') + self.assertEqual(repr(~(Perm.R | Perm.W | Perm.X)), '') + self.assertEqual(repr(Perm(~0)), '') + + Open = self.Open + self.assertEqual(repr(Open.RO), '') + self.assertEqual(repr(Open.WO), '') + self.assertEqual(repr(Open.AC), '') + self.assertEqual(repr(Open.RO | Open.CE), '') + self.assertEqual(repr(Open.WO | Open.CE), '') + self.assertEqual(repr(~Open.RO), '') + self.assertEqual(repr(~Open.WO), '') + self.assertEqual(repr(~Open.AC), '') + self.assertEqual(repr(~(Open.RO | Open.CE)), '') + self.assertEqual(repr(~(Open.WO | Open.CE)), '') + + def test_or(self): + Perm = self.Perm + for i in Perm: + for j in Perm: + self.assertEqual((i | j), Perm(i.value | j.value)) + self.assertEqual((i | j).value, i.value | j.value) + self.assertIs(type(i | j), Perm) + for i in Perm: + self.assertIs(i | i, i) + Open = self.Open + self.assertIs(Open.RO | Open.CE, Open.CE) + + def test_and(self): + Perm = self.Perm + RW = Perm.R | Perm.W + RX = Perm.R | Perm.X + WX = Perm.W | Perm.X + RWX = Perm.R | Perm.W | Perm.X + values = list(Perm) + [RW, RX, WX, RWX, Perm(0)] + for i in values: + for j in values: + self.assertEqual((i & j).value, i.value & j.value) + self.assertIs(type(i & j), Perm) + for i in Perm: + self.assertIs(i & i, i) + self.assertIs(i & RWX, i) + self.assertIs(RWX & i, i) + Open = self.Open + self.assertIs(Open.RO & Open.CE, Open.RO) + + def test_xor(self): + Perm = self.Perm + for i in Perm: + for j in Perm: + self.assertEqual((i ^ j).value, i.value ^ j.value) + self.assertIs(type(i ^ j), Perm) + for i in Perm: + self.assertIs(i ^ Perm(0), i) + self.assertIs(Perm(0) ^ i, i) + Open = self.Open + self.assertIs(Open.RO ^ Open.CE, Open.CE) + self.assertIs(Open.CE ^ Open.CE, Open.RO) + + def test_invert(self): + Perm = self.Perm + RW = Perm.R | Perm.W + RX = Perm.R | Perm.X + WX = Perm.W | Perm.X + RWX = Perm.R | Perm.W | Perm.X + values = list(Perm) + [RW, RX, WX, RWX, Perm(0)] + for i in values: + self.assertIs(type(~i), Perm) + self.assertEqual(~~i, i) + for i in Perm: + self.assertIs(~~i, i) + Open = self.Open + self.assertIs(Open.WO & ~Open.WO, Open.RO) + self.assertIs((Open.WO|Open.CE) & ~Open.WO, Open.CE) + + def test_programatic_function_string(self): + Perm = Flags('Perm', 'R W X') + lst = list(Perm) + self.assertEqual(len(lst), len(Perm)) + self.assertEqual(len(Perm), 3, Perm) + self.assertEqual(lst, [Perm.R, Perm.W, Perm.X]) + for i, n in enumerate('R W X'.split()): + v = 1<') + self.assertEqual(repr(Perm.W), '') + self.assertEqual(repr(Perm.X), '') + self.assertEqual(repr(Perm.R | Perm.W), '') + self.assertEqual(repr(Perm.R | Perm.W | Perm.X), '') + self.assertEqual(repr(Perm.R | 8), '') + self.assertEqual(repr(Perm(0)), '') + self.assertEqual(repr(Perm(8)), '') + self.assertEqual(repr(~Perm.R), '') + self.assertEqual(repr(~Perm.W), '') + self.assertEqual(repr(~Perm.X), '') + self.assertEqual(repr(~(Perm.R | Perm.W)), '') + self.assertEqual(repr(~(Perm.R | Perm.W | Perm.X)), '') + self.assertEqual(repr(~(Perm.R | 8)), '') + self.assertEqual(repr(Perm(~0)), '') + self.assertEqual(repr(Perm(~8)), '') + + Open = self.Open + self.assertEqual(repr(Open.RO), '') + self.assertEqual(repr(Open.WO), '') + self.assertEqual(repr(Open.AC), '') + self.assertEqual(repr(Open.RO | Open.CE), '') + self.assertEqual(repr(Open.WO | Open.CE), '') + self.assertEqual(repr(Open(4)), '') + self.assertEqual(repr(~Open.RO), '') + self.assertEqual(repr(~Open.WO), '') + self.assertEqual(repr(~Open.AC), '') + self.assertEqual(repr(~(Open.RO | Open.CE)), '') + self.assertEqual(repr(~(Open.WO | Open.CE)), '') + self.assertEqual(repr(Open(~4)), '') + + def test_or(self): + Perm = self.Perm + for i in Perm: + for j in Perm: + self.assertEqual(i | j, i.value | j.value) + self.assertEqual((i | j).value, i.value | j.value) + self.assertIs(type(i | j), Perm) + for j in range(8): + self.assertEqual(i | j, i.value | j) + self.assertEqual((i | j).value, i.value | j) + self.assertIs(type(i | j), Perm) + self.assertEqual(j | i, j | i.value) + self.assertEqual((j | i).value, j | i.value) + self.assertIs(type(j | i), Perm) + for i in Perm: + self.assertIs(i | i, i) + self.assertIs(i | 0, i) + self.assertIs(0 | i, i) + Open = self.Open + self.assertIs(Open.RO | Open.CE, Open.CE) + + def test_and(self): + Perm = self.Perm + RW = Perm.R | Perm.W + RX = Perm.R | Perm.X + WX = Perm.W | Perm.X + RWX = Perm.R | Perm.W | Perm.X + values = list(Perm) + [RW, RX, WX, RWX, Perm(0)] + for i in values: + for j in values: + self.assertEqual(i & j, i.value & j.value, 'i is %r, j is %r' % (i, j)) + self.assertEqual((i & j).value, i.value & j.value, 'i is %r, j is %r' % (i, j)) + self.assertIs(type(i & j), Perm, 'i is %r, j is %r' % (i, j)) + for j in range(8): + self.assertEqual(i & j, i.value & j) + self.assertEqual((i & j).value, i.value & j) + self.assertIs(type(i & j), Perm) + self.assertEqual(j & i, j & i.value) + self.assertEqual((j & i).value, j & i.value) + self.assertIs(type(j & i), Perm) + for i in Perm: + self.assertIs(i & i, i) + self.assertIs(i & 7, i) + self.assertIs(7 & i, i) + Open = self.Open + self.assertIs(Open.RO & Open.CE, Open.RO) + + def test_xor(self): + Perm = self.Perm + for i in Perm: + for j in Perm: + self.assertEqual(i ^ j, i.value ^ j.value) + self.assertEqual((i ^ j).value, i.value ^ j.value) + self.assertIs(type(i ^ j), Perm) + for j in range(8): + self.assertEqual(i ^ j, i.value ^ j) + self.assertEqual((i ^ j).value, i.value ^ j) + self.assertIs(type(i ^ j), Perm) + self.assertEqual(j ^ i, j ^ i.value) + self.assertEqual((j ^ i).value, j ^ i.value) + self.assertIs(type(j ^ i), Perm) + for i in Perm: + self.assertIs(i ^ 0, i) + self.assertIs(0 ^ i, i) + Open = self.Open + self.assertIs(Open.RO ^ Open.CE, Open.CE) + self.assertIs(Open.CE ^ Open.CE, Open.RO) + + def test_invert(self): + Perm = self.Perm + RW = Perm.R | Perm.W + RX = Perm.R | Perm.X + WX = Perm.W | Perm.X + RWX = Perm.R | Perm.W | Perm.X + values = list(Perm) + [RW, RX, WX, RWX, Perm(0)] + for i in values: + self.assertEqual(~i, ~i.value) + self.assertEqual((~i).value, ~i.value) + self.assertIs(type(~i), Perm) + self.assertEqual(~~i, i) + for i in Perm: + self.assertIs(~~i, i) + Open = self.Open + self.assertIs(Open.WO & ~Open.WO, Open.RO) + self.assertIs((Open.WO|Open.CE) & ~Open.WO, Open.CE) + + def test_programatic_function_string(self): + Perm = IntFlags('Perm', 'R W X') + lst = list(Perm) + self.assertEqual(len(lst), len(Perm)) + self.assertEqual(len(Perm), 3, Perm) + self.assertEqual(lst, [Perm.R, Perm.W, Perm.X]) + for i, n in enumerate('R W X'.split()): + v = 1< https://hg.python.org/cpython/rev/1340e298aa7e changeset: 102967:1340e298aa7e user: Vinay Sajip date: Wed Aug 31 08:22:29 2016 +0100 summary: Closes #27904: Improved logging statements to defer formatting until needed. files: Doc/library/contextlib.rst | 4 ++-- Doc/library/shutil.rst | 2 +- Doc/library/typing.rst | 2 +- Doc/whatsnew/3.2.rst | 4 ++-- Lib/asyncio/base_events.py | 4 ++-- Lib/distutils/archive_util.py | 2 +- Lib/distutils/cmd.py | 3 +-- Lib/distutils/command/bdist_dumb.py | 2 +- Lib/distutils/command/build_ext.py | 6 +++--- Lib/distutils/command/config.py | 2 +- Lib/distutils/command/install.py | 2 +- Lib/distutils/command/register.py | 6 +++--- Lib/distutils/command/sdist.py | 2 +- Tools/ssl/test_multiple_versions.py | 6 +++--- setup.py | 4 ++-- 15 files changed, 25 insertions(+), 26 deletions(-) diff --git a/Doc/library/contextlib.rst b/Doc/library/contextlib.rst --- a/Doc/library/contextlib.rst +++ b/Doc/library/contextlib.rst @@ -590,10 +590,10 @@ self.name = name def __enter__(self): - logging.info('Entering: {}'.format(self.name)) + logging.info('Entering: %s', self.name) def __exit__(self, exc_type, exc, exc_tb): - logging.info('Exiting: {}'.format(self.name)) + logging.info('Exiting: %s', self.name) Instances of this class can be used as both a context manager:: diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -425,7 +425,7 @@ import logging def _logpath(path, names): - logging.info('Working in %s' % path) + logging.info('Working in %s', path) return [] # nothing will be ignored copytree(source, destination, ignore=_logpath) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -204,7 +204,7 @@ return self.value def log(self, message: str) -> None: - self.logger.info('{}: {}'.format(self.name, message)) + self.logger.info('%s: %s', self.name, message) ``Generic[T]`` as a base class defines that the class ``LoggedVar`` takes a single type parameter ``T`` . This also makes ``T`` valid as a type within the diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst --- a/Doc/whatsnew/3.2.rst +++ b/Doc/whatsnew/3.2.rst @@ -1253,9 +1253,9 @@ @contextmanager def track_entry_and_exit(name): - logging.info('Entering: {}'.format(name)) + logging.info('Entering: %s', name) yield - logging.info('Exiting: {}'.format(name)) + logging.info('Exiting: %s', name) Formerly, this would have only been usable as a context manager:: diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -1069,7 +1069,7 @@ transport = yield from self._make_subprocess_transport( protocol, cmd, True, stdin, stdout, stderr, bufsize, **kwargs) if self._debug: - logger.info('%s: %r' % (debug_log, transport)) + logger.info('%s: %r', debug_log, transport) return transport, protocol @coroutine @@ -1099,7 +1099,7 @@ protocol, popen_args, False, stdin, stdout, stderr, bufsize, **kwargs) if self._debug: - logger.info('%s: %r' % (debug_log, transport)) + logger.info('%s: %r', debug_log, transport) return transport, protocol def get_exception_handler(self): diff --git a/Lib/distutils/archive_util.py b/Lib/distutils/archive_util.py --- a/Lib/distutils/archive_util.py +++ b/Lib/distutils/archive_util.py @@ -171,7 +171,7 @@ path = os.path.normpath(os.path.join(dirpath, name)) if os.path.isfile(path): zip.write(path, path) - log.info("adding '%s'" % path) + log.info("adding '%s'", path) zip.close() return zip_filename diff --git a/Lib/distutils/cmd.py b/Lib/distutils/cmd.py --- a/Lib/distutils/cmd.py +++ b/Lib/distutils/cmd.py @@ -329,8 +329,7 @@ # -- External world manipulation ----------------------------------- def warn(self, msg): - log.warn("warning: %s: %s\n" % - (self.get_command_name(), msg)) + log.warn("warning: %s: %s\n", self.get_command_name(), msg) def execute(self, func, args, msg=None, level=1): util.execute(func, args, msg, dry_run=self.dry_run) diff --git a/Lib/distutils/command/bdist_dumb.py b/Lib/distutils/command/bdist_dumb.py --- a/Lib/distutils/command/bdist_dumb.py +++ b/Lib/distutils/command/bdist_dumb.py @@ -85,7 +85,7 @@ install.skip_build = self.skip_build install.warn_dir = 0 - log.info("installing to %s" % self.bdist_dir) + log.info("installing to %s", self.bdist_dir) self.run_command('install') # And make an archive relative to the root of the diff --git a/Lib/distutils/command/build_ext.py b/Lib/distutils/command/build_ext.py --- a/Lib/distutils/command/build_ext.py +++ b/Lib/distutils/command/build_ext.py @@ -363,9 +363,9 @@ ext_name, build_info = ext - log.warn(("old-style (ext_name, build_info) tuple found in " - "ext_modules for extension '%s'" - "-- please convert to Extension instance" % ext_name)) + log.warn("old-style (ext_name, build_info) tuple found in " + "ext_modules for extension '%s'" + "-- please convert to Extension instance", ext_name) if not (isinstance(ext_name, str) and extension_name_re.match(ext_name)): diff --git a/Lib/distutils/command/config.py b/Lib/distutils/command/config.py --- a/Lib/distutils/command/config.py +++ b/Lib/distutils/command/config.py @@ -337,7 +337,7 @@ If head is not None, will be dumped before the file content. """ if head is None: - log.info('%s' % filename) + log.info('%s', filename) else: log.info(head) file = open(filename) diff --git a/Lib/distutils/command/install.py b/Lib/distutils/command/install.py --- a/Lib/distutils/command/install.py +++ b/Lib/distutils/command/install.py @@ -385,7 +385,7 @@ else: opt_name = opt_name.translate(longopt_xlate) val = getattr(self, opt_name) - log.debug(" %s: %s" % (opt_name, val)) + log.debug(" %s: %s", opt_name, val) def finalize_unix(self): """Finalizes options for posix platforms.""" diff --git a/Lib/distutils/command/register.py b/Lib/distutils/command/register.py --- a/Lib/distutils/command/register.py +++ b/Lib/distutils/command/register.py @@ -94,7 +94,7 @@ ''' # send the info to the server and report the result (code, result) = self.post_to_server(self.build_post_data('verify')) - log.info('Server response (%s): %s' % (code, result)) + log.info('Server response (%s): %s', code, result) def send_metadata(self): ''' Send the metadata to the package index server. @@ -205,7 +205,7 @@ data['email'] = input(' EMail: ') code, result = self.post_to_server(data) if code != 200: - log.info('Server response (%s): %s' % (code, result)) + log.info('Server response (%s): %s', code, result) else: log.info('You will receive an email shortly.') log.info(('Follow the instructions in it to ' @@ -216,7 +216,7 @@ while not data['email']: data['email'] = input('Your email address: ') code, result = self.post_to_server(data) - log.info('Server response (%s): %s' % (code, result)) + log.info('Server response (%s): %s', code, result) def build_post_data(self, action): # figure the data to send - the metadata plus some additional diff --git a/Lib/distutils/command/sdist.py b/Lib/distutils/command/sdist.py --- a/Lib/distutils/command/sdist.py +++ b/Lib/distutils/command/sdist.py @@ -412,7 +412,7 @@ log.info(msg) for file in files: if not os.path.isfile(file): - log.warn("'%s' not a regular file -- skipping" % file) + log.warn("'%s' not a regular file -- skipping", file) else: dest = os.path.join(base_dir, file) self.copy_file(file, dest, link=link) diff --git a/Tools/ssl/test_multiple_versions.py b/Tools/ssl/test_multiple_versions.py --- a/Tools/ssl/test_multiple_versions.py +++ b/Tools/ssl/test_multiple_versions.py @@ -105,11 +105,11 @@ def _subprocess_call(self, cmd, stdout=subprocess.DEVNULL, env=None, **kwargs): - log.debug("Call '{}'".format(" ".join(cmd))) + log.debug("Call '%s'", " ".join(cmd)) return subprocess.check_call(cmd, stdout=stdout, env=env, **kwargs) def _subprocess_output(self, cmd, env=None, **kwargs): - log.debug("Call '{}'".format(" ".join(cmd))) + log.debug("Call '%s'", " ".join(cmd)) out = subprocess.check_output(cmd, env=env) return out.strip().decode("utf-8") @@ -168,7 +168,7 @@ if not self.has_src: self._download_openssl() else: - log.debug("Already has src {}".format(self.src_file)) + log.debug("Already has src %s", self.src_file) self._unpack_openssl() self._build_openssl() self._install_openssl() diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -186,7 +186,7 @@ if not list: return module if len(list) > 1: - log.info("WARNING: multiple copies of %s found"%module) + log.info("WARNING: multiple copies of %s found", module) return os.path.join(list[0], module) class PyBuildExt(build_ext): @@ -2213,7 +2213,7 @@ newfilename = filename + fullversion else: newfilename = filename + minoronly - log.info('renaming {} to {}'.format(filename, newfilename)) + log.info('renaming %s to %s', filename, newfilename) os.rename(filename, newfilename) newoutfiles.append(newfilename) if filename in updated_files: -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Wed Aug 31 10:17:13 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 31 Aug 2016 15:17:13 +0100 Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7 2016-08-31 Message-ID: <3f9fcc3d-264d-4ec3-adb4-b1898cd771b6@irsmsx101.ger.corp.intel.com> Results for project Python 2.7, build date 2016-08-31 02:47:12 +0000 commit: f478f9b88319 previous commit: 58ea646ef657 revision date: 2016-08-31 00:19:07 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.13% 0.23% 5.06% 5.03% :-) pybench 0.18% -0.06% 6.30% 3.74% :-( regex_v8 0.61% 0.09% -2.26% 10.97% :-) nbody 0.09% 0.10% 8.26% 2.29% :-) json_dump_v2 0.65% 0.53% 2.18% 11.18% :-| normal_startup 0.70% 0.05% -0.75% 2.19% :-) ssbench 0.33% 0.34% 3.23% 0.71% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-08-31/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Wed Aug 31 10:26:34 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 31 Aug 2016 15:26:34 +0100 Subject: [Python-checkins] NEUTRAL Benchmark Results for Python Default 2016-08-31 Message-ID: Results for project Python default, build date 2016-08-31 02:00:27 +0000 commit: 059f9f518834 previous commit: 7d73555c83ff revision date: 2016-08-31 01:17:25 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.24% 0.83% 10.07% 17.16% :-) pybench 3.05% -0.03% 6.25% 6.96% :-( regex_v8 2.76% -0.16% -3.99% 5.60% :-| nbody 0.09% 0.00% -1.30% 11.97% :-| json_dump_v2 0.38% 0.49% -1.97% 13.05% :-) normal_startup 0.99% 0.13% 2.25% 5.57% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-default-2016-08-31/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Wed Aug 31 11:40:15 2016 From: python-checkins at python.org (r.david.murray) Date: Wed, 31 Aug 2016 15:40:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2327904=3A_fix_distutils_?= =?utf-8?q?tests=2E?= Message-ID: <20160831154011.8356.82178.F2F593A7@psf.io> https://hg.python.org/cpython/rev/5f6dac170b9b changeset: 102968:5f6dac170b9b user: R David Murray date: Wed Aug 31 11:39:35 2016 -0400 summary: #27904: fix distutils tests. Patch by Ville Skytt?. files: Lib/distutils/tests/test_build_py.py | 3 ++- Lib/distutils/tests/test_install_lib.py | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Lib/distutils/tests/test_build_py.py b/Lib/distutils/tests/test_build_py.py --- a/Lib/distutils/tests/test_build_py.py +++ b/Lib/distutils/tests/test_build_py.py @@ -168,7 +168,8 @@ finally: sys.dont_write_bytecode = old_dont_write_bytecode - self.assertIn('byte-compiling is disabled', self.logs[0][1]) + self.assertIn('byte-compiling is disabled', + self.logs[0][1] % self.logs[0][2]) def test_suite(): diff --git a/Lib/distutils/tests/test_install_lib.py b/Lib/distutils/tests/test_install_lib.py --- a/Lib/distutils/tests/test_install_lib.py +++ b/Lib/distutils/tests/test_install_lib.py @@ -104,7 +104,8 @@ finally: sys.dont_write_bytecode = old_dont_write_bytecode - self.assertIn('byte-compiling is disabled', self.logs[0][1]) + self.assertIn('byte-compiling is disabled', + self.logs[0][1] % self.logs[0][2]) def test_suite(): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 11:44:34 2016 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 31 Aug 2016 15:44:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <20160831154432.14052.53941.2A680492@psf.io> https://hg.python.org/cpython/rev/16bba5b49441 changeset: 102970:16bba5b49441 parent: 102968:5f6dac170b9b parent: 102969:caf547c9e589 user: Raymond Hettinger date: Wed Aug 31 08:44:26 2016 -0700 summary: Merge files: Python/import.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Python/import.c b/Python/import.c --- a/Python/import.c +++ b/Python/import.c @@ -1047,7 +1047,7 @@ mod = _PyImport_FindExtensionObject(name, name); if (mod || PyErr_Occurred()) { Py_DECREF(name); - Py_INCREF(mod); + Py_XINCREF(mod); return mod; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 11:44:35 2016 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 31 Aug 2016 15:44:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3OTA5?= =?utf-8?q?=3A_Fix_INCREF_for_possible_NULL_value?= Message-ID: <20160831154432.22430.57262.CDAA9026@psf.io> https://hg.python.org/cpython/rev/caf547c9e589 changeset: 102969:caf547c9e589 branch: 3.5 parent: 102963:b8dd9ae08a91 user: Raymond Hettinger date: Wed Aug 31 08:44:11 2016 -0700 summary: Issue #27909: Fix INCREF for possible NULL value files: Python/import.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Python/import.c b/Python/import.c --- a/Python/import.c +++ b/Python/import.c @@ -1056,7 +1056,7 @@ mod = _PyImport_FindExtensionObject(name, name); if (mod || PyErr_Occurred()) { Py_DECREF(name); - Py_INCREF(mod); + Py_XINCREF(mod); return mod; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 12:09:15 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 31 Aug 2016 16:09:15 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3OTA3?= =?utf-8?q?=3A_variable_rename=2E_=28Ville_Skytt=C3=A4=29?= Message-ID: <20160831160914.98158.8001.DF67F1B6@psf.io> https://hg.python.org/cpython/rev/2a748320616b changeset: 102971:2a748320616b branch: 3.5 parent: 102969:caf547c9e589 user: Guido van Rossum date: Wed Aug 31 09:08:41 2016 -0700 summary: Issue #27907: variable rename. (Ville Skytt?) files: Lib/test/test_asyncio/test_events.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -763,11 +763,11 @@ addr = lsock.getsockname() message = b'test data' - reponse = None + response = None expected_response = b'roger' def client(): - global response + nonlocal response try: csock = socket.socket() if client_ssl is not None: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 12:09:15 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 31 Aug 2016 16:09:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327907=3A_variable_rename=2E_=28Ville_Skytt?= =?utf-8?b?w6QpIChNZXJnZSAzLjUtPjMuNik=?= Message-ID: <20160831160914.98179.7879.696A8F7D@psf.io> https://hg.python.org/cpython/rev/c9592e878dfa changeset: 102972:c9592e878dfa parent: 102970:16bba5b49441 parent: 102971:2a748320616b user: Guido van Rossum date: Wed Aug 31 09:09:04 2016 -0700 summary: Issue #27907: variable rename. (Ville Skytt?) (Merge 3.5->3.6) files: Lib/test/test_asyncio/test_events.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -763,11 +763,11 @@ addr = lsock.getsockname() message = b'test data' - reponse = None + response = None expected_response = b'roger' def client(): - global response + nonlocal response try: csock = socket.socket() if client_ssl is not None: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 12:53:32 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 31 Aug 2016 16:53:32 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogRG9uJ3Qgc2VsZWN0?= =?utf-8?q?_for_read_on_character_devices_in_=5FUnixWritePipeTransport=2E?= Message-ID: <20160831165332.16394.98524.22272967@psf.io> https://hg.python.org/cpython/rev/fa8fadc7f065 changeset: 102973:fa8fadc7f065 branch: 3.5 parent: 102971:2a748320616b user: Guido van Rossum date: Wed Aug 31 09:40:18 2016 -0700 summary: Don't select for read on character devices in _UnixWritePipeTransport. Upstream https://github.com/python/asyncio/pull/374 by Ron Frederick. files: Lib/asyncio/unix_events.py | 8 +- Lib/test/test_asyncio/test_events.py | 75 ++++++++++++++++ 2 files changed, 79 insertions(+), 4 deletions(-) diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py --- a/Lib/asyncio/unix_events.py +++ b/Lib/asyncio/unix_events.py @@ -422,10 +422,10 @@ self._pipe = pipe self._fileno = pipe.fileno() mode = os.fstat(self._fileno).st_mode + is_char = stat.S_ISCHR(mode) + is_fifo = stat.S_ISFIFO(mode) is_socket = stat.S_ISSOCK(mode) - if not (is_socket or - stat.S_ISFIFO(mode) or - stat.S_ISCHR(mode)): + if not (is_char or is_fifo or is_socket): raise ValueError("Pipe transport is only for " "pipes, sockets and character devices") _set_nonblocking(self._fileno) @@ -439,7 +439,7 @@ # On AIX, the reader trick (to be notified when the read end of the # socket is closed) only works for sockets. On other platforms it # works for pipes and sockets. (Exception: OS X 10.4? Issue #19294.) - if is_socket or not sys.platform.startswith("aix"): + if is_socket or (is_fifo and not sys.platform.startswith("aix")): # only start reading when connection_made() has been called self._loop.call_soon(self._loop.add_reader, self._fileno, self._read_ready) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -21,6 +21,8 @@ from unittest import mock import weakref +if sys.platform != 'win32': + import tty import asyncio from asyncio import proactor_events @@ -1626,6 +1628,79 @@ self.loop.run_until_complete(proto.done) self.assertEqual('CLOSED', proto.state) + @unittest.skipUnless(sys.platform != 'win32', + "Don't support pipes for Windows") + # select, poll and kqueue don't support character devices (PTY) on Mac OS X + # older than 10.6 (Snow Leopard) + @support.requires_mac_ver(10, 6) + def test_bidirectional_pty(self): + master, read_slave = os.openpty() + write_slave = os.dup(read_slave) + tty.setraw(read_slave) + + slave_read_obj = io.open(read_slave, 'rb', 0) + read_proto = MyReadPipeProto(loop=self.loop) + read_connect = self.loop.connect_read_pipe(lambda: read_proto, + slave_read_obj) + read_transport, p = self.loop.run_until_complete(read_connect) + self.assertIs(p, read_proto) + self.assertIs(read_transport, read_proto.transport) + self.assertEqual(['INITIAL', 'CONNECTED'], read_proto.state) + self.assertEqual(0, read_proto.nbytes) + + + slave_write_obj = io.open(write_slave, 'wb', 0) + write_proto = MyWritePipeProto(loop=self.loop) + write_connect = self.loop.connect_write_pipe(lambda: write_proto, + slave_write_obj) + write_transport, p = self.loop.run_until_complete(write_connect) + self.assertIs(p, write_proto) + self.assertIs(write_transport, write_proto.transport) + self.assertEqual('CONNECTED', write_proto.state) + + data = bytearray() + def reader(data): + chunk = os.read(master, 1024) + data += chunk + return len(data) + + write_transport.write(b'1') + test_utils.run_until(self.loop, lambda: reader(data) >= 1, timeout=10) + self.assertEqual(b'1', data) + self.assertEqual(['INITIAL', 'CONNECTED'], read_proto.state) + self.assertEqual('CONNECTED', write_proto.state) + + os.write(master, b'a') + test_utils.run_until(self.loop, lambda: read_proto.nbytes >= 1, + timeout=10) + self.assertEqual(['INITIAL', 'CONNECTED'], read_proto.state) + self.assertEqual(1, read_proto.nbytes) + self.assertEqual('CONNECTED', write_proto.state) + + write_transport.write(b'2345') + test_utils.run_until(self.loop, lambda: reader(data) >= 5, timeout=10) + self.assertEqual(b'12345', data) + self.assertEqual(['INITIAL', 'CONNECTED'], read_proto.state) + self.assertEqual('CONNECTED', write_proto.state) + + os.write(master, b'bcde') + test_utils.run_until(self.loop, lambda: read_proto.nbytes >= 5, + timeout=10) + self.assertEqual(['INITIAL', 'CONNECTED'], read_proto.state) + self.assertEqual(5, read_proto.nbytes) + self.assertEqual('CONNECTED', write_proto.state) + + os.close(master) + + read_transport.close() + self.loop.run_until_complete(read_proto.done) + self.assertEqual( + ['INITIAL', 'CONNECTED', 'EOF', 'CLOSED'], read_proto.state) + + write_transport.close() + self.loop.run_until_complete(write_proto.done) + self.assertEqual('CLOSED', write_proto.state) + def test_prompt_cancellation(self): r, w = test_utils.socketpair() r.setblocking(False) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 12:53:32 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 31 Aug 2016 16:53:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_ordering_i?= =?utf-8?q?ssues_in_UNIX_read/write_pipe_transport_constructors=2E?= Message-ID: <20160831165332.129728.2017.883A2895@psf.io> https://hg.python.org/cpython/rev/dfaa38a8c7ec changeset: 102974:dfaa38a8c7ec branch: 3.5 user: Guido van Rossum date: Wed Aug 31 09:42:38 2016 -0700 summary: Fix ordering issues in UNIX read/write pipe transport constructors. Upstream https://github.com/python/asyncio/pull/408 by Ron Frederick. files: Lib/asyncio/unix_events.py | 23 +++++++++++++++++------ 1 files changed, 17 insertions(+), 6 deletions(-) diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py --- a/Lib/asyncio/unix_events.py +++ b/Lib/asyncio/unix_events.py @@ -305,14 +305,20 @@ self._loop = loop self._pipe = pipe self._fileno = pipe.fileno() + self._protocol = protocol + self._closing = False + mode = os.fstat(self._fileno).st_mode if not (stat.S_ISFIFO(mode) or stat.S_ISSOCK(mode) or stat.S_ISCHR(mode)): + self._pipe = None + self._fileno = None + self._protocol = None raise ValueError("Pipe transport is for pipes/sockets only.") + _set_nonblocking(self._fileno) - self._protocol = protocol - self._closing = False + self._loop.call_soon(self._protocol.connection_made, self) # only start reading when connection_made() has been called self._loop.call_soon(self._loop.add_reader, @@ -421,18 +427,23 @@ self._extra['pipe'] = pipe self._pipe = pipe self._fileno = pipe.fileno() + self._protocol = protocol + self._buffer = [] + self._conn_lost = 0 + self._closing = False # Set when close() or write_eof() called. + mode = os.fstat(self._fileno).st_mode is_char = stat.S_ISCHR(mode) is_fifo = stat.S_ISFIFO(mode) is_socket = stat.S_ISSOCK(mode) if not (is_char or is_fifo or is_socket): + self._pipe = None + self._fileno = None + self._protocol = None raise ValueError("Pipe transport is only for " "pipes, sockets and character devices") + _set_nonblocking(self._fileno) - self._protocol = protocol - self._buffer = [] - self._conn_lost = 0 - self._closing = False # Set when close() or write_eof() called. self._loop.call_soon(self._protocol.connection_made, self) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 12:53:33 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 31 Aug 2016 16:53:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_asyncio_from_3=2E5?= Message-ID: <20160831165332.12898.54395.98CAA307@psf.io> https://hg.python.org/cpython/rev/fcf65749ef71 changeset: 102975:fcf65749ef71 parent: 102972:c9592e878dfa parent: 102974:dfaa38a8c7ec user: Guido van Rossum date: Wed Aug 31 09:47:08 2016 -0700 summary: Merge asyncio from 3.5 files: Lib/asyncio/unix_events.py | 33 ++++-- Lib/test/test_asyncio/test_events.py | 75 ++++++++++++++++ 2 files changed, 97 insertions(+), 11 deletions(-) diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py --- a/Lib/asyncio/unix_events.py +++ b/Lib/asyncio/unix_events.py @@ -305,14 +305,20 @@ self._loop = loop self._pipe = pipe self._fileno = pipe.fileno() + self._protocol = protocol + self._closing = False + mode = os.fstat(self._fileno).st_mode if not (stat.S_ISFIFO(mode) or stat.S_ISSOCK(mode) or stat.S_ISCHR(mode)): + self._pipe = None + self._fileno = None + self._protocol = None raise ValueError("Pipe transport is for pipes/sockets only.") + _set_nonblocking(self._fileno) - self._protocol = protocol - self._closing = False + self._loop.call_soon(self._protocol.connection_made, self) # only start reading when connection_made() has been called self._loop.call_soon(self._loop.add_reader, @@ -422,25 +428,30 @@ self._extra['pipe'] = pipe self._pipe = pipe self._fileno = pipe.fileno() - mode = os.fstat(self._fileno).st_mode - is_socket = stat.S_ISSOCK(mode) - if not (is_socket or - stat.S_ISFIFO(mode) or - stat.S_ISCHR(mode)): - raise ValueError("Pipe transport is only for " - "pipes, sockets and character devices") - _set_nonblocking(self._fileno) self._protocol = protocol self._buffer = [] self._conn_lost = 0 self._closing = False # Set when close() or write_eof() called. + mode = os.fstat(self._fileno).st_mode + is_char = stat.S_ISCHR(mode) + is_fifo = stat.S_ISFIFO(mode) + is_socket = stat.S_ISSOCK(mode) + if not (is_char or is_fifo or is_socket): + self._pipe = None + self._fileno = None + self._protocol = None + raise ValueError("Pipe transport is only for " + "pipes, sockets and character devices") + + _set_nonblocking(self._fileno) + self._loop.call_soon(self._protocol.connection_made, self) # On AIX, the reader trick (to be notified when the read end of the # socket is closed) only works for sockets. On other platforms it # works for pipes and sockets. (Exception: OS X 10.4? Issue #19294.) - if is_socket or not sys.platform.startswith("aix"): + if is_socket or (is_fifo and not sys.platform.startswith("aix")): # only start reading when connection_made() has been called self._loop.call_soon(self._loop.add_reader, self._fileno, self._read_ready) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -21,6 +21,8 @@ from unittest import mock import weakref +if sys.platform != 'win32': + import tty import asyncio from asyncio import proactor_events @@ -1626,6 +1628,79 @@ self.loop.run_until_complete(proto.done) self.assertEqual('CLOSED', proto.state) + @unittest.skipUnless(sys.platform != 'win32', + "Don't support pipes for Windows") + # select, poll and kqueue don't support character devices (PTY) on Mac OS X + # older than 10.6 (Snow Leopard) + @support.requires_mac_ver(10, 6) + def test_bidirectional_pty(self): + master, read_slave = os.openpty() + write_slave = os.dup(read_slave) + tty.setraw(read_slave) + + slave_read_obj = io.open(read_slave, 'rb', 0) + read_proto = MyReadPipeProto(loop=self.loop) + read_connect = self.loop.connect_read_pipe(lambda: read_proto, + slave_read_obj) + read_transport, p = self.loop.run_until_complete(read_connect) + self.assertIs(p, read_proto) + self.assertIs(read_transport, read_proto.transport) + self.assertEqual(['INITIAL', 'CONNECTED'], read_proto.state) + self.assertEqual(0, read_proto.nbytes) + + + slave_write_obj = io.open(write_slave, 'wb', 0) + write_proto = MyWritePipeProto(loop=self.loop) + write_connect = self.loop.connect_write_pipe(lambda: write_proto, + slave_write_obj) + write_transport, p = self.loop.run_until_complete(write_connect) + self.assertIs(p, write_proto) + self.assertIs(write_transport, write_proto.transport) + self.assertEqual('CONNECTED', write_proto.state) + + data = bytearray() + def reader(data): + chunk = os.read(master, 1024) + data += chunk + return len(data) + + write_transport.write(b'1') + test_utils.run_until(self.loop, lambda: reader(data) >= 1, timeout=10) + self.assertEqual(b'1', data) + self.assertEqual(['INITIAL', 'CONNECTED'], read_proto.state) + self.assertEqual('CONNECTED', write_proto.state) + + os.write(master, b'a') + test_utils.run_until(self.loop, lambda: read_proto.nbytes >= 1, + timeout=10) + self.assertEqual(['INITIAL', 'CONNECTED'], read_proto.state) + self.assertEqual(1, read_proto.nbytes) + self.assertEqual('CONNECTED', write_proto.state) + + write_transport.write(b'2345') + test_utils.run_until(self.loop, lambda: reader(data) >= 5, timeout=10) + self.assertEqual(b'12345', data) + self.assertEqual(['INITIAL', 'CONNECTED'], read_proto.state) + self.assertEqual('CONNECTED', write_proto.state) + + os.write(master, b'bcde') + test_utils.run_until(self.loop, lambda: read_proto.nbytes >= 5, + timeout=10) + self.assertEqual(['INITIAL', 'CONNECTED'], read_proto.state) + self.assertEqual(5, read_proto.nbytes) + self.assertEqual('CONNECTED', write_proto.state) + + os.close(master) + + read_transport.close() + self.loop.run_until_complete(read_proto.done) + self.assertEqual( + ['INITIAL', 'CONNECTED', 'EOF', 'CLOSED'], read_proto.state) + + write_transport.close() + self.loop.run_until_complete(write_proto.done) + self.assertEqual('CLOSED', write_proto.state) + def test_prompt_cancellation(self): r, w = test_utils.socketpair() r.setblocking(False) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 17:57:40 2016 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 31 Aug 2016 21:57:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3NzA2?= =?utf-8?q?=3A__Document_that_random=2Eseed=28=29_is_non-deterministic_whe?= =?utf-8?q?n?= Message-ID: <20160831215738.77205.26242.9BB3862F@psf.io> https://hg.python.org/cpython/rev/1f37903e6040 changeset: 102976:1f37903e6040 branch: 2.7 parent: 102961:f478f9b88319 user: Raymond Hettinger date: Wed Aug 31 14:57:32 2016 -0700 summary: Issue #27706: Document that random.seed() is non-deterministic when PYTHONHASHSEED is enabled files: Doc/library/random.rst | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Doc/library/random.rst b/Doc/library/random.rst --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -80,6 +80,9 @@ they are used instead of the system time (see the :func:`os.urandom` function for details on availability). + If a :term:`hashable` object is given, deterministic results are only assured + when :envvar:`PYTHONHASHSEED` is disabled. + .. versionchanged:: 2.4 formerly, operating system resources were not used. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 18:01:35 2016 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 31 Aug 2016 22:01:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NzA2?= =?utf-8?q?=3A_Fix_regression_in_random=2Eseed=28somestr=2C_version=3D1=29?= Message-ID: <20160831220134.52990.64865.9EC58995@psf.io> https://hg.python.org/cpython/rev/5ae941fef3be changeset: 102977:5ae941fef3be branch: 3.5 parent: 102974:dfaa38a8c7ec user: Raymond Hettinger date: Wed Aug 31 15:01:08 2016 -0700 summary: Issue #27706: Fix regression in random.seed(somestr, version=1) files: Lib/random.py | 7 +++++++ Lib/test/test_random.py | 18 ++++++++++++++++++ Misc/NEWS | 5 +++++ 3 files changed, 30 insertions(+), 0 deletions(-) diff --git a/Lib/random.py b/Lib/random.py --- a/Lib/random.py +++ b/Lib/random.py @@ -112,6 +112,13 @@ import time a = int(time.time() * 256) # use fractional seconds + if version == 1 and isinstance(a, (str, bytes)): + x = ord(a[0]) << 7 if a else 0 + for c in a: + x = ((1000003 * x) ^ ord(c)) & 0xFFFFFFFFFFFFFFFF + x ^= len(a) + a = -2 if x == -1 else x + if version == 2: if isinstance(a, (str, bytes, bytearray)): if isinstance(a, str): diff --git a/Lib/test/test_random.py b/Lib/test/test_random.py --- a/Lib/test/test_random.py +++ b/Lib/test/test_random.py @@ -326,6 +326,24 @@ ['0x1.1239ddfb11b7cp-3', '0x1.b3cbb5c51b120p-4', '0x1.8c4f55116b60fp-1', '0x1.63eb525174a27p-1']) + def test_bug_27706(self): + # Verify that version 1 seeds are unaffected by hash randomization + + self.gen.seed('nofar', version=1) # hash('nofar') == 5990528763808513177 + self.assertEqual([self.gen.random().hex() for i in range(4)], + ['0x1.8645314505ad7p-1', '0x1.afb1f82e40a40p-5', + '0x1.2a59d2285e971p-1', '0x1.56977142a7880p-6']) + + self.gen.seed('rachel', version=1) # hash('rachel') == -9091735575445484789 + self.assertEqual([self.gen.random().hex() for i in range(4)], + ['0x1.0b294cc856fcdp-1', '0x1.2ad22d79e77b8p-3', + '0x1.3052b9c072678p-2', '0x1.578f332106574p-3']) + + self.gen.seed('', version=1) # hash('') == 0 + self.assertEqual([self.gen.random().hex() for i in range(4)], + ['0x1.b0580f98a7dbep-1', '0x1.84129978f9c1ap-1', + '0x1.aeaa51052e978p-2', '0x1.092178fb945a6p-2']) + def test_setstate_first_arg(self): self.assertRaises(ValueError, self.gen.setstate, (1, None, None)) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -55,6 +55,11 @@ - Issue #19884: Avoid spurious output on OS X with Gnu Readline. +- Issue #27706: Restore deterministic behavior of random.Random().seed() + for string seeds using seeding version 1. Allows sequences of calls + to random() to exactly match those obtained in Python 2. + Patch by Nofar Schnider. + - Issue #10513: Fix a regression in Connection.commit(). Statements should not be reset after a commit. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 18:01:40 2016 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 31 Aug 2016 22:01:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <20160831220134.14223.37140.F7A59EF0@psf.io> https://hg.python.org/cpython/rev/f23fc0f48ebe changeset: 102978:f23fc0f48ebe parent: 102975:fcf65749ef71 parent: 102977:5ae941fef3be user: Raymond Hettinger date: Wed Aug 31 15:01:28 2016 -0700 summary: Merge files: Lib/random.py | 7 +++++++ Lib/test/test_random.py | 18 ++++++++++++++++++ Misc/NEWS | 5 +++++ 3 files changed, 30 insertions(+), 0 deletions(-) diff --git a/Lib/random.py b/Lib/random.py --- a/Lib/random.py +++ b/Lib/random.py @@ -112,6 +112,13 @@ import time a = int(time.time() * 256) # use fractional seconds + if version == 1 and isinstance(a, (str, bytes)): + x = ord(a[0]) << 7 if a else 0 + for c in a: + x = ((1000003 * x) ^ ord(c)) & 0xFFFFFFFFFFFFFFFF + x ^= len(a) + a = -2 if x == -1 else x + if version == 2: if isinstance(a, (str, bytes, bytearray)): if isinstance(a, str): diff --git a/Lib/test/test_random.py b/Lib/test/test_random.py --- a/Lib/test/test_random.py +++ b/Lib/test/test_random.py @@ -326,6 +326,24 @@ ['0x1.1239ddfb11b7cp-3', '0x1.b3cbb5c51b120p-4', '0x1.8c4f55116b60fp-1', '0x1.63eb525174a27p-1']) + def test_bug_27706(self): + # Verify that version 1 seeds are unaffected by hash randomization + + self.gen.seed('nofar', version=1) # hash('nofar') == 5990528763808513177 + self.assertEqual([self.gen.random().hex() for i in range(4)], + ['0x1.8645314505ad7p-1', '0x1.afb1f82e40a40p-5', + '0x1.2a59d2285e971p-1', '0x1.56977142a7880p-6']) + + self.gen.seed('rachel', version=1) # hash('rachel') == -9091735575445484789 + self.assertEqual([self.gen.random().hex() for i in range(4)], + ['0x1.0b294cc856fcdp-1', '0x1.2ad22d79e77b8p-3', + '0x1.3052b9c072678p-2', '0x1.578f332106574p-3']) + + self.gen.seed('', version=1) # hash('') == 0 + self.assertEqual([self.gen.random().hex() for i in range(4)], + ['0x1.b0580f98a7dbep-1', '0x1.84129978f9c1ap-1', + '0x1.aeaa51052e978p-2', '0x1.092178fb945a6p-2']) + def test_setstate_first_arg(self): self.assertRaises(ValueError, self.gen.setstate, (1, None, None)) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -72,6 +72,11 @@ - Issue #19884: Avoid spurious output on OS X with Gnu Readline. +- Issue #27706: Restore deterministic behavior of random.Random().seed() + for string seeds using seeding version 1. Allows sequences of calls + to random() to exactly match those obtained in Python 2. + Patch by Nofar Schnider. + - Issue #10513: Fix a regression in Connection.commit(). Statements should not be reset after a commit. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 19:37:43 2016 From: python-checkins at python.org (terry.reedy) Date: Wed, 31 Aug 2016 23:37:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327891=3A_Tweak_ne?= =?utf-8?q?w_idlelib_README_entry=2E?= Message-ID: <20160831233743.31432.57640.99890EF8@psf.io> https://hg.python.org/cpython/rev/28ce37a2d888 changeset: 102979:28ce37a2d888 user: Terry Jan Reedy date: Wed Aug 31 19:37:28 2016 -0400 summary: Issue #27891: Tweak new idlelib README entry. files: Lib/idlelib/README.txt | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/README.txt b/Lib/idlelib/README.txt --- a/Lib/idlelib/README.txt +++ b/Lib/idlelib/README.txt @@ -242,8 +242,8 @@ together by module, ignoring within module objects. Put 'import __main__' after other idlelib imports. -Imports only needed for testing are not at the top but are put in the -htest function def or the "if __name__ == '__main__'" clause. +Imports only needed for testing are put not at the top but in an +htest function def or "if __name__ == '__main__'" clause. Within module imports like "from idlelib.mod import class" may cause circular imports to deadlock. Even without this, circular imports may -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 19:45:52 2016 From: python-checkins at python.org (terry.reedy) Date: Wed, 31 Aug 2016 23:45:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Improve_idlelib=2Etextview?= =?utf-8?q?_comments=2E?= Message-ID: <20160831234552.14277.96463.884197F6@psf.io> https://hg.python.org/cpython/rev/f87239a2499b changeset: 102980:f87239a2499b user: Terry Jan Reedy date: Wed Aug 31 19:45:39 2016 -0400 summary: Improve idlelib.textview comments. files: Lib/idlelib/textview.py | 13 ++++++------- 1 files changed, 6 insertions(+), 7 deletions(-) diff --git a/Lib/idlelib/textview.py b/Lib/idlelib/textview.py --- a/Lib/idlelib/textview.py +++ b/Lib/idlelib/textview.py @@ -7,9 +7,8 @@ class TextViewer(Toplevel): - """A simple text viewer dialog for IDLE + "A simple text viewer dialog for IDLE." - """ def __init__(self, parent, title, text, modal=True, _htest=False): """Show the given text in a scrollable window with a 'close' button @@ -21,11 +20,11 @@ """ Toplevel.__init__(self, parent) self.configure(borderwidth=5) - # place dialog below parent if running htest + # Place dialog below parent if running htest. self.geometry("=%dx%d+%d+%d" % (750, 500, parent.winfo_rootx() + 10, parent.winfo_rooty() + (10 if not _htest else 100))) - #elguavas - config placeholders til config stuff completed + # TODO: get fg/bg from theme. self.bg = '#ffffff' self.fg = '#000000' @@ -34,9 +33,9 @@ self.protocol("WM_DELETE_WINDOW", self.Ok) self.parent = parent self.textView.focus_set() - #key bindings for this dialog - self.bind('',self.Ok) #dismiss dialog - self.bind('',self.Ok) #dismiss dialog + # Bind keys for closing this dialog. + self.bind('',self.Ok) + self.bind('',self.Ok) self.textView.insert(0.0, text) self.textView.config(state=DISABLED) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 21:09:17 2016 From: python-checkins at python.org (terry.reedy) Date: Thu, 01 Sep 2016 01:09:17 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3OTIy?= =?utf-8?q?=3A_IDLE_tests_no_longer_flash_tk_widgets=2E?= Message-ID: <20160901010917.14223.45704.8E932B91@psf.io> https://hg.python.org/cpython/rev/bf0cb86c6219 changeset: 102981:bf0cb86c6219 branch: 2.7 parent: 102976:1f37903e6040 user: Terry Jan Reedy date: Wed Aug 31 21:03:08 2016 -0400 summary: Issue #27922: IDLE tests no longer flash tk widgets. files: Lib/idlelib/idle_test/test_configdialog.py | 1 + Lib/idlelib/idle_test/test_editmenu.py | 1 + Lib/idlelib/idle_test/test_hyperparser.py | 1 + Lib/idlelib/idle_test/test_textview.py | 13 ++++++--- Lib/idlelib/idle_test/test_widgetredir.py | 2 + 5 files changed, 13 insertions(+), 5 deletions(-) diff --git a/Lib/idlelib/idle_test/test_configdialog.py b/Lib/idlelib/idle_test/test_configdialog.py --- a/Lib/idlelib/idle_test/test_configdialog.py +++ b/Lib/idlelib/idle_test/test_configdialog.py @@ -16,6 +16,7 @@ def setUpClass(cls): requires('gui') cls.root = Tk() + cls.root.withdraw() _initializeTkVariantTests(cls.root) @classmethod diff --git a/Lib/idlelib/idle_test/test_editmenu.py b/Lib/idlelib/idle_test/test_editmenu.py --- a/Lib/idlelib/idle_test/test_editmenu.py +++ b/Lib/idlelib/idle_test/test_editmenu.py @@ -18,6 +18,7 @@ def setUpClass(cls): requires('gui') cls.root = root = tk.Tk() + root.withdraw() PyShell.fix_x11_paste(root) cls.text = tk.Text(root) cls.entry = tk.Entry(root) diff --git a/Lib/idlelib/idle_test/test_hyperparser.py b/Lib/idlelib/idle_test/test_hyperparser.py --- a/Lib/idlelib/idle_test/test_hyperparser.py +++ b/Lib/idlelib/idle_test/test_hyperparser.py @@ -36,6 +36,7 @@ def setUpClass(cls): requires('gui') cls.root = Tk() + cls.root.withdraw() cls.text = Text(cls.root) cls.editwin = DummyEditwin(cls.text) diff --git a/Lib/idlelib/idle_test/test_textview.py b/Lib/idlelib/idle_test/test_textview.py --- a/Lib/idlelib/idle_test/test_textview.py +++ b/Lib/idlelib/idle_test/test_textview.py @@ -9,7 +9,7 @@ from idlelib.idle_test.mock_tk import Mbox -class TV(tv.TextViewer): # used by TextViewTest +class TV(tv.TextViewer): # Use in TextViewTest transient = Func() grab_set = Func() wait_window = Func() @@ -20,6 +20,7 @@ def setUpClass(cls): requires('gui') cls.root = Tk() + cls.root.withdraw() @classmethod def tearDownClass(cls): @@ -50,8 +51,8 @@ view.destroy = Func() view.Ok() self.assertTrue(view.destroy.called) - del view.destroy # unmask real function - view.destroy + del view.destroy # Unmask the real function. + view.destroy() class ViewFunctionTest(unittest.TestCase): @@ -60,6 +61,7 @@ def setUpClass(cls): requires('gui') cls.root = Tk() + cls.root.withdraw() cls.orig_mbox = tv.tkMessageBox tv.tkMessageBox = Mbox @@ -71,9 +73,10 @@ del cls.orig_mbox def test_view_text(self): - # If modal True, tkinter will error with 'can't invoke "event" command' + # If modal True, get tkinter error 'can't invoke "event" command'. view = tv.view_text(self.root, 'Title', 'test text', modal=False) self.assertIsInstance(view, tv.TextViewer) + view.Ok() def test_view_file(self): test_dir = os.path.dirname(__file__) @@ -83,7 +86,7 @@ self.assertIn('Test', view.textView.get('1.0', '1.end')) view.Ok() - # Mock messagebox will be used and view_file will not return anything + # Mock messagebox will be used; view_file will return None. testfile = os.path.join(test_dir, '../notthere.py') view = tv.view_file(self.root, 'Title', testfile, modal=False) self.assertIsNone(view) diff --git a/Lib/idlelib/idle_test/test_widgetredir.py b/Lib/idlelib/idle_test/test_widgetredir.py --- a/Lib/idlelib/idle_test/test_widgetredir.py +++ b/Lib/idlelib/idle_test/test_widgetredir.py @@ -15,6 +15,7 @@ def setUpClass(cls): requires('gui') cls.root = Tk() + cls.root.withdraw() cls.text = Text(cls.root) @classmethod @@ -44,6 +45,7 @@ def setUpClass(cls): requires('gui') cls.root = Tk() + cls.root.withdraw() cls.text = Text(cls.root) @classmethod -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 21:09:18 2016 From: python-checkins at python.org (terry.reedy) Date: Thu, 01 Sep 2016 01:09:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327922=3A_IDLE_tests_no_longer_flash_tk_widgets_?= =?utf-8?b?KE1lcmdlIDMuNSku?= Message-ID: <20160901010917.2636.93959.06901973@psf.io> https://hg.python.org/cpython/rev/fc711879c64a changeset: 102983:fc711879c64a parent: 102980:f87239a2499b parent: 102982:ff3a6303c5b1 user: Terry Jan Reedy date: Wed Aug 31 21:09:02 2016 -0400 summary: Issue #27922: IDLE tests no longer flash tk widgets (Merge 3.5). files: Lib/idlelib/idle_test/test_config_key.py | 1 + Lib/idlelib/idle_test/test_configdialog.py | 1 + Lib/idlelib/idle_test/test_editmenu.py | 1 + Lib/idlelib/idle_test/test_history.py | 1 + Lib/idlelib/idle_test/test_hyperparser.py | 1 + Lib/idlelib/idle_test/test_macosx.py | 2 + Lib/idlelib/idle_test/test_query.py | 4 ++ Lib/idlelib/idle_test/test_redirector.py | 2 + Lib/idlelib/idle_test/test_textview.py | 14 +++++---- 9 files changed, 21 insertions(+), 6 deletions(-) diff --git a/Lib/idlelib/idle_test/test_config_key.py b/Lib/idlelib/idle_test/test_config_key.py --- a/Lib/idlelib/idle_test/test_config_key.py +++ b/Lib/idlelib/idle_test/test_config_key.py @@ -14,6 +14,7 @@ @classmethod def setUpClass(cls): cls.root = Tk() + cls.root.withdraw() @classmethod def tearDownClass(cls): diff --git a/Lib/idlelib/idle_test/test_configdialog.py b/Lib/idlelib/idle_test/test_configdialog.py --- a/Lib/idlelib/idle_test/test_configdialog.py +++ b/Lib/idlelib/idle_test/test_configdialog.py @@ -14,6 +14,7 @@ @classmethod def setUpClass(cls): cls.root = Tk() + cls.root.withdraw() @classmethod def tearDownClass(cls): diff --git a/Lib/idlelib/idle_test/test_editmenu.py b/Lib/idlelib/idle_test/test_editmenu.py --- a/Lib/idlelib/idle_test/test_editmenu.py +++ b/Lib/idlelib/idle_test/test_editmenu.py @@ -17,6 +17,7 @@ @classmethod def setUpClass(cls): cls.root = root = tk.Tk() + cls.root.withdraw() pyshell.fix_x11_paste(root) cls.text = tk.Text(root) cls.entry = tk.Entry(root) diff --git a/Lib/idlelib/idle_test/test_history.py b/Lib/idlelib/idle_test/test_history.py --- a/Lib/idlelib/idle_test/test_history.py +++ b/Lib/idlelib/idle_test/test_history.py @@ -68,6 +68,7 @@ def setUpClass(cls): requires('gui') cls.root = tk.Tk() + cls.root.withdraw() def setUp(self): self.text = text = TextWrapper(self.root) diff --git a/Lib/idlelib/idle_test/test_hyperparser.py b/Lib/idlelib/idle_test/test_hyperparser.py --- a/Lib/idlelib/idle_test/test_hyperparser.py +++ b/Lib/idlelib/idle_test/test_hyperparser.py @@ -37,6 +37,7 @@ def setUpClass(cls): requires('gui') cls.root = Tk() + cls.root.withdraw() cls.text = Text(cls.root) cls.editwin = DummyEditwin(cls.text) diff --git a/Lib/idlelib/idle_test/test_macosx.py b/Lib/idlelib/idle_test/test_macosx.py --- a/Lib/idlelib/idle_test/test_macosx.py +++ b/Lib/idlelib/idle_test/test_macosx.py @@ -22,6 +22,7 @@ def setUpClass(cls): requires('gui') cls.root = tk.Tk() + cls.root.withdraw() cls.orig_platform = macosx.platform @classmethod @@ -76,6 +77,7 @@ def setUpClass(cls): requires('gui') cls.root = tk.Tk() + cls.root.withdraw() @classmethod def tearDownClass(cls): diff --git a/Lib/idlelib/idle_test/test_query.py b/Lib/idlelib/idle_test/test_query.py --- a/Lib/idlelib/idle_test/test_query.py +++ b/Lib/idlelib/idle_test/test_query.py @@ -252,6 +252,7 @@ def setUpClass(cls): requires('gui') cls.root = root = Tk() + cls.root.withdraw() cls.dialog = query.Query(root, 'TEST', 'test', _utest=True) cls.dialog.destroy = mock.Mock() @@ -295,6 +296,7 @@ def test_click_section_name(self): root = Tk() + root.withdraw() dialog = query.SectionName(root, 'T', 't', {'abc'}, _utest=True) Equal = self.assertEqual self.assertEqual(dialog.used_names, {'abc'}) @@ -314,6 +316,7 @@ def test_click_module_name(self): root = Tk() + root.withdraw() dialog = query.ModuleName(root, 'T', 't', 'idlelib', _utest=True) self.assertEqual(dialog.text0, 'idlelib') self.assertEqual(dialog.entry.get(), 'idlelib') @@ -332,6 +335,7 @@ def test_click_help_source(self): root = Tk() + root.withdraw() dialog = query.HelpSource(root, 'T', menuitem='__test__', filepath=__file__, _utest=True) Equal = self.assertEqual diff --git a/Lib/idlelib/idle_test/test_redirector.py b/Lib/idlelib/idle_test/test_redirector.py --- a/Lib/idlelib/idle_test/test_redirector.py +++ b/Lib/idlelib/idle_test/test_redirector.py @@ -15,6 +15,7 @@ def setUpClass(cls): requires('gui') cls.root = Tk() + cls.root.withdraw() cls.text = Text(cls.root) @classmethod @@ -44,6 +45,7 @@ def setUpClass(cls): requires('gui') cls.root = Tk() + cls.root.withdraw() cls.text = Text(cls.root) @classmethod diff --git a/Lib/idlelib/idle_test/test_textview.py b/Lib/idlelib/idle_test/test_textview.py --- a/Lib/idlelib/idle_test/test_textview.py +++ b/Lib/idlelib/idle_test/test_textview.py @@ -20,15 +20,16 @@ def setUpModule(): global root root = Tk() + root.withdraw() def tearDownModule(): global root root.update_idletasks() - root.destroy() # pyflakes falsely sees root as undefined + root.destroy() # Pyflakes falsely sees root as undefined. del root -class TV(tv.TextViewer): # used by TextViewTest +class TV(tv.TextViewer): # Used in TextViewTest. transient = Func() grab_set = Func() wait_window = Func() @@ -59,8 +60,8 @@ view.destroy = Func() view.Ok() self.assertTrue(view.destroy.called) - del view.destroy # unmask real function - view.destroy + del view.destroy # Unmask real function. + view.destroy() class ViewFunctionTest(unittest.TestCase): @@ -76,9 +77,10 @@ del cls.orig_error def test_view_text(self): - # If modal True, tkinter will error with 'can't invoke "event" command' + # If modal True, get tk error 'can't invoke "event" command'. view = tv.view_text(root, 'Title', 'test text', modal=False) self.assertIsInstance(view, tv.TextViewer) + view.Ok() def test_view_file(self): test_dir = os.path.dirname(__file__) @@ -88,7 +90,7 @@ self.assertIn('Test', view.textView.get('1.0', '1.end')) view.Ok() - # Mock showerror will be used and view_file will return None + # Mock showerror will be used; view_file will return None. testfile = os.path.join(test_dir, '../notthere.py') view = tv.view_file(root, 'Title', testfile, modal=False) self.assertIsNone(view) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Aug 31 21:09:18 2016 From: python-checkins at python.org (terry.reedy) Date: Thu, 01 Sep 2016 01:09:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3OTIy?= =?utf-8?q?=3A_IDLE_tests_no_longer_flash_tk_widgets=2E?= Message-ID: <20160901010917.98179.75959.1E582D99@psf.io> https://hg.python.org/cpython/rev/ff3a6303c5b1 changeset: 102982:ff3a6303c5b1 branch: 3.5 parent: 102977:5ae941fef3be user: Terry Jan Reedy date: Wed Aug 31 21:03:16 2016 -0400 summary: Issue #27922: IDLE tests no longer flash tk widgets. files: Lib/idlelib/idle_test/test_configdialog.py | 1 + Lib/idlelib/idle_test/test_editmenu.py | 1 + Lib/idlelib/idle_test/test_hyperparser.py | 1 + Lib/idlelib/idle_test/test_idlehistory.py | 1 + Lib/idlelib/idle_test/test_textview.py | 12 +++++---- Lib/idlelib/idle_test/test_widgetredir.py | 2 + 6 files changed, 13 insertions(+), 5 deletions(-) diff --git a/Lib/idlelib/idle_test/test_configdialog.py b/Lib/idlelib/idle_test/test_configdialog.py --- a/Lib/idlelib/idle_test/test_configdialog.py +++ b/Lib/idlelib/idle_test/test_configdialog.py @@ -15,6 +15,7 @@ @classmethod def setUpClass(cls): cls.root = Tk() + cls.root.withdraw() macosx._initializeTkVariantTests(cls.root) @classmethod diff --git a/Lib/idlelib/idle_test/test_editmenu.py b/Lib/idlelib/idle_test/test_editmenu.py --- a/Lib/idlelib/idle_test/test_editmenu.py +++ b/Lib/idlelib/idle_test/test_editmenu.py @@ -16,6 +16,7 @@ @classmethod def setUpClass(cls): cls.root = root = tk.Tk() + root.withdraw() PyShell.fix_x11_paste(root) cls.text = tk.Text(root) cls.entry = tk.Entry(root) diff --git a/Lib/idlelib/idle_test/test_hyperparser.py b/Lib/idlelib/idle_test/test_hyperparser.py --- a/Lib/idlelib/idle_test/test_hyperparser.py +++ b/Lib/idlelib/idle_test/test_hyperparser.py @@ -37,6 +37,7 @@ def setUpClass(cls): requires('gui') cls.root = Tk() + cls.root.withdraw() cls.text = Text(cls.root) cls.editwin = DummyEditwin(cls.text) diff --git a/Lib/idlelib/idle_test/test_idlehistory.py b/Lib/idlelib/idle_test/test_idlehistory.py --- a/Lib/idlelib/idle_test/test_idlehistory.py +++ b/Lib/idlelib/idle_test/test_idlehistory.py @@ -68,6 +68,7 @@ def setUpClass(cls): requires('gui') cls.root = tk.Tk() + cls.root.withdraw() def setUp(self): self.text = text = TextWrapper(self.root) diff --git a/Lib/idlelib/idle_test/test_textview.py b/Lib/idlelib/idle_test/test_textview.py --- a/Lib/idlelib/idle_test/test_textview.py +++ b/Lib/idlelib/idle_test/test_textview.py @@ -20,15 +20,16 @@ def setUpModule(): global root root = Tk() + root.withdraw() def tearDownModule(): global root root.update_idletasks() - root.destroy() # pyflakes falsely sees root as undefined + root.destroy() # Pyflakes falsely sees root as undefined. del root -class TV(tv.TextViewer): # used by TextViewTest +class TV(tv.TextViewer): # Used in TextViewTest. transient = Func() grab_set = Func() wait_window = Func() @@ -59,8 +60,8 @@ view.destroy = Func() view.Ok() self.assertTrue(view.destroy.called) - del view.destroy # unmask real function - view.destroy + del view.destroy # Unmask real function. + view.destroy() class textviewTest(unittest.TestCase): @@ -76,9 +77,10 @@ del cls.orig_mbox def test_view_text(self): - # If modal True, tkinter will error with 'can't invoke "event" command' + # If modal True, get tk error 'can't invoke "event" command'. view = tv.view_text(root, 'Title', 'test text', modal=False) self.assertIsInstance(view, tv.TextViewer) + view.Ok() def test_view_file(self): test_dir = os.path.dirname(__file__) diff --git a/Lib/idlelib/idle_test/test_widgetredir.py b/Lib/idlelib/idle_test/test_widgetredir.py --- a/Lib/idlelib/idle_test/test_widgetredir.py +++ b/Lib/idlelib/idle_test/test_widgetredir.py @@ -15,6 +15,7 @@ def setUpClass(cls): requires('gui') cls.root = Tk() + cls.root.withdraw() cls.text = Text(cls.root) @classmethod @@ -44,6 +45,7 @@ def setUpClass(cls): requires('gui') cls.root = Tk() + cls.root.withdraw() cls.text = Text(cls.root) @classmethod -- Repository URL: https://hg.python.org/cpython