From python-checkins at python.org  Tue Nov  1 00:46:01 2016
From: python-checkins at python.org (ned.deily)
Date: Tue, 01 Nov 2016 04:46:01 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogU3RhcnQgMy42LjBi?=
 =?utf-8?q?4?=
Message-ID: <20161101044601.32327.6132.6BE411DD@psf.io>

https://hg.python.org/cpython/rev/d3a42b934a0d
changeset:   104862:d3a42b934a0d
branch:      3.6
user:        Ned Deily <nad at python.org>
date:        Tue Nov 01 00:35:39 2016 -0400
summary:
  Start 3.6.0b4

files:
  Include/patchlevel.h |   4 ++--
  Misc/NEWS            |  12 ++++++++++++
  2 files changed, 14 insertions(+), 2 deletions(-)


diff --git a/Include/patchlevel.h b/Include/patchlevel.h
--- a/Include/patchlevel.h
+++ b/Include/patchlevel.h
@@ -20,10 +20,10 @@
 #define PY_MINOR_VERSION	6
 #define PY_MICRO_VERSION	0
 #define PY_RELEASE_LEVEL	PY_RELEASE_LEVEL_BETA
-#define PY_RELEASE_SERIAL	3
+#define PY_RELEASE_SERIAL	4
 
 /* Version as a string */
-#define PY_VERSION      	"3.6.0b3"
+#define PY_VERSION      	"3.6.0b4+"
 /*--end constants--*/
 
 /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2.
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -2,6 +2,18 @@
 Python News
 +++++++++++
 
+What's New in Python 3.6.0 beta 4
+=================================
+
+*Release date: XXXX-XX-XX*
+
+Core and Builtins
+-----------------
+
+Library
+-------
+
+
 What's New in Python 3.6.0 beta 3
 =================================
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  1 00:46:01 2016
From: python-checkins at python.org (ned.deily)
Date: Tue, 01 Nov 2016 04:46:01 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Update_pydoc_t?=
 =?utf-8?q?opics_for_3=2E6=2E0b3?=
Message-ID: <20161101044601.32001.36525.36B2431F@psf.io>

https://hg.python.org/cpython/rev/6f0b41a7a621
changeset:   104859:6f0b41a7a621
branch:      3.6
parent:      104857:c7e551f8c5d8
user:        Ned Deily <nad at python.org>
date:        Mon Oct 31 20:39:38 2016 -0400
summary:
  Update pydoc topics for 3.6.0b3

files:
  Lib/pydoc_data/topics.py |  14 +++++++-------
  1 files changed, 7 insertions(+), 7 deletions(-)


diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py
--- a/Lib/pydoc_data/topics.py
+++ b/Lib/pydoc_data/topics.py
@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-# Autogenerated by Sphinx on Mon Oct 10 15:59:17 2016
+# Autogenerated by Sphinx on Mon Oct 31 20:37:53 2016
 topics = {'assert': '\n'
            'The "assert" statement\n'
            '**********************\n'
@@ -5066,9 +5066,9 @@
                   'be formatted\n'
                   'with the floating point presentation types listed below '
                   '(except "\'n\'"\n'
-                  'and None). When doing so, "float()" is used to convert the '
-                  'integer to\n'
-                  'a floating point number before formatting.\n'
+                  'and "None"). When doing so, "float()" is used to convert '
+                  'the integer\n'
+                  'to a floating point number before formatting.\n'
                   '\n'
                   'The available presentation types for floating point and '
                   'decimal values\n'
@@ -9626,7 +9626,7 @@
                    '   Unicode ordinals (integers) or characters (strings of '
                    'length 1) to\n'
                    '   Unicode ordinals, strings (of arbitrary lengths) or '
-                   'None.\n'
+                   '"None".\n'
                    '   Character keys will then be converted to ordinals.\n'
                    '\n'
                    '   If there are two arguments, they must be strings of '
@@ -9637,7 +9637,7 @@
                    'is a third\n'
                    '   argument, it must be a string, whose characters will be '
                    'mapped to\n'
-                   '   None in the result.\n'
+                   '   "None" in the result.\n'
                    '\n'
                    'str.partition(sep)\n'
                    '\n'
@@ -11204,7 +11204,7 @@
           'the\n'
           '   order of their occurrence in the base class list; "__doc__" is '
           'the\n'
-          "   class's documentation string, or None if undefined;\n"
+          '   class\'s documentation string, or "None" if undefined;\n'
           '   "__annotations__" (optional) is a dictionary containing '
           '*variable\n'
           '   annotations* collected during class body execution.\n'

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  1 00:46:01 2016
From: python-checkins at python.org (ned.deily)
Date: Tue, 01 Nov 2016 04:46:01 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Version_bump_f?=
 =?utf-8?q?or_3=2E6=2E0b3?=
Message-ID: <20161101044601.34695.58432.E596F1C6@psf.io>

https://hg.python.org/cpython/rev/8345e066c0ed
changeset:   104860:8345e066c0ed
branch:      3.6
tag:         v3.6.0b3
user:        Ned Deily <nad at python.org>
date:        Mon Oct 31 20:43:30 2016 -0400
summary:
  Version bump for 3.6.0b3

files:
  Include/patchlevel.h |  4 ++--
  Misc/NEWS            |  2 +-
  README               |  2 +-
  3 files changed, 4 insertions(+), 4 deletions(-)


diff --git a/Include/patchlevel.h b/Include/patchlevel.h
--- a/Include/patchlevel.h
+++ b/Include/patchlevel.h
@@ -20,10 +20,10 @@
 #define PY_MINOR_VERSION	6
 #define PY_MICRO_VERSION	0
 #define PY_RELEASE_LEVEL	PY_RELEASE_LEVEL_BETA
-#define PY_RELEASE_SERIAL	2
+#define PY_RELEASE_SERIAL	3
 
 /* Version as a string */
-#define PY_VERSION      	"3.6.0b2+"
+#define PY_VERSION      	"3.6.0b3"
 /*--end constants--*/
 
 /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2.
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -5,7 +5,7 @@
 What's New in Python 3.6.0 beta 3
 =================================
 
-*Release date: XXXX-XX-XX*
+*Release date: 2016-10-31*
 
 Core and Builtins
 -----------------
diff --git a/README b/README
--- a/README
+++ b/README
@@ -1,4 +1,4 @@
-This is Python version 3.6.0 beta 2
+This is Python version 3.6.0 beta 3
 ===================================
 
 Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011,

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  1 00:46:01 2016
From: python-checkins at python.org (ned.deily)
Date: Tue, 01 Nov 2016 04:46:01 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Added_tag_v3?=
 =?utf-8?q?=2E6=2E0b3_for_changeset_8345e066c0ed?=
Message-ID: <20161101044601.7609.13691.BD33D6A9@psf.io>

https://hg.python.org/cpython/rev/710426990fb0
changeset:   104861:710426990fb0
branch:      3.6
user:        Ned Deily <nad at python.org>
date:        Mon Oct 31 20:45:16 2016 -0400
summary:
  Added tag v3.6.0b3 for changeset 8345e066c0ed

files:
  .hgtags |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -172,3 +172,4 @@
 017cf260936b444788c9b671d195b7bfd83dbd25 v3.6.0a4
 5b0ca4ed5e2f0669d76ece7ef975c544580f12b4 v3.6.0b1
 b9fadc7d1c3f9c3c77f32f35afbe1a1cc38070e6 v3.6.0b2
+8345e066c0ed713c3e510cbc8fafc1c38d6d306b v3.6.0b3

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  1 00:46:01 2016
From: python-checkins at python.org (ned.deily)
Date: Tue, 01 Nov 2016 04:46:01 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_tag_from_3=2E6?=
Message-ID: <20161101044601.32315.29273.E09B89B5@psf.io>

https://hg.python.org/cpython/rev/42e44501dc1e
changeset:   104863:42e44501dc1e
parent:      104858:9e66ffa7a791
parent:      104862:d3a42b934a0d
user:        Ned Deily <nad at python.org>
date:        Tue Nov 01 00:44:09 2016 -0400
summary:
  Merge tag from 3.6

files:
  .hgtags |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -172,3 +172,4 @@
 017cf260936b444788c9b671d195b7bfd83dbd25 v3.6.0a4
 5b0ca4ed5e2f0669d76ece7ef975c544580f12b4 v3.6.0b1
 b9fadc7d1c3f9c3c77f32f35afbe1a1cc38070e6 v3.6.0b2
+8345e066c0ed713c3e510cbc8fafc1c38d6d306b v3.6.0b3

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  1 01:54:20 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 01 Nov 2016 05:54:20 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161101055419.114637.59649.CD4EDE93@psf.io>

https://hg.python.org/cpython/rev/a6e59a2e880e
changeset:   104865:a6e59a2e880e
parent:      104863:42e44501dc1e
parent:      104864:fc6f90d8022b
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Oct 31 22:54:12 2016 -0700
summary:
  merge

files:



-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  1 01:54:19 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 01 Nov 2016 05:54:19 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogQWRkIGN1bV93ZWln?=
 =?utf-8?q?hts_example_=28simulation_of_a_cumulative_binomial_distribution?=
 =?utf-8?b?KS4=?=
Message-ID: <20161101055419.3290.24495.A39C7360@psf.io>

https://hg.python.org/cpython/rev/fc6f90d8022b
changeset:   104864:fc6f90d8022b
branch:      3.6
parent:      104862:d3a42b934a0d
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Oct 31 22:53:52 2016 -0700
summary:
  Add cum_weights example (simulation of a cumulative binomial distribution).

files:
  Doc/library/random.rst |  7 +++++++
  1 files changed, 7 insertions(+), 0 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -351,6 +351,13 @@
    >>> choices(['red', 'black', 'green'], [18, 18, 2], k=6)
    ['red', 'green', 'black', 'black', 'red', 'black']
 
+   # Probability of getting 5 or more heads from 7 spins of a biased coin
+   # that settles on heads 60% of the time.
+   >>> n = 10000
+   >>> cw = [0.60, 1.00]
+   >>> sum(choices('HT', cum_weights=cw, k=7).count('H') >= 5 for i in range(n)) / n
+   0.4169
+
 Example of `statistical bootstrapping
 <https://en.wikipedia.org/wiki/Bootstrapping_(statistics)>`_ using resampling
 with replacement to estimate a confidence interval for the mean of a small

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  1 02:44:47 2016
From: python-checkins at python.org (zach.ware)
Date: Tue, 01 Nov 2016 06:44:47 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogQ2xvc2VzICMyODU2?=
 =?utf-8?q?8=3A_Fix_VS9=2E0_build_files_to_use_sqlite_3=2E8=2E11=2E0?=
Message-ID: <20161101064446.114895.19900.D92DCF1E@psf.io>

https://hg.python.org/cpython/rev/8f9c54a75c3d
changeset:   104866:8f9c54a75c3d
branch:      2.7
parent:      104855:33ad26897e30
user:        Zachary Ware <zachary.ware at gmail.com>
date:        Tue Nov 01 01:44:04 2016 -0500
summary:
  Closes #28568: Fix VS9.0 build files to use sqlite 3.8.11.0

files:
  PC/VS9.0/pyproject.vsprops |  2 +-
  PC/VS9.0/readme.txt        |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/PC/VS9.0/pyproject.vsprops b/PC/VS9.0/pyproject.vsprops
--- a/PC/VS9.0/pyproject.vsprops
+++ b/PC/VS9.0/pyproject.vsprops
@@ -74,7 +74,7 @@
 	/>
 	<UserMacro
 		Name="sqlite3Dir"
-		Value="$(externalsDir)\sqlite-3.6.21"
+		Value="$(externalsDir)\sqlite-3.8.11.0"
 	/>
 	<UserMacro
 		Name="bz2Dir"
diff --git a/PC/VS9.0/readme.txt b/PC/VS9.0/readme.txt
--- a/PC/VS9.0/readme.txt
+++ b/PC/VS9.0/readme.txt
@@ -107,7 +107,7 @@
     Wraps Berkeley DB 4.7.25, which is currently built by _bsddb.vcproj.
     project.
 _sqlite3
-    Wraps SQLite 3.6.21, which is currently built by sqlite3.vcproj.
+    Wraps SQLite 3.8.11.0, which is currently built by sqlite3.vcproj.
 _tkinter
     Wraps the Tk windowing system.  Unlike _bsddb and _sqlite3, there's no
     corresponding tcltk.vcproj-type project that builds Tcl/Tk from vcproj's

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Tue Nov  1 12:03:25 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Tue, 1 Nov 2016 16:03:25 +0000
Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7
 2016-11-01
Message-ID: <c04b9f23-a54f-4d9d-a8b9-2c950658bb76@irsmsx151.ger.corp.intel.com>

Results for project Python 2.7, build date 2016-11-01 03:48:22 +0000
commit:		33ad26897e30
previous commit:	e2dd0f48e643
revision date:	2016-10-31 23:48:38 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.21%         -0.05%          4.26%             4.00%
:-)             pybench      0.12%          0.01%          5.75%             3.47%
:-(            regex_v8      0.66%         -0.26%         -2.26%            10.95%
:-)               nbody      0.07%         -0.00%          7.96%             4.89%
:-)        json_dump_v2      0.30%         -0.11%          2.58%             9.69%
:-|      normal_startup      0.52%         -0.16%         -0.46%             2.49%
:-)             ssbench      0.32%          0.03%          2.18%             2.05%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-11-01/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Tue Nov  1 12:05:38 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Tue, 1 Nov 2016 16:05:38 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python Default
 2016-11-01
Message-ID: <7014e780-70ab-4ffc-9929-3bf0d90ddb94@irsmsx151.ger.corp.intel.com>

Results for project Python default, build date 2016-11-01 03:01:46 +0000
commit:		9e66ffa7a791
previous commit:	e9169a8c0692
revision date:	2016-10-31 23:58:25 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.21%          0.41%          4.50%            16.30%
:-)             pybench      0.10%         -0.57%          5.06%             5.96%
:-(            regex_v8      3.78%         -0.52%         -3.75%             3.86%
:-)               nbody      0.09%          1.15%          3.56%             4.65%
:-(        json_dump_v2      0.33%          1.26%        -11.36%            15.87%
:-|      normal_startup      0.51%         -0.17%          0.29%             6.95%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-11-01/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Wed Nov  2 01:23:40 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Wed, 02 Nov 2016 05:23:40 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161102052340.62801.42193.C7E7488B@psf.io>

https://hg.python.org/cpython/rev/b671422240cd
changeset:   104868:b671422240cd
parent:      104865:a6e59a2e880e
parent:      104867:3786e5830c73
user:        Raymond Hettinger <python at rcn.com>
date:        Tue Nov 01 22:23:34 2016 -0700
summary:
  merge

files:
  Lib/random.py |  3 +--
  1 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/Lib/random.py b/Lib/random.py
--- a/Lib/random.py
+++ b/Lib/random.py
@@ -350,8 +350,7 @@
                 _int = int
                 total = len(population)
                 return [population[_int(random() * total)] for i in range(k)]
-            else:
-                cum_weights = list(_itertools.accumulate(weights))
+            cum_weights = list(_itertools.accumulate(weights))
         elif weights is not None:
             raise TypeError('Cannot specify both weights and cumulative_weights')
         if len(cum_weights) != len(population):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 01:23:40 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Wed, 02 Nov 2016 05:23:40 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Minor_code_bea?=
 =?utf-8?q?utification?=
Message-ID: <20161102052340.62927.81526.0C63EE1E@psf.io>

https://hg.python.org/cpython/rev/3786e5830c73
changeset:   104867:3786e5830c73
branch:      3.6
parent:      104864:fc6f90d8022b
user:        Raymond Hettinger <python at rcn.com>
date:        Tue Nov 01 22:23:11 2016 -0700
summary:
  Minor code beautification

files:
  Lib/random.py |  3 +--
  1 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/Lib/random.py b/Lib/random.py
--- a/Lib/random.py
+++ b/Lib/random.py
@@ -350,8 +350,7 @@
                 _int = int
                 total = len(population)
                 return [population[_int(random() * total)] for i in range(k)]
-            else:
-                cum_weights = list(_itertools.accumulate(weights))
+            cum_weights = list(_itertools.accumulate(weights))
         elif weights is not None:
             raise TypeError('Cannot specify both weights and cumulative_weights')
         if len(cum_weights) != len(population):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 05:47:45 2016
From: python-checkins at python.org (inada.naoki)
Date: Wed, 02 Nov 2016 09:47:45 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328583=3A_PyDict=5FSetDefault_didn=27t_combine_s?=
 =?utf-8?q?plit_table_when_needed=2E?=
Message-ID: <20161102094731.3308.51335.8B8A60DD@psf.io>

https://hg.python.org/cpython/rev/a6a79053aec4
changeset:   104870:a6a79053aec4
parent:      104868:b671422240cd
parent:      104869:4e9c7704f373
user:        INADA Naoki <songofacandy at gmail.com>
date:        Wed Nov 02 18:47:24 2016 +0900
summary:
  Issue #28583: PyDict_SetDefault didn't combine split table when needed.

Patch by Xiang Zhang.

files:
  Lib/test/test_dict.py |  17 ++++++++
  Misc/NEWS             |   3 +
  Objects/dictobject.c  |  60 +++++++++++++++++++++++-------
  3 files changed, 65 insertions(+), 15 deletions(-)


diff --git a/Lib/test/test_dict.py b/Lib/test/test_dict.py
--- a/Lib/test/test_dict.py
+++ b/Lib/test/test_dict.py
@@ -852,6 +852,23 @@
         return dicts
 
     @support.cpython_only
+    def test_splittable_setdefault(self):
+        """split table must be combined when setdefault()
+        breaks insertion order"""
+        a, b = self.make_shared_key_dict(2)
+
+        a['a'] = 1
+        size_a = sys.getsizeof(a)
+        a['b'] = 2
+        b.setdefault('b', 2)
+        size_b = sys.getsizeof(b)
+        b['a'] = 1
+
+        self.assertGreater(size_b, size_a)
+        self.assertEqual(list(a), ['x', 'y', 'z', 'a', 'b'])
+        self.assertEqual(list(b), ['x', 'y', 'z', 'b', 'a'])
+
+    @support.cpython_only
     def test_splittable_del(self):
         """split table must be combined when del d[k]"""
         a, b = self.make_shared_key_dict(2)
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #28583: PyDict_SetDefault didn't combine split table when needed.
+  Patch by Xiang Zhang.
+
 - Issue #28128: Deprecation warning for invalid str and byte escape
   sequences now prints better information about where the error
   occurs. Patch by Serhiy Storchaka and Eric Smith.
diff --git a/Objects/dictobject.c b/Objects/dictobject.c
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -2758,58 +2758,88 @@
 PyDict_SetDefault(PyObject *d, PyObject *key, PyObject *defaultobj)
 {
     PyDictObject *mp = (PyDictObject *)d;
-    PyObject *val = NULL;
+    PyObject *value;
     Py_hash_t hash;
     Py_ssize_t hashpos, ix;
-    PyDictKeyEntry *ep;
     PyObject **value_addr;
 
     if (!PyDict_Check(d)) {
         PyErr_BadInternalCall();
         return NULL;
     }
+
     if (!PyUnicode_CheckExact(key) ||
         (hash = ((PyASCIIObject *) key)->hash) == -1) {
         hash = PyObject_Hash(key);
         if (hash == -1)
             return NULL;
     }
+
+    if (mp->ma_values != NULL && !PyUnicode_CheckExact(key)) {
+        if (insertion_resize(mp) < 0)
+            return NULL;
+    }
+
     ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, &hashpos);
     if (ix == DKIX_ERROR)
         return NULL;
-    if (ix == DKIX_EMPTY || *value_addr == NULL) {
-        val = defaultobj;
+
+    if (_PyDict_HasSplitTable(mp) &&
+        ((ix >= 0 && *value_addr == NULL && mp->ma_used != ix) ||
+         (ix == DKIX_EMPTY && mp->ma_used != mp->ma_keys->dk_nentries))) {
+        if (insertion_resize(mp) < 0) {
+            return NULL;
+        }
+        find_empty_slot(mp, key, hash, &value_addr, &hashpos);
+        ix = DKIX_EMPTY;
+    }
+
+    if (ix == DKIX_EMPTY) {
+        PyDictKeyEntry *ep, *ep0;
+        value = defaultobj;
         if (mp->ma_keys->dk_usable <= 0) {
-            /* Need to resize. */
             if (insertion_resize(mp) < 0) {
                 return NULL;
             }
             find_empty_slot(mp, key, hash, &value_addr, &hashpos);
         }
-        ix = mp->ma_keys->dk_nentries;
-        Py_INCREF(defaultobj);
+        ep0 = DK_ENTRIES(mp->ma_keys);
+        ep = &ep0[mp->ma_keys->dk_nentries];
+        dk_set_index(mp->ma_keys, hashpos, mp->ma_keys->dk_nentries);
         Py_INCREF(key);
-        MAINTAIN_TRACKING(mp, key, defaultobj);
-        dk_set_index(mp->ma_keys, hashpos, ix);
-        ep = &DK_ENTRIES(mp->ma_keys)[ix];
+        Py_INCREF(value);
+        MAINTAIN_TRACKING(mp, key, value);
         ep->me_key = key;
         ep->me_hash = hash;
         if (mp->ma_values) {
-            mp->ma_values[ix] = val;
+            assert(mp->ma_values[mp->ma_keys->dk_nentries] == NULL);
+            mp->ma_values[mp->ma_keys->dk_nentries] = value;
         }
         else {
-            ep->me_value = val;
+            ep->me_value = value;
         }
+        mp->ma_used++;
+        mp->ma_version_tag = DICT_NEXT_VERSION();
         mp->ma_keys->dk_usable--;
         mp->ma_keys->dk_nentries++;
+        assert(mp->ma_keys->dk_usable >= 0);
+    }
+    else if (*value_addr == NULL) {
+        value = defaultobj;
+        assert(_PyDict_HasSplitTable(mp));
+        assert(ix == mp->ma_used);
+        Py_INCREF(value);
+        MAINTAIN_TRACKING(mp, key, value);
+        *value_addr = value;
         mp->ma_used++;
         mp->ma_version_tag = DICT_NEXT_VERSION();
-        assert(_PyDict_CheckConsistency(mp));
     }
     else {
-        val = *value_addr;
+        value = *value_addr;
     }
-    return val;
+
+    assert(_PyDict_CheckConsistency(mp));
+    return value;
 }
 
 static PyObject *

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 05:47:45 2016
From: python-checkins at python.org (inada.naoki)
Date: Wed, 02 Nov 2016 09:47:45 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NTgz?=
 =?utf-8?q?=3A_PyDict=5FSetDefault_didn=27t_combine_split_table_when_neede?=
 =?utf-8?q?d=2E?=
Message-ID: <20161102094731.30731.85572.33500FAD@psf.io>

https://hg.python.org/cpython/rev/4e9c7704f373
changeset:   104869:4e9c7704f373
branch:      3.6
parent:      104867:3786e5830c73
user:        INADA Naoki <songofacandy at gmail.com>
date:        Wed Nov 02 18:45:16 2016 +0900
summary:
  Issue #28583: PyDict_SetDefault didn't combine split table when needed.

Patch by Xiang Zhang.

files:
  Lib/test/test_dict.py |  17 ++++++++
  Misc/NEWS             |   3 +
  Objects/dictobject.c  |  60 +++++++++++++++++++++++-------
  3 files changed, 65 insertions(+), 15 deletions(-)


diff --git a/Lib/test/test_dict.py b/Lib/test/test_dict.py
--- a/Lib/test/test_dict.py
+++ b/Lib/test/test_dict.py
@@ -852,6 +852,23 @@
         return dicts
 
     @support.cpython_only
+    def test_splittable_setdefault(self):
+        """split table must be combined when setdefault()
+        breaks insertion order"""
+        a, b = self.make_shared_key_dict(2)
+
+        a['a'] = 1
+        size_a = sys.getsizeof(a)
+        a['b'] = 2
+        b.setdefault('b', 2)
+        size_b = sys.getsizeof(b)
+        b['a'] = 1
+
+        self.assertGreater(size_b, size_a)
+        self.assertEqual(list(a), ['x', 'y', 'z', 'a', 'b'])
+        self.assertEqual(list(b), ['x', 'y', 'z', 'b', 'a'])
+
+    @support.cpython_only
     def test_splittable_del(self):
         """split table must be combined when del d[k]"""
         a, b = self.make_shared_key_dict(2)
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #28583: PyDict_SetDefault didn't combine split table when needed.
+  Patch by Xiang Zhang.
+
 Library
 -------
 
diff --git a/Objects/dictobject.c b/Objects/dictobject.c
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -2757,58 +2757,88 @@
 PyDict_SetDefault(PyObject *d, PyObject *key, PyObject *defaultobj)
 {
     PyDictObject *mp = (PyDictObject *)d;
-    PyObject *val = NULL;
+    PyObject *value;
     Py_hash_t hash;
     Py_ssize_t hashpos, ix;
-    PyDictKeyEntry *ep;
     PyObject **value_addr;
 
     if (!PyDict_Check(d)) {
         PyErr_BadInternalCall();
         return NULL;
     }
+
     if (!PyUnicode_CheckExact(key) ||
         (hash = ((PyASCIIObject *) key)->hash) == -1) {
         hash = PyObject_Hash(key);
         if (hash == -1)
             return NULL;
     }
+
+    if (mp->ma_values != NULL && !PyUnicode_CheckExact(key)) {
+        if (insertion_resize(mp) < 0)
+            return NULL;
+    }
+
     ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, &hashpos);
     if (ix == DKIX_ERROR)
         return NULL;
-    if (ix == DKIX_EMPTY || *value_addr == NULL) {
-        val = defaultobj;
+
+    if (_PyDict_HasSplitTable(mp) &&
+        ((ix >= 0 && *value_addr == NULL && mp->ma_used != ix) ||
+         (ix == DKIX_EMPTY && mp->ma_used != mp->ma_keys->dk_nentries))) {
+        if (insertion_resize(mp) < 0) {
+            return NULL;
+        }
+        find_empty_slot(mp, key, hash, &value_addr, &hashpos);
+        ix = DKIX_EMPTY;
+    }
+
+    if (ix == DKIX_EMPTY) {
+        PyDictKeyEntry *ep, *ep0;
+        value = defaultobj;
         if (mp->ma_keys->dk_usable <= 0) {
-            /* Need to resize. */
             if (insertion_resize(mp) < 0) {
                 return NULL;
             }
             find_empty_slot(mp, key, hash, &value_addr, &hashpos);
         }
-        ix = mp->ma_keys->dk_nentries;
-        Py_INCREF(defaultobj);
+        ep0 = DK_ENTRIES(mp->ma_keys);
+        ep = &ep0[mp->ma_keys->dk_nentries];
+        dk_set_index(mp->ma_keys, hashpos, mp->ma_keys->dk_nentries);
         Py_INCREF(key);
-        MAINTAIN_TRACKING(mp, key, defaultobj);
-        dk_set_index(mp->ma_keys, hashpos, ix);
-        ep = &DK_ENTRIES(mp->ma_keys)[ix];
+        Py_INCREF(value);
+        MAINTAIN_TRACKING(mp, key, value);
         ep->me_key = key;
         ep->me_hash = hash;
         if (mp->ma_values) {
-            mp->ma_values[ix] = val;
+            assert(mp->ma_values[mp->ma_keys->dk_nentries] == NULL);
+            mp->ma_values[mp->ma_keys->dk_nentries] = value;
         }
         else {
-            ep->me_value = val;
+            ep->me_value = value;
         }
+        mp->ma_used++;
+        mp->ma_version_tag = DICT_NEXT_VERSION();
         mp->ma_keys->dk_usable--;
         mp->ma_keys->dk_nentries++;
+        assert(mp->ma_keys->dk_usable >= 0);
+    }
+    else if (*value_addr == NULL) {
+        value = defaultobj;
+        assert(_PyDict_HasSplitTable(mp));
+        assert(ix == mp->ma_used);
+        Py_INCREF(value);
+        MAINTAIN_TRACKING(mp, key, value);
+        *value_addr = value;
         mp->ma_used++;
         mp->ma_version_tag = DICT_NEXT_VERSION();
-        assert(_PyDict_CheckConsistency(mp));
     }
     else {
-        val = *value_addr;
+        value = *value_addr;
     }
-    return val;
+
+    assert(_PyDict_CheckConsistency(mp));
+    return value;
 }
 
 static PyObject *

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 06:14:15 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 02 Nov 2016 10:14:15 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4NTEz?=
 =?utf-8?q?=3A_Documented_command-line_interface_of_zipfile=2E?=
Message-ID: <20161102101407.31966.9605.30EBD427@psf.io>

https://hg.python.org/cpython/rev/feb6e8678512
changeset:   104871:feb6e8678512
branch:      2.7
parent:      104866:8f9c54a75c3d
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 02 12:05:54 2016 +0200
summary:
  Issue #28513: Documented command-line interface of zipfile.

files:
  Doc/library/zipfile.rst |  57 +++++++++++++++++++++++++++++
  Misc/NEWS               |   2 +
  2 files changed, 59 insertions(+), 0 deletions(-)


diff --git a/Doc/library/zipfile.rst b/Doc/library/zipfile.rst
--- a/Doc/library/zipfile.rst
+++ b/Doc/library/zipfile.rst
@@ -497,4 +497,61 @@
 
    Size of the uncompressed file.
 
+
+.. _zipfile-commandline:
+.. program:: zipfile
+
+Command-Line Interface
+----------------------
+
+The :mod:`zipfile` module provides a simple command-line interface to interact
+with ZIP archives.
+
+If you want to create a new ZIP archive, specify its name after the :option:`-c`
+option and then list the filename(s) that should be included:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -c monty.zip spam.txt eggs.txt
+
+Passing a directory is also acceptable:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -c monty.zip life-of-brian_1979/
+
+If you want to extract a ZIP archive into the specified directory, use
+the :option:`-e` option:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -e monty.zip target-dir/
+
+For a list of the files in a ZIP archive, use the :option:`-l` option:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -l monty.zip
+
+
+Command-line options
+~~~~~~~~~~~~~~~~~~~~
+
+.. cmdoption:: -l <zipfile>
+
+   List files in a zipfile.
+
+.. cmdoption:: -c <zipfile> <source1> ... <sourceN>
+
+   Create zipfile from source files.
+
+.. cmdoption:: -e <zipfile> <output_dir>
+
+   Extract zipfile into target directory.
+
+.. cmdoption:: -t <zipfile>
+
+   Test whether the zipfile is valid or not.
+
+
 .. _PKZIP Application Note: https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -236,6 +236,8 @@
 Documentation
 -------------
 
+- Issue #28513: Documented command-line interface of zipfile.
+
 - Issue #16484: Change the default PYTHONDOCS URL to "https:", and fix the
   resulting links to use lowercase.  Patch by Sean Rodman, test by Kaushik
   Nadikuditi.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 06:14:15 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 02 Nov 2016 10:14:15 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NTEz?=
 =?utf-8?q?=3A_Documented_command-line_interface_of_zipfile=2E?=
Message-ID: <20161102101407.7195.75919.C2E4B86F@psf.io>

https://hg.python.org/cpython/rev/b51bf32defb1
changeset:   104872:b51bf32defb1
branch:      3.5
parent:      104856:a8799a63feb7
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 02 12:06:15 2016 +0200
summary:
  Issue #28513: Documented command-line interface of zipfile.

files:
  Doc/library/zipfile.rst |  57 +++++++++++++++++++++++++++++
  Misc/NEWS               |   5 ++
  2 files changed, 62 insertions(+), 0 deletions(-)


diff --git a/Doc/library/zipfile.rst b/Doc/library/zipfile.rst
--- a/Doc/library/zipfile.rst
+++ b/Doc/library/zipfile.rst
@@ -574,4 +574,61 @@
 
    Size of the uncompressed file.
 
+
+.. _zipfile-commandline:
+.. program:: zipfile
+
+Command-Line Interface
+----------------------
+
+The :mod:`zipfile` module provides a simple command-line interface to interact
+with ZIP archives.
+
+If you want to create a new ZIP archive, specify its name after the :option:`-c`
+option and then list the filename(s) that should be included:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -c monty.zip spam.txt eggs.txt
+
+Passing a directory is also acceptable:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -c monty.zip life-of-brian_1979/
+
+If you want to extract a ZIP archive into the specified directory, use
+the :option:`-e` option:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -e monty.zip target-dir/
+
+For a list of the files in a ZIP archive, use the :option:`-l` option:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -l monty.zip
+
+
+Command-line options
+~~~~~~~~~~~~~~~~~~~~
+
+.. cmdoption:: -l <zipfile>
+
+   List files in a zipfile.
+
+.. cmdoption:: -c <zipfile> <source1> ... <sourceN>
+
+   Create zipfile from source files.
+
+.. cmdoption:: -e <zipfile> <output_dir>
+
+   Extract zipfile into target directory.
+
+.. cmdoption:: -t <zipfile>
+
+   Test whether the zipfile is valid or not.
+
+
 .. _PKZIP Application Note: https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -471,6 +471,11 @@
 - Issue #26754: PyUnicode_FSDecoder() accepted a filename argument encoded as
   an iterable of integers. Now only strings and bytes-like objects are accepted.
 
+Documentation
+-------------
+
+- Issue #28513: Documented command-line interface of zipfile.
+
 Tests
 -----
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 06:14:15 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 02 Nov 2016 10:14:15 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328513=3A_Documented_command-line_interface_of_z?=
 =?utf-8?q?ipfile=2E?=
Message-ID: <20161102101408.32001.6420.1C3A58D5@psf.io>

https://hg.python.org/cpython/rev/0c8ffa562f3a
changeset:   104874:0c8ffa562f3a
parent:      104870:a6a79053aec4
parent:      104873:843538a4094b
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 02 12:13:48 2016 +0200
summary:
  Issue #28513: Documented command-line interface of zipfile.

files:
  Doc/library/zipfile.rst |  60 +++++++++++++++++++++++++++++
  Misc/NEWS               |   5 ++
  2 files changed, 65 insertions(+), 0 deletions(-)


diff --git a/Doc/library/zipfile.rst b/Doc/library/zipfile.rst
--- a/Doc/library/zipfile.rst
+++ b/Doc/library/zipfile.rst
@@ -632,4 +632,64 @@
    Size of the uncompressed file.
 
 
+.. _zipfile-commandline:
+.. program:: zipfile
+
+Command-Line Interface
+----------------------
+
+The :mod:`zipfile` module provides a simple command-line interface to interact
+with ZIP archives.
+
+If you want to create a new ZIP archive, specify its name after the :option:`-c`
+option and then list the filename(s) that should be included:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -c monty.zip spam.txt eggs.txt
+
+Passing a directory is also acceptable:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -c monty.zip life-of-brian_1979/
+
+If you want to extract a ZIP archive into the specified directory, use
+the :option:`-e` option:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -e monty.zip target-dir/
+
+For a list of the files in a ZIP archive, use the :option:`-l` option:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -l monty.zip
+
+
+Command-line options
+~~~~~~~~~~~~~~~~~~~~
+
+.. cmdoption:: -l <zipfile>
+               --list <zipfile>
+
+   List files in a zipfile.
+
+.. cmdoption:: -c <zipfile> <source1> ... <sourceN>
+               --create <zipfile> <source1> ... <sourceN>
+
+   Create zipfile from source files.
+
+.. cmdoption:: -e <zipfile> <output_dir>
+               --extract <zipfile> <output_dir>
+
+   Extract zipfile into target directory.
+
+.. cmdoption:: -t <zipfile>
+               --test <zipfile>
+
+   Test whether the zipfile is valid or not.
+
+
 .. _PKZIP Application Note: https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -374,6 +374,11 @@
   PyUnicode_AsDecodedObject(), PyUnicode_AsDecodedUnicode() and
   PyUnicode_AsEncodedUnicode().
 
+Documentation
+-------------
+
+- Issue #28513: Documented command-line interface of zipfile.
+
 Build
 -----
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 06:14:15 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 02 Nov 2016 10:14:15 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328513=3A_Documented_command-line_interface_of_zipfile?=
 =?utf-8?q?=2E?=
Message-ID: <20161102101407.22278.10086.416BFCE0@psf.io>

https://hg.python.org/cpython/rev/843538a4094b
changeset:   104873:843538a4094b
branch:      3.6
parent:      104869:4e9c7704f373
parent:      104872:b51bf32defb1
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 02 12:11:32 2016 +0200
summary:
  Issue #28513: Documented command-line interface of zipfile.

files:
  Doc/library/zipfile.rst |  56 +++++++++++++++++++++++++++++
  Misc/NEWS               |   5 ++
  2 files changed, 61 insertions(+), 0 deletions(-)


diff --git a/Doc/library/zipfile.rst b/Doc/library/zipfile.rst
--- a/Doc/library/zipfile.rst
+++ b/Doc/library/zipfile.rst
@@ -632,4 +632,60 @@
    Size of the uncompressed file.
 
 
+.. _zipfile-commandline:
+.. program:: zipfile
+
+Command-Line Interface
+----------------------
+
+The :mod:`zipfile` module provides a simple command-line interface to interact
+with ZIP archives.
+
+If you want to create a new ZIP archive, specify its name after the :option:`-c`
+option and then list the filename(s) that should be included:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -c monty.zip spam.txt eggs.txt
+
+Passing a directory is also acceptable:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -c monty.zip life-of-brian_1979/
+
+If you want to extract a ZIP archive into the specified directory, use
+the :option:`-e` option:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -e monty.zip target-dir/
+
+For a list of the files in a ZIP archive, use the :option:`-l` option:
+
+.. code-block:: shell-session
+
+    $ python -m zipfile -l monty.zip
+
+
+Command-line options
+~~~~~~~~~~~~~~~~~~~~
+
+.. cmdoption:: -l <zipfile>
+
+   List files in a zipfile.
+
+.. cmdoption:: -c <zipfile> <source1> ... <sourceN>
+
+   Create zipfile from source files.
+
+.. cmdoption:: -e <zipfile> <output_dir>
+
+   Extract zipfile into target directory.
+
+.. cmdoption:: -t <zipfile>
+
+   Test whether the zipfile is valid or not.
+
+
 .. _PKZIP Application Note: https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -16,6 +16,11 @@
 Library
 -------
 
+Documentation
+-------------
+
+- Issue #28513: Documented command-line interface of zipfile.
+
 
 What's New in Python 3.6.0 beta 3
 =================================

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Wed Nov  2 12:56:13 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Wed, 2 Nov 2016 16:56:13 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python 2.7 2016-11-02
Message-ID: <79fea726-df18-489b-ad79-afcacb030a19@irsmsx104.ger.corp.intel.com>

Results for project Python 2.7, build date 2016-11-02 03:47:49 +0000
commit:		8f9c54a75c3d
previous commit:	33ad26897e30
revision date:	2016-11-01 06:44:04 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.14%          1.33%          5.53%             4.22%
:-)             pybench      0.15%          0.06%          5.81%             3.18%
:-(            regex_v8      0.69%          0.18%         -2.07%            10.47%
:-)               nbody      0.10%         -0.03%          7.93%             0.51%
:-)        json_dump_v2      0.32%         -0.41%          2.17%             8.21%
:-|      normal_startup      0.74%          0.16%         -0.30%             2.35%
:-)             ssbench      0.21%         -0.13%          2.04%             2.16%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-2-7-2016-11-02/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Wed Nov  2 12:58:07 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Wed, 2 Nov 2016 16:58:07 +0000
Subject: [Python-checkins] NEUTRAL Benchmark Results for Python Default
 2016-11-02
Message-ID: <40819b02-557c-41c8-be83-30dae1837f50@irsmsx104.ger.corp.intel.com>

Results for project Python default, build date 2016-11-02 03:01:27 +0000
commit:		a6e59a2e880e
previous commit:	9e66ffa7a791
revision date:	2016-11-01 05:54:12 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.19%         -0.14%          4.36%            15.07%
:-)             pybench      0.12%         -0.04%          5.02%             4.95%
:-(            regex_v8      3.73%          0.11%         -3.64%             3.74%
:-)               nbody      0.19%         -0.02%          3.55%             3.31%
:-(        json_dump_v2      0.24%         -0.42%        -11.83%            15.89%
:-|      normal_startup      0.38%         -0.17%         -0.77%             6.51%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-default-2016-11-02/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Wed Nov  2 15:06:33 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 02 Nov 2016 19:06:33 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Fixed_hyperlinks_for_tarfile_CLI_options=2E?=
Message-ID: <20161102190633.30882.40006.6B0EE642@psf.io>

https://hg.python.org/cpython/rev/45968424e68d
changeset:   104876:45968424e68d
branch:      3.6
parent:      104873:843538a4094b
parent:      104875:a08eeb10d700
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 02 21:05:49 2016 +0200
summary:
  Fixed hyperlinks for tarfile CLI options.

files:
  Doc/library/tarfile.rst |  13 +++++++------
  1 files changed, 7 insertions(+), 6 deletions(-)


diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst
--- a/Doc/library/tarfile.rst
+++ b/Doc/library/tarfile.rst
@@ -644,13 +644,14 @@
 
 
 .. _tarfile-commandline:
+.. program:: tarfile
 
-Command Line Interface
+Command-Line Interface
 ----------------------
 
 .. versionadded:: 3.4
 
-The :mod:`tarfile` module provides a simple command line interface to interact
+The :mod:`tarfile` module provides a simple command-line interface to interact
 with tar archives.
 
 If you want to create a new tar archive, specify its name after the :option:`-c`
@@ -687,7 +688,7 @@
     $ python -m tarfile -l monty.tar
 
 
-Command line options
+Command-line options
 ~~~~~~~~~~~~~~~~~~~~
 
 .. cmdoption:: -l <tarfile>
@@ -695,8 +696,8 @@
 
    List files in a tarfile.
 
-.. cmdoption:: -c <tarfile> <source1> <sourceN>
-               --create <tarfile> <source1> <sourceN>
+.. cmdoption:: -c <tarfile> <source1> ... <sourceN>
+               --create <tarfile> <source1> ... <sourceN>
 
    Create tarfile from source files.
 
@@ -712,7 +713,7 @@
 
 .. cmdoption:: -v, --verbose
 
-   Verbose output
+   Verbose output.
 
 .. _tar-examples:
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 15:06:33 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 02 Nov 2016 19:06:33 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Fixed_hyperlinks_for_tarfile_CLI_options=2E?=
Message-ID: <20161102190633.22214.20566.DB2E6270@psf.io>

https://hg.python.org/cpython/rev/3ee7dcad1af0
changeset:   104877:3ee7dcad1af0
parent:      104874:0c8ffa562f3a
parent:      104876:45968424e68d
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 02 21:06:13 2016 +0200
summary:
  Fixed hyperlinks for tarfile CLI options.

files:
  Doc/library/tarfile.rst |  13 +++++++------
  1 files changed, 7 insertions(+), 6 deletions(-)


diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst
--- a/Doc/library/tarfile.rst
+++ b/Doc/library/tarfile.rst
@@ -644,13 +644,14 @@
 
 
 .. _tarfile-commandline:
+.. program:: tarfile
 
-Command Line Interface
+Command-Line Interface
 ----------------------
 
 .. versionadded:: 3.4
 
-The :mod:`tarfile` module provides a simple command line interface to interact
+The :mod:`tarfile` module provides a simple command-line interface to interact
 with tar archives.
 
 If you want to create a new tar archive, specify its name after the :option:`-c`
@@ -687,7 +688,7 @@
     $ python -m tarfile -l monty.tar
 
 
-Command line options
+Command-line options
 ~~~~~~~~~~~~~~~~~~~~
 
 .. cmdoption:: -l <tarfile>
@@ -695,8 +696,8 @@
 
    List files in a tarfile.
 
-.. cmdoption:: -c <tarfile> <source1> <sourceN>
-               --create <tarfile> <source1> <sourceN>
+.. cmdoption:: -c <tarfile> <source1> ... <sourceN>
+               --create <tarfile> <source1> ... <sourceN>
 
    Create tarfile from source files.
 
@@ -712,7 +713,7 @@
 
 .. cmdoption:: -v, --verbose
 
-   Verbose output
+   Verbose output.
 
 .. _tar-examples:
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 15:06:33 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 02 Nov 2016 19:06:33 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fixed_hyperlin?=
 =?utf-8?q?ks_for_tarfile_CLI_options=2E?=
Message-ID: <20161102190633.7863.43094.5979CF27@psf.io>

https://hg.python.org/cpython/rev/a08eeb10d700
changeset:   104875:a08eeb10d700
branch:      3.5
parent:      104872:b51bf32defb1
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 02 21:04:45 2016 +0200
summary:
  Fixed hyperlinks for tarfile CLI options.

files:
  Doc/library/tarfile.rst |  13 +++++++------
  1 files changed, 7 insertions(+), 6 deletions(-)


diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst
--- a/Doc/library/tarfile.rst
+++ b/Doc/library/tarfile.rst
@@ -644,13 +644,14 @@
 
 
 .. _tarfile-commandline:
+.. program:: tarfile
 
-Command Line Interface
+Command-Line Interface
 ----------------------
 
 .. versionadded:: 3.4
 
-The :mod:`tarfile` module provides a simple command line interface to interact
+The :mod:`tarfile` module provides a simple command-line interface to interact
 with tar archives.
 
 If you want to create a new tar archive, specify its name after the :option:`-c`
@@ -687,7 +688,7 @@
     $ python -m tarfile -l monty.tar
 
 
-Command line options
+Command-line options
 ~~~~~~~~~~~~~~~~~~~~
 
 .. cmdoption:: -l <tarfile>
@@ -695,8 +696,8 @@
 
    List files in a tarfile.
 
-.. cmdoption:: -c <tarfile> <source1> <sourceN>
-               --create <tarfile> <source1> <sourceN>
+.. cmdoption:: -c <tarfile> <source1> ... <sourceN>
+               --create <tarfile> <source1> ... <sourceN>
 
    Create tarfile from source files.
 
@@ -712,7 +713,7 @@
 
 .. cmdoption:: -v, --verbose
 
-   Verbose output
+   Verbose output.
 
 .. _tar-examples:
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 15:55:34 2016
From: python-checkins at python.org (donald.stufft)
Date: Wed, 02 Nov 2016 19:55:34 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?=
 =?utf-8?q?_Merge_new_pip/setuptools_from_3=2E4?=
Message-ID: <20161102195534.30792.82575.B7B7072E@psf.io>

https://hg.python.org/cpython/rev/b06f232b4a38
changeset:   104879:b06f232b4a38
branch:      3.5
parent:      104875:a08eeb10d700
parent:      104878:fc4d1fb3f795
user:        Donald Stufft <donald at stufft.io>
date:        Wed Nov 02 15:53:58 2016 -0400
summary:
  Merge new pip/setuptools from 3.4

files:
  Lib/ensurepip/__init__.py                                     |    4 ++--
  Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl |  Bin 
  Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl |  Bin 
  5 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -8,9 +8,9 @@
 __all__ = ["version", "bootstrap"]
 
 
-_SETUPTOOLS_VERSION = "27.1.2"
+_SETUPTOOLS_VERSION = "28.7.1"
 
-_PIP_VERSION = "8.1.2"
+_PIP_VERSION = "9.0.0"
 
 # pip currently requires ssl support, so we try to provide a nicer
 # error message when that is missing (http://bugs.python.org/issue19744)
diff --git a/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl
deleted file mode 100644
index cc49227a0c7e13757f4863a9b7ace1eb56c3ce61..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..2c631a846c10cf51d32241263aff9c38a5bd8e27
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl
deleted file mode 100644
index de213bb72715caa07d12e94f97033110c1767e8d..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..cbcc2200f46eb05d707ad7c2c581ac36e561c256
GIT binary patch
[stripped]

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 15:55:34 2016
From: python-checkins at python.org (donald.stufft)
Date: Wed, 02 Nov 2016 19:55:34 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Upgrade_pip_to?=
 =?utf-8?q?_9=2E0_and_setuptools_to_28=2E7=2E1?=
Message-ID: <20161102195533.31813.66230.B12D345A@psf.io>

https://hg.python.org/cpython/rev/fc4d1fb3f795
changeset:   104878:fc4d1fb3f795
branch:      3.4
parent:      104691:71dce630dc02
user:        Donald Stufft <donald at stufft.io>
date:        Wed Nov 02 15:42:49 2016 -0400
summary:
  Upgrade pip to 9.0 and setuptools to 28.7.1

files:
  Lib/ensurepip/__init__.py                                     |    4 ++--
  Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl |  Bin 
  Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl |  Bin 
  5 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -8,9 +8,9 @@
 __all__ = ["version", "bootstrap"]
 
 
-_SETUPTOOLS_VERSION = "27.1.2"
+_SETUPTOOLS_VERSION = "28.7.1"
 
-_PIP_VERSION = "8.1.2"
+_PIP_VERSION = "9.0.0"
 
 # pip currently requires ssl support, so we try to provide a nicer
 # error message when that is missing (http://bugs.python.org/issue19744)
diff --git a/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl
deleted file mode 100644
index cc49227a0c7e13757f4863a9b7ace1eb56c3ce61..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..2c631a846c10cf51d32241263aff9c38a5bd8e27
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl
deleted file mode 100644
index de213bb72715caa07d12e94f97033110c1767e8d..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..cbcc2200f46eb05d707ad7c2c581ac36e561c256
GIT binary patch
[stripped]

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 15:55:34 2016
From: python-checkins at python.org (donald.stufft)
Date: Wed, 02 Nov 2016 19:55:34 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_updated_pip/setuptools_from_3=2E5?=
Message-ID: <20161102195534.30882.52892.5B7B99EE@psf.io>

https://hg.python.org/cpython/rev/0e69d97a408e
changeset:   104880:0e69d97a408e
branch:      3.6
parent:      104876:45968424e68d
parent:      104879:b06f232b4a38
user:        Donald Stufft <donald at stufft.io>
date:        Wed Nov 02 15:55:01 2016 -0400
summary:
  Merge updated pip/setuptools from 3.5

files:
  Lib/ensurepip/__init__.py                                     |    4 ++--
  Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl |  Bin 
  Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl |  Bin 
  5 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -8,9 +8,9 @@
 __all__ = ["version", "bootstrap"]
 
 
-_SETUPTOOLS_VERSION = "27.1.2"
+_SETUPTOOLS_VERSION = "28.7.1"
 
-_PIP_VERSION = "8.1.2"
+_PIP_VERSION = "9.0.0"
 
 # pip currently requires ssl support, so we try to provide a nicer
 # error message when that is missing (http://bugs.python.org/issue19744)
diff --git a/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl
deleted file mode 100644
index cc49227a0c7e13757f4863a9b7ace1eb56c3ce61..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..2c631a846c10cf51d32241263aff9c38a5bd8e27
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl
deleted file mode 100644
index de213bb72715caa07d12e94f97033110c1767e8d..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..cbcc2200f46eb05d707ad7c2c581ac36e561c256
GIT binary patch
[stripped]

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 15:55:34 2016
From: python-checkins at python.org (donald.stufft)
Date: Wed, 02 Nov 2016 19:55:34 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_updated_pip/setuptools_from_3=2E6?=
Message-ID: <20161102195534.32327.85569.63C45E06@psf.io>

https://hg.python.org/cpython/rev/be8b133d5d3e
changeset:   104881:be8b133d5d3e
parent:      104877:3ee7dcad1af0
parent:      104880:0e69d97a408e
user:        Donald Stufft <donald at stufft.io>
date:        Wed Nov 02 15:55:26 2016 -0400
summary:
  Merge updated pip/setuptools from 3.6

files:
  Lib/ensurepip/__init__.py                                     |    4 ++--
  Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl |  Bin 
  Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl |  Bin 
  5 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -8,9 +8,9 @@
 __all__ = ["version", "bootstrap"]
 
 
-_SETUPTOOLS_VERSION = "27.1.2"
+_SETUPTOOLS_VERSION = "28.7.1"
 
-_PIP_VERSION = "8.1.2"
+_PIP_VERSION = "9.0.0"
 
 # pip currently requires ssl support, so we try to provide a nicer
 # error message when that is missing (http://bugs.python.org/issue19744)
diff --git a/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl
deleted file mode 100644
index cc49227a0c7e13757f4863a9b7ace1eb56c3ce61..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..2c631a846c10cf51d32241263aff9c38a5bd8e27
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl
deleted file mode 100644
index de213bb72715caa07d12e94f97033110c1767e8d..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..cbcc2200f46eb05d707ad7c2c581ac36e561c256
GIT binary patch
[stripped]

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 16:01:34 2016
From: python-checkins at python.org (donald.stufft)
Date: Wed, 02 Nov 2016 20:01:34 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Update_pip_to_?=
 =?utf-8?q?9=2E0=2E0_and_setuptools_to_28=2E7=2E1?=
Message-ID: <20161102200133.114891.85958.A3238CE8@psf.io>

https://hg.python.org/cpython/rev/088d7dcd25d1
changeset:   104882:088d7dcd25d1
branch:      2.7
parent:      104871:feb6e8678512
user:        Donald Stufft <donald at stufft.io>
date:        Wed Nov 02 16:01:17 2016 -0400
summary:
  Update pip to 9.0.0 and setuptools to 28.7.1

files:
  Lib/ensurepip/__init__.py                                     |    4 ++--
  Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl |  Bin 
  Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl |  Bin 
  5 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -12,9 +12,9 @@
 __all__ = ["version", "bootstrap"]
 
 
-_SETUPTOOLS_VERSION = "27.1.2"
+_SETUPTOOLS_VERSION = "28.7.1"
 
-_PIP_VERSION = "8.1.2"
+_PIP_VERSION = "9.0.0"
 
 # pip currently requires ssl support, so we try to provide a nicer
 # error message when that is missing (http://bugs.python.org/issue19744)
diff --git a/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.1.2-py2.py3-none-any.whl
deleted file mode 100644
index cc49227a0c7e13757f4863a9b7ace1eb56c3ce61..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..2c631a846c10cf51d32241263aff9c38a5bd8e27
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl
deleted file mode 100644
index de213bb72715caa07d12e94f97033110c1767e8d..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..cbcc2200f46eb05d707ad7c2c581ac36e561c256
GIT binary patch
[stripped]

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 16:20:26 2016
From: python-checkins at python.org (donald.stufft)
Date: Wed, 02 Nov 2016 20:20:26 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogcGlwIDkuMCB3b3Jr?=
 =?utf-8?q?s_fine_without_the_ssl_module?=
Message-ID: <20161102202025.25285.56065.F3531EED@psf.io>

https://hg.python.org/cpython/rev/c0a0abce38f2
changeset:   104883:c0a0abce38f2
branch:      2.7
user:        Donald Stufft <donald at stufft.io>
date:        Wed Nov 02 16:20:20 2016 -0400
summary:
  pip 9.0 works fine without the ssl module

files:
  Lib/ensurepip/__init__.py  |  21 -------
  Lib/test/test_ensurepip.py |  69 --------------------------
  2 files changed, 0 insertions(+), 90 deletions(-)


diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -16,20 +16,6 @@
 
 _PIP_VERSION = "9.0.0"
 
-# pip currently requires ssl support, so we try to provide a nicer
-# error message when that is missing (http://bugs.python.org/issue19744)
-_MISSING_SSL_MESSAGE = ("pip {} requires SSL/TLS".format(_PIP_VERSION))
-try:
-    import ssl
-except ImportError:
-    ssl = None
-
-    def _require_ssl_for_pip():
-        raise RuntimeError(_MISSING_SSL_MESSAGE)
-else:
-    def _require_ssl_for_pip():
-        pass
-
 _PROJECTS = [
     ("setuptools", _SETUPTOOLS_VERSION),
     ("pip", _PIP_VERSION),
@@ -77,7 +63,6 @@
     if altinstall and default_pip:
         raise ValueError("Cannot use altinstall and default_pip together")
 
-    _require_ssl_for_pip()
     _disable_pip_configuration_settings()
 
     # By default, installing pip and setuptools installs all of the
@@ -143,7 +128,6 @@
         print(msg.format(pip.__version__, _PIP_VERSION), file=sys.stderr)
         return
 
-    _require_ssl_for_pip()
     _disable_pip_configuration_settings()
 
     # Construct the arguments to be passed to the pip command
@@ -155,11 +139,6 @@
 
 
 def _main(argv=None):
-    if ssl is None:
-        print("Ignoring ensurepip failure: {}".format(_MISSING_SSL_MESSAGE),
-              file=sys.stderr)
-        return
-
     import argparse
     parser = argparse.ArgumentParser(prog="python -m ensurepip")
     parser.add_argument(
diff --git a/Lib/test/test_ensurepip.py b/Lib/test/test_ensurepip.py
--- a/Lib/test/test_ensurepip.py
+++ b/Lib/test/test_ensurepip.py
@@ -9,22 +9,6 @@
 import ensurepip
 import ensurepip._uninstall
 
-# pip currently requires ssl support, so we ensure we handle
-# it being missing (http://bugs.python.org/issue19744)
-ensurepip_no_ssl = test.test_support.import_fresh_module("ensurepip",
-                                                         blocked=["ssl"])
-try:
-    import ssl
-except ImportError:
-    ssl = None
-
-    def requires_usable_pip(f):
-        deco = unittest.skip(ensurepip._MISSING_SSL_MESSAGE)
-        return deco(f)
-else:
-    def requires_usable_pip(f):
-        return f
-
 
 class TestEnsurePipVersion(unittest.TestCase):
 
@@ -51,7 +35,6 @@
 
 class TestBootstrap(EnsurepipMixin, unittest.TestCase):
 
-    @requires_usable_pip
     def test_basic_bootstrapping(self):
         ensurepip.bootstrap()
 
@@ -66,7 +49,6 @@
         additional_paths = self.run_pip.call_args[0][1]
         self.assertEqual(len(additional_paths), 2)
 
-    @requires_usable_pip
     def test_bootstrapping_with_root(self):
         ensurepip.bootstrap(root="/foo/bar/")
 
@@ -79,7 +61,6 @@
             mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_user(self):
         ensurepip.bootstrap(user=True)
 
@@ -91,7 +72,6 @@
             mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_upgrade(self):
         ensurepip.bootstrap(upgrade=True)
 
@@ -103,7 +83,6 @@
             mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_verbosity_1(self):
         ensurepip.bootstrap(verbosity=1)
 
@@ -115,7 +94,6 @@
             mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_verbosity_2(self):
         ensurepip.bootstrap(verbosity=2)
 
@@ -127,7 +105,6 @@
             mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_verbosity_3(self):
         ensurepip.bootstrap(verbosity=3)
 
@@ -139,17 +116,14 @@
             mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_regular_install(self):
         ensurepip.bootstrap()
         self.assertEqual(self.os_environ["ENSUREPIP_OPTIONS"], "install")
 
-    @requires_usable_pip
     def test_bootstrapping_with_alt_install(self):
         ensurepip.bootstrap(altinstall=True)
         self.assertEqual(self.os_environ["ENSUREPIP_OPTIONS"], "altinstall")
 
-    @requires_usable_pip
     def test_bootstrapping_with_default_pip(self):
         ensurepip.bootstrap(default_pip=True)
         self.assertNotIn("ENSUREPIP_OPTIONS", self.os_environ)
@@ -159,7 +133,6 @@
             ensurepip.bootstrap(altinstall=True, default_pip=True)
         self.assertFalse(self.run_pip.called)
 
-    @requires_usable_pip
     def test_pip_environment_variables_removed(self):
         # ensurepip deliberately ignores all pip environment variables
         # See http://bugs.python.org/issue19734 for details
@@ -167,7 +140,6 @@
         ensurepip.bootstrap()
         self.assertNotIn("PIP_THIS_SHOULD_GO_AWAY", self.os_environ)
 
-    @requires_usable_pip
     def test_pip_config_file_disabled(self):
         # ensurepip deliberately ignores the pip config file
         # See http://bugs.python.org/issue20053 for details
@@ -210,7 +182,6 @@
         self.assertIn("only uninstall a matching version", warning)
         self.assertFalse(self.run_pip.called)
 
-    @requires_usable_pip
     def test_uninstall(self):
         with fake_pip():
             ensurepip._uninstall_helper()
@@ -222,7 +193,6 @@
             ]
         )
 
-    @requires_usable_pip
     def test_uninstall_with_verbosity_1(self):
         with fake_pip():
             ensurepip._uninstall_helper(verbosity=1)
@@ -234,7 +204,6 @@
             ]
         )
 
-    @requires_usable_pip
     def test_uninstall_with_verbosity_2(self):
         with fake_pip():
             ensurepip._uninstall_helper(verbosity=2)
@@ -246,7 +215,6 @@
             ]
         )
 
-    @requires_usable_pip
     def test_uninstall_with_verbosity_3(self):
         with fake_pip():
             ensurepip._uninstall_helper(verbosity=3)
@@ -258,7 +226,6 @@
             ]
         )
 
-    @requires_usable_pip
     def test_pip_environment_variables_removed(self):
         # ensurepip deliberately ignores all pip environment variables
         # See http://bugs.python.org/issue19734 for details
@@ -267,7 +234,6 @@
             ensurepip._uninstall_helper()
         self.assertNotIn("PIP_THIS_SHOULD_GO_AWAY", self.os_environ)
 
-    @requires_usable_pip
     def test_pip_config_file_disabled(self):
         # ensurepip deliberately ignores the pip config file
         # See http://bugs.python.org/issue20053 for details
@@ -276,38 +242,6 @@
         self.assertEqual(self.os_environ["PIP_CONFIG_FILE"], os.devnull)
 
 
-class TestMissingSSL(EnsurepipMixin, unittest.TestCase):
-
-    def setUp(self):
-        sys.modules["ensurepip"] = ensurepip_no_ssl
-
-        @self.addCleanup
-        def restore_module():
-            sys.modules["ensurepip"] = ensurepip
-        super(TestMissingSSL, self).setUp()
-
-    def test_bootstrap_requires_ssl(self):
-        self.os_environ["PIP_THIS_SHOULD_STAY"] = "test fodder"
-        with self.assertRaisesRegexp(RuntimeError, "requires SSL/TLS"):
-            ensurepip_no_ssl.bootstrap()
-        self.assertFalse(self.run_pip.called)
-        self.assertIn("PIP_THIS_SHOULD_STAY", self.os_environ)
-
-    def test_uninstall_requires_ssl(self):
-        self.os_environ["PIP_THIS_SHOULD_STAY"] = "test fodder"
-        with self.assertRaisesRegexp(RuntimeError, "requires SSL/TLS"):
-            with fake_pip():
-                ensurepip_no_ssl._uninstall_helper()
-        self.assertFalse(self.run_pip.called)
-        self.assertIn("PIP_THIS_SHOULD_STAY", self.os_environ)
-
-    def test_main_exits_early_with_warning(self):
-        with test.test_support.captured_stderr() as stderr:
-            ensurepip_no_ssl._main(["--version"])
-        warning = stderr.getvalue().strip()
-        self.assertTrue(warning.endswith("requires SSL/TLS"), warning)
-        self.assertFalse(self.run_pip.called)
-
 # Basic testing of the main functions and their argument parsing
 
 EXPECTED_VERSION_OUTPUT = "pip " + ensurepip._PIP_VERSION
@@ -315,7 +249,6 @@
 
 class TestBootstrappingMainFunction(EnsurepipMixin, unittest.TestCase):
 
-    @requires_usable_pip
     def test_bootstrap_version(self):
         with test.test_support.captured_stderr() as stderr:
             with self.assertRaises(SystemExit):
@@ -324,7 +257,6 @@
         self.assertEqual(result, EXPECTED_VERSION_OUTPUT)
         self.assertFalse(self.run_pip.called)
 
-    @requires_usable_pip
     def test_basic_bootstrapping(self):
         ensurepip._main([])
 
@@ -350,7 +282,6 @@
         self.assertEqual(result, EXPECTED_VERSION_OUTPUT)
         self.assertFalse(self.run_pip.called)
 
-    @requires_usable_pip
     def test_basic_uninstall(self):
         with fake_pip():
             ensurepip._uninstall._main([])

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 20:33:12 2016
From: python-checkins at python.org (donald.stufft)
Date: Thu, 03 Nov 2016 00:33:12 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Allow_ensurepi?=
 =?utf-8?q?p_even_when_ssl_is_unavailable?=
Message-ID: <20161103003312.63013.37632.15FD7760@psf.io>

https://hg.python.org/cpython/rev/d1e6b12e33fd
changeset:   104884:d1e6b12e33fd
branch:      3.6
parent:      104880:0e69d97a408e
user:        Donald Stufft <donald at stufft.io>
date:        Wed Nov 02 20:32:37 2016 -0400
summary:
  Allow ensurepip even when ssl is unavailable

files:
  Lib/ensurepip/__init__.py  |  20 --------
  Lib/test/test_ensurepip.py |  65 --------------------------
  Lib/test/test_venv.py      |   8 ---
  3 files changed, 0 insertions(+), 93 deletions(-)


diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -12,19 +12,6 @@
 
 _PIP_VERSION = "9.0.0"
 
-# pip currently requires ssl support, so we try to provide a nicer
-# error message when that is missing (http://bugs.python.org/issue19744)
-_MISSING_SSL_MESSAGE = ("pip {} requires SSL/TLS".format(_PIP_VERSION))
-try:
-    import ssl
-except ImportError:
-    ssl = None
-    def _require_ssl_for_pip():
-        raise RuntimeError(_MISSING_SSL_MESSAGE)
-else:
-    def _require_ssl_for_pip():
-        pass
-
 _PROJECTS = [
     ("setuptools", _SETUPTOOLS_VERSION),
     ("pip", _PIP_VERSION),
@@ -71,7 +58,6 @@
     if altinstall and default_pip:
         raise ValueError("Cannot use altinstall and default_pip together")
 
-    _require_ssl_for_pip()
     _disable_pip_configuration_settings()
 
     # By default, installing pip and setuptools installs all of the
@@ -133,7 +119,6 @@
         print(msg.format(pip.__version__, _PIP_VERSION), file=sys.stderr)
         return
 
-    _require_ssl_for_pip()
     _disable_pip_configuration_settings()
 
     # Construct the arguments to be passed to the pip command
@@ -145,11 +130,6 @@
 
 
 def _main(argv=None):
-    if ssl is None:
-        print("Ignoring ensurepip failure: {}".format(_MISSING_SSL_MESSAGE),
-              file=sys.stderr)
-        return
-
     import argparse
     parser = argparse.ArgumentParser(prog="python -m ensurepip")
     parser.add_argument(
diff --git a/Lib/test/test_ensurepip.py b/Lib/test/test_ensurepip.py
--- a/Lib/test/test_ensurepip.py
+++ b/Lib/test/test_ensurepip.py
@@ -9,19 +9,6 @@
 import ensurepip
 import ensurepip._uninstall
 
-# pip currently requires ssl support, so we ensure we handle
-# it being missing (http://bugs.python.org/issue19744)
-ensurepip_no_ssl = test.support.import_fresh_module("ensurepip",
-                                                    blocked=["ssl"])
-try:
-    import ssl
-except ImportError:
-    def requires_usable_pip(f):
-        deco = unittest.skip(ensurepip._MISSING_SSL_MESSAGE)
-        return deco(f)
-else:
-    def requires_usable_pip(f):
-        return f
 
 class TestEnsurePipVersion(unittest.TestCase):
 
@@ -47,7 +34,6 @@
 
 class TestBootstrap(EnsurepipMixin, unittest.TestCase):
 
-    @requires_usable_pip
     def test_basic_bootstrapping(self):
         ensurepip.bootstrap()
 
@@ -62,7 +48,6 @@
         additional_paths = self.run_pip.call_args[0][1]
         self.assertEqual(len(additional_paths), 2)
 
-    @requires_usable_pip
     def test_bootstrapping_with_root(self):
         ensurepip.bootstrap(root="/foo/bar/")
 
@@ -75,7 +60,6 @@
             unittest.mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_user(self):
         ensurepip.bootstrap(user=True)
 
@@ -87,7 +71,6 @@
             unittest.mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_upgrade(self):
         ensurepip.bootstrap(upgrade=True)
 
@@ -99,7 +82,6 @@
             unittest.mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_verbosity_1(self):
         ensurepip.bootstrap(verbosity=1)
 
@@ -111,7 +93,6 @@
             unittest.mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_verbosity_2(self):
         ensurepip.bootstrap(verbosity=2)
 
@@ -123,7 +104,6 @@
             unittest.mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_verbosity_3(self):
         ensurepip.bootstrap(verbosity=3)
 
@@ -135,17 +115,14 @@
             unittest.mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_regular_install(self):
         ensurepip.bootstrap()
         self.assertEqual(self.os_environ["ENSUREPIP_OPTIONS"], "install")
 
-    @requires_usable_pip
     def test_bootstrapping_with_alt_install(self):
         ensurepip.bootstrap(altinstall=True)
         self.assertEqual(self.os_environ["ENSUREPIP_OPTIONS"], "altinstall")
 
-    @requires_usable_pip
     def test_bootstrapping_with_default_pip(self):
         ensurepip.bootstrap(default_pip=True)
         self.assertNotIn("ENSUREPIP_OPTIONS", self.os_environ)
@@ -155,7 +132,6 @@
             ensurepip.bootstrap(altinstall=True, default_pip=True)
         self.assertFalse(self.run_pip.called)
 
-    @requires_usable_pip
     def test_pip_environment_variables_removed(self):
         # ensurepip deliberately ignores all pip environment variables
         # See http://bugs.python.org/issue19734 for details
@@ -163,7 +139,6 @@
         ensurepip.bootstrap()
         self.assertNotIn("PIP_THIS_SHOULD_GO_AWAY", self.os_environ)
 
-    @requires_usable_pip
     def test_pip_config_file_disabled(self):
         # ensurepip deliberately ignores the pip config file
         # See http://bugs.python.org/issue20053 for details
@@ -205,7 +180,6 @@
         self.assertFalse(self.run_pip.called)
 
 
-    @requires_usable_pip
     def test_uninstall(self):
         with fake_pip():
             ensurepip._uninstall_helper()
@@ -217,7 +191,6 @@
             ]
         )
 
-    @requires_usable_pip
     def test_uninstall_with_verbosity_1(self):
         with fake_pip():
             ensurepip._uninstall_helper(verbosity=1)
@@ -229,7 +202,6 @@
             ]
         )
 
-    @requires_usable_pip
     def test_uninstall_with_verbosity_2(self):
         with fake_pip():
             ensurepip._uninstall_helper(verbosity=2)
@@ -241,7 +213,6 @@
             ]
         )
 
-    @requires_usable_pip
     def test_uninstall_with_verbosity_3(self):
         with fake_pip():
             ensurepip._uninstall_helper(verbosity=3)
@@ -253,7 +224,6 @@
             ]
         )
 
-    @requires_usable_pip
     def test_pip_environment_variables_removed(self):
         # ensurepip deliberately ignores all pip environment variables
         # See http://bugs.python.org/issue19734 for details
@@ -262,7 +232,6 @@
             ensurepip._uninstall_helper()
         self.assertNotIn("PIP_THIS_SHOULD_GO_AWAY", self.os_environ)
 
-    @requires_usable_pip
     def test_pip_config_file_disabled(self):
         # ensurepip deliberately ignores the pip config file
         # See http://bugs.python.org/issue20053 for details
@@ -271,44 +240,12 @@
         self.assertEqual(self.os_environ["PIP_CONFIG_FILE"], os.devnull)
 
 
-class TestMissingSSL(EnsurepipMixin, unittest.TestCase):
-
-    def setUp(self):
-        sys.modules["ensurepip"] = ensurepip_no_ssl
-        @self.addCleanup
-        def restore_module():
-            sys.modules["ensurepip"] = ensurepip
-        super().setUp()
-
-    def test_bootstrap_requires_ssl(self):
-        self.os_environ["PIP_THIS_SHOULD_STAY"] = "test fodder"
-        with self.assertRaisesRegex(RuntimeError, "requires SSL/TLS"):
-            ensurepip_no_ssl.bootstrap()
-        self.assertFalse(self.run_pip.called)
-        self.assertIn("PIP_THIS_SHOULD_STAY", self.os_environ)
-
-    def test_uninstall_requires_ssl(self):
-        self.os_environ["PIP_THIS_SHOULD_STAY"] = "test fodder"
-        with self.assertRaisesRegex(RuntimeError, "requires SSL/TLS"):
-            with fake_pip():
-                ensurepip_no_ssl._uninstall_helper()
-        self.assertFalse(self.run_pip.called)
-        self.assertIn("PIP_THIS_SHOULD_STAY", self.os_environ)
-
-    def test_main_exits_early_with_warning(self):
-        with test.support.captured_stderr() as stderr:
-            ensurepip_no_ssl._main(["--version"])
-        warning = stderr.getvalue().strip()
-        self.assertTrue(warning.endswith("requires SSL/TLS"), warning)
-        self.assertFalse(self.run_pip.called)
-
 # Basic testing of the main functions and their argument parsing
 
 EXPECTED_VERSION_OUTPUT = "pip " + ensurepip._PIP_VERSION
 
 class TestBootstrappingMainFunction(EnsurepipMixin, unittest.TestCase):
 
-    @requires_usable_pip
     def test_bootstrap_version(self):
         with test.support.captured_stdout() as stdout:
             with self.assertRaises(SystemExit):
@@ -317,7 +254,6 @@
         self.assertEqual(result, EXPECTED_VERSION_OUTPUT)
         self.assertFalse(self.run_pip.called)
 
-    @requires_usable_pip
     def test_basic_bootstrapping(self):
         ensurepip._main([])
 
@@ -342,7 +278,6 @@
         self.assertEqual(result, EXPECTED_VERSION_OUTPUT)
         self.assertFalse(self.run_pip.called)
 
-    @requires_usable_pip
     def test_basic_uninstall(self):
         with fake_pip():
             ensurepip._uninstall._main([])
diff --git a/Lib/test/test_venv.py b/Lib/test/test_venv.py
--- a/Lib/test/test_venv.py
+++ b/Lib/test/test_venv.py
@@ -18,12 +18,6 @@
 import unittest
 import venv
 
-# pip currently requires ssl support, so we ensure we handle
-# it being missing (http://bugs.python.org/issue19744)
-try:
-    import ssl
-except ImportError:
-    ssl = None
 
 try:
     import threading
@@ -337,8 +331,6 @@
             self.assertTrue(os.path.exists(os.devnull))
 
 
-    # Requesting pip fails without SSL (http://bugs.python.org/issue19744)
-    @unittest.skipIf(ssl is None, ensurepip._MISSING_SSL_MESSAGE)
     @unittest.skipUnless(threading, 'some dependencies of pip import threading'
                                     ' module unconditionally')
     # Issue #26610: pip/pep425tags.py requires ctypes

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  2 20:33:13 2016
From: python-checkins at python.org (donald.stufft)
Date: Thu, 03 Nov 2016 00:33:13 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_3=2E6_-_allow_ensurepip_w/o_ssl?=
Message-ID: <20161103003312.114637.52705.35CE85C7@psf.io>

https://hg.python.org/cpython/rev/b74897b69184
changeset:   104885:b74897b69184
parent:      104881:be8b133d5d3e
parent:      104884:d1e6b12e33fd
user:        Donald Stufft <donald at stufft.io>
date:        Wed Nov 02 20:33:07 2016 -0400
summary:
  Merge 3.6 - allow ensurepip w/o ssl

files:
  Lib/ensurepip/__init__.py  |  20 --------
  Lib/test/test_ensurepip.py |  65 --------------------------
  Lib/test/test_venv.py      |   8 ---
  3 files changed, 0 insertions(+), 93 deletions(-)


diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -12,19 +12,6 @@
 
 _PIP_VERSION = "9.0.0"
 
-# pip currently requires ssl support, so we try to provide a nicer
-# error message when that is missing (http://bugs.python.org/issue19744)
-_MISSING_SSL_MESSAGE = ("pip {} requires SSL/TLS".format(_PIP_VERSION))
-try:
-    import ssl
-except ImportError:
-    ssl = None
-    def _require_ssl_for_pip():
-        raise RuntimeError(_MISSING_SSL_MESSAGE)
-else:
-    def _require_ssl_for_pip():
-        pass
-
 _PROJECTS = [
     ("setuptools", _SETUPTOOLS_VERSION),
     ("pip", _PIP_VERSION),
@@ -71,7 +58,6 @@
     if altinstall and default_pip:
         raise ValueError("Cannot use altinstall and default_pip together")
 
-    _require_ssl_for_pip()
     _disable_pip_configuration_settings()
 
     # By default, installing pip and setuptools installs all of the
@@ -133,7 +119,6 @@
         print(msg.format(pip.__version__, _PIP_VERSION), file=sys.stderr)
         return
 
-    _require_ssl_for_pip()
     _disable_pip_configuration_settings()
 
     # Construct the arguments to be passed to the pip command
@@ -145,11 +130,6 @@
 
 
 def _main(argv=None):
-    if ssl is None:
-        print("Ignoring ensurepip failure: {}".format(_MISSING_SSL_MESSAGE),
-              file=sys.stderr)
-        return
-
     import argparse
     parser = argparse.ArgumentParser(prog="python -m ensurepip")
     parser.add_argument(
diff --git a/Lib/test/test_ensurepip.py b/Lib/test/test_ensurepip.py
--- a/Lib/test/test_ensurepip.py
+++ b/Lib/test/test_ensurepip.py
@@ -9,19 +9,6 @@
 import ensurepip
 import ensurepip._uninstall
 
-# pip currently requires ssl support, so we ensure we handle
-# it being missing (http://bugs.python.org/issue19744)
-ensurepip_no_ssl = test.support.import_fresh_module("ensurepip",
-                                                    blocked=["ssl"])
-try:
-    import ssl
-except ImportError:
-    def requires_usable_pip(f):
-        deco = unittest.skip(ensurepip._MISSING_SSL_MESSAGE)
-        return deco(f)
-else:
-    def requires_usable_pip(f):
-        return f
 
 class TestEnsurePipVersion(unittest.TestCase):
 
@@ -47,7 +34,6 @@
 
 class TestBootstrap(EnsurepipMixin, unittest.TestCase):
 
-    @requires_usable_pip
     def test_basic_bootstrapping(self):
         ensurepip.bootstrap()
 
@@ -62,7 +48,6 @@
         additional_paths = self.run_pip.call_args[0][1]
         self.assertEqual(len(additional_paths), 2)
 
-    @requires_usable_pip
     def test_bootstrapping_with_root(self):
         ensurepip.bootstrap(root="/foo/bar/")
 
@@ -75,7 +60,6 @@
             unittest.mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_user(self):
         ensurepip.bootstrap(user=True)
 
@@ -87,7 +71,6 @@
             unittest.mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_upgrade(self):
         ensurepip.bootstrap(upgrade=True)
 
@@ -99,7 +82,6 @@
             unittest.mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_verbosity_1(self):
         ensurepip.bootstrap(verbosity=1)
 
@@ -111,7 +93,6 @@
             unittest.mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_verbosity_2(self):
         ensurepip.bootstrap(verbosity=2)
 
@@ -123,7 +104,6 @@
             unittest.mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_verbosity_3(self):
         ensurepip.bootstrap(verbosity=3)
 
@@ -135,17 +115,14 @@
             unittest.mock.ANY,
         )
 
-    @requires_usable_pip
     def test_bootstrapping_with_regular_install(self):
         ensurepip.bootstrap()
         self.assertEqual(self.os_environ["ENSUREPIP_OPTIONS"], "install")
 
-    @requires_usable_pip
     def test_bootstrapping_with_alt_install(self):
         ensurepip.bootstrap(altinstall=True)
         self.assertEqual(self.os_environ["ENSUREPIP_OPTIONS"], "altinstall")
 
-    @requires_usable_pip
     def test_bootstrapping_with_default_pip(self):
         ensurepip.bootstrap(default_pip=True)
         self.assertNotIn("ENSUREPIP_OPTIONS", self.os_environ)
@@ -155,7 +132,6 @@
             ensurepip.bootstrap(altinstall=True, default_pip=True)
         self.assertFalse(self.run_pip.called)
 
-    @requires_usable_pip
     def test_pip_environment_variables_removed(self):
         # ensurepip deliberately ignores all pip environment variables
         # See http://bugs.python.org/issue19734 for details
@@ -163,7 +139,6 @@
         ensurepip.bootstrap()
         self.assertNotIn("PIP_THIS_SHOULD_GO_AWAY", self.os_environ)
 
-    @requires_usable_pip
     def test_pip_config_file_disabled(self):
         # ensurepip deliberately ignores the pip config file
         # See http://bugs.python.org/issue20053 for details
@@ -205,7 +180,6 @@
         self.assertFalse(self.run_pip.called)
 
 
-    @requires_usable_pip
     def test_uninstall(self):
         with fake_pip():
             ensurepip._uninstall_helper()
@@ -217,7 +191,6 @@
             ]
         )
 
-    @requires_usable_pip
     def test_uninstall_with_verbosity_1(self):
         with fake_pip():
             ensurepip._uninstall_helper(verbosity=1)
@@ -229,7 +202,6 @@
             ]
         )
 
-    @requires_usable_pip
     def test_uninstall_with_verbosity_2(self):
         with fake_pip():
             ensurepip._uninstall_helper(verbosity=2)
@@ -241,7 +213,6 @@
             ]
         )
 
-    @requires_usable_pip
     def test_uninstall_with_verbosity_3(self):
         with fake_pip():
             ensurepip._uninstall_helper(verbosity=3)
@@ -253,7 +224,6 @@
             ]
         )
 
-    @requires_usable_pip
     def test_pip_environment_variables_removed(self):
         # ensurepip deliberately ignores all pip environment variables
         # See http://bugs.python.org/issue19734 for details
@@ -262,7 +232,6 @@
             ensurepip._uninstall_helper()
         self.assertNotIn("PIP_THIS_SHOULD_GO_AWAY", self.os_environ)
 
-    @requires_usable_pip
     def test_pip_config_file_disabled(self):
         # ensurepip deliberately ignores the pip config file
         # See http://bugs.python.org/issue20053 for details
@@ -271,44 +240,12 @@
         self.assertEqual(self.os_environ["PIP_CONFIG_FILE"], os.devnull)
 
 
-class TestMissingSSL(EnsurepipMixin, unittest.TestCase):
-
-    def setUp(self):
-        sys.modules["ensurepip"] = ensurepip_no_ssl
-        @self.addCleanup
-        def restore_module():
-            sys.modules["ensurepip"] = ensurepip
-        super().setUp()
-
-    def test_bootstrap_requires_ssl(self):
-        self.os_environ["PIP_THIS_SHOULD_STAY"] = "test fodder"
-        with self.assertRaisesRegex(RuntimeError, "requires SSL/TLS"):
-            ensurepip_no_ssl.bootstrap()
-        self.assertFalse(self.run_pip.called)
-        self.assertIn("PIP_THIS_SHOULD_STAY", self.os_environ)
-
-    def test_uninstall_requires_ssl(self):
-        self.os_environ["PIP_THIS_SHOULD_STAY"] = "test fodder"
-        with self.assertRaisesRegex(RuntimeError, "requires SSL/TLS"):
-            with fake_pip():
-                ensurepip_no_ssl._uninstall_helper()
-        self.assertFalse(self.run_pip.called)
-        self.assertIn("PIP_THIS_SHOULD_STAY", self.os_environ)
-
-    def test_main_exits_early_with_warning(self):
-        with test.support.captured_stderr() as stderr:
-            ensurepip_no_ssl._main(["--version"])
-        warning = stderr.getvalue().strip()
-        self.assertTrue(warning.endswith("requires SSL/TLS"), warning)
-        self.assertFalse(self.run_pip.called)
-
 # Basic testing of the main functions and their argument parsing
 
 EXPECTED_VERSION_OUTPUT = "pip " + ensurepip._PIP_VERSION
 
 class TestBootstrappingMainFunction(EnsurepipMixin, unittest.TestCase):
 
-    @requires_usable_pip
     def test_bootstrap_version(self):
         with test.support.captured_stdout() as stdout:
             with self.assertRaises(SystemExit):
@@ -317,7 +254,6 @@
         self.assertEqual(result, EXPECTED_VERSION_OUTPUT)
         self.assertFalse(self.run_pip.called)
 
-    @requires_usable_pip
     def test_basic_bootstrapping(self):
         ensurepip._main([])
 
@@ -342,7 +278,6 @@
         self.assertEqual(result, EXPECTED_VERSION_OUTPUT)
         self.assertFalse(self.run_pip.called)
 
-    @requires_usable_pip
     def test_basic_uninstall(self):
         with fake_pip():
             ensurepip._uninstall._main([])
diff --git a/Lib/test/test_venv.py b/Lib/test/test_venv.py
--- a/Lib/test/test_venv.py
+++ b/Lib/test/test_venv.py
@@ -18,12 +18,6 @@
 import unittest
 import venv
 
-# pip currently requires ssl support, so we ensure we handle
-# it being missing (http://bugs.python.org/issue19744)
-try:
-    import ssl
-except ImportError:
-    ssl = None
 
 try:
     import threading
@@ -337,8 +331,6 @@
             self.assertTrue(os.path.exists(os.devnull))
 
 
-    # Requesting pip fails without SSL (http://bugs.python.org/issue19744)
-    @unittest.skipIf(ssl is None, ensurepip._MISSING_SSL_MESSAGE)
     @unittest.skipUnless(threading, 'some dependencies of pip import threading'
                                     ' module unconditionally')
     # Issue #26610: pip/pep425tags.py requires ctypes

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov  3 09:39:22 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Thu, 03 Nov 2016 13:39:22 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328387=3A_Fixed_possible_crash_in_=5Fio=2ETextIOWrappe?=
 =?utf-8?q?r_deallocator_when?=
Message-ID: <20161103133919.22278.92812.46412F46@psf.io>

https://hg.python.org/cpython/rev/c4319c0d0131
changeset:   104888:c4319c0d0131
branch:      3.6
parent:      104884:d1e6b12e33fd
parent:      104887:89f7386104e2
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Thu Nov 03 15:38:17 2016 +0200
summary:
  Issue #28387: Fixed possible crash in _io.TextIOWrapper deallocator when
the garbage collector is invoked in other thread.
Based on patch by Sebastian Cufre.

files:
  Misc/ACKS            |   1 +
  Misc/NEWS            |   4 ++++
  Modules/_io/textio.c |  17 +++++------------
  3 files changed, 10 insertions(+), 12 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -321,6 +321,7 @@
 Drew Csillag
 Alessandro Cucci
 Joaquin Cuenca Abela
+Sebastian Cufre
 John Cugini
 Tom Culliton
 Ra?l Cumplido
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -16,6 +16,10 @@
 Library
 -------
 
+- Issue #28387: Fixed possible crash in _io.TextIOWrapper deallocator when
+  the garbage collector is invoked in other thread.  Based on patch by
+  Sebastian Cufre.
+
 Documentation
 -------------
 
diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c
--- a/Modules/_io/textio.c
+++ b/Modules/_io/textio.c
@@ -1103,7 +1103,7 @@
 }
 
 static int
-_textiowrapper_clear(textio *self)
+textiowrapper_clear(textio *self)
 {
     self->ok = 0;
     Py_CLEAR(self->buffer);
@@ -1116,6 +1116,8 @@
     Py_CLEAR(self->snapshot);
     Py_CLEAR(self->errors);
     Py_CLEAR(self->raw);
+
+    Py_CLEAR(self->dict);
     return 0;
 }
 
@@ -1125,11 +1127,11 @@
     self->finalizing = 1;
     if (_PyIOBase_finalize((PyObject *) self) < 0)
         return;
-    _textiowrapper_clear(self);
+    self->ok = 0;
     _PyObject_GC_UNTRACK(self);
     if (self->weakreflist != NULL)
         PyObject_ClearWeakRefs((PyObject *)self);
-    Py_CLEAR(self->dict);
+    textiowrapper_clear(self);
     Py_TYPE(self)->tp_free((PyObject *)self);
 }
 
@@ -1151,15 +1153,6 @@
     return 0;
 }
 
-static int
-textiowrapper_clear(textio *self)
-{
-    if (_textiowrapper_clear(self) < 0)
-        return -1;
-    Py_CLEAR(self->dict);
-    return 0;
-}
-
 static PyObject *
 textiowrapper_closed_get(textio *self, void *context);
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov  3 09:39:28 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Thu, 03 Nov 2016 13:39:28 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328387=3A_Fixed_possible_crash_in_=5Fio=2ETextIO?=
 =?utf-8?q?Wrapper_deallocator_when?=
Message-ID: <20161103133919.25322.75418.54DD27A8@psf.io>

https://hg.python.org/cpython/rev/36af3566b67a
changeset:   104889:36af3566b67a
parent:      104885:b74897b69184
parent:      104888:c4319c0d0131
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Thu Nov 03 15:38:58 2016 +0200
summary:
  Issue #28387: Fixed possible crash in _io.TextIOWrapper deallocator when
the garbage collector is invoked in other thread.
Based on patch by Sebastian Cufre.

files:
  Misc/ACKS            |   1 +
  Misc/NEWS            |   4 ++++
  Modules/_io/textio.c |  17 +++++------------
  3 files changed, 10 insertions(+), 12 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -321,6 +321,7 @@
 Drew Csillag
 Alessandro Cucci
 Joaquin Cuenca Abela
+Sebastian Cufre
 John Cugini
 Tom Culliton
 Ra?l Cumplido
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -106,6 +106,10 @@
 Library
 -------
 
+- Issue #28387: Fixed possible crash in _io.TextIOWrapper deallocator when
+  the garbage collector is invoked in other thread.  Based on patch by
+  Sebastian Cufre.
+
 - Issue #27517: LZMA compressor and decompressor no longer raise exceptions if
   given empty data twice.  Patch by Benjamin Fogle.
 
diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c
--- a/Modules/_io/textio.c
+++ b/Modules/_io/textio.c
@@ -1103,7 +1103,7 @@
 }
 
 static int
-_textiowrapper_clear(textio *self)
+textiowrapper_clear(textio *self)
 {
     self->ok = 0;
     Py_CLEAR(self->buffer);
@@ -1116,6 +1116,8 @@
     Py_CLEAR(self->snapshot);
     Py_CLEAR(self->errors);
     Py_CLEAR(self->raw);
+
+    Py_CLEAR(self->dict);
     return 0;
 }
 
@@ -1125,11 +1127,11 @@
     self->finalizing = 1;
     if (_PyIOBase_finalize((PyObject *) self) < 0)
         return;
-    _textiowrapper_clear(self);
+    self->ok = 0;
     _PyObject_GC_UNTRACK(self);
     if (self->weakreflist != NULL)
         PyObject_ClearWeakRefs((PyObject *)self);
-    Py_CLEAR(self->dict);
+    textiowrapper_clear(self);
     Py_TYPE(self)->tp_free((PyObject *)self);
 }
 
@@ -1151,15 +1153,6 @@
     return 0;
 }
 
-static int
-textiowrapper_clear(textio *self)
-{
-    if (_textiowrapper_clear(self) < 0)
-        return -1;
-    Py_CLEAR(self->dict);
-    return 0;
-}
-
 static PyObject *
 textiowrapper_closed_get(textio *self, void *context);
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov  3 09:39:23 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Thu, 03 Nov 2016 13:39:23 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4Mzg3?=
 =?utf-8?q?=3A_Fixed_possible_crash_in_=5Fio=2ETextIOWrapper_deallocator_w?=
 =?utf-8?q?hen?=
Message-ID: <20161103133919.45169.98992.6F4BD4BE@psf.io>

https://hg.python.org/cpython/rev/89f7386104e2
changeset:   104887:89f7386104e2
branch:      3.5
parent:      104879:b06f232b4a38
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Thu Nov 03 15:37:01 2016 +0200
summary:
  Issue #28387: Fixed possible crash in _io.TextIOWrapper deallocator when
the garbage collector is invoked in other thread.
Based on patch by Sebastian Cufre.

files:
  Misc/ACKS            |   1 +
  Misc/NEWS            |   4 ++++
  Modules/_io/textio.c |  17 +++++------------
  3 files changed, 10 insertions(+), 12 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -315,6 +315,7 @@
 Felipe Cruz
 Drew Csillag
 Joaquin Cuenca Abela
+Sebastian Cufre
 John Cugini
 Tom Culliton
 Ra?l Cumplido
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -113,6 +113,10 @@
 Library
 -------
 
+- Issue #28387: Fixed possible crash in _io.TextIOWrapper deallocator when
+  the garbage collector is invoked in other thread.  Based on patch by
+  Sebastian Cufre.
+
 - Issue #27517: LZMA compressor and decompressor no longer raise exceptions if
   given empty data twice.  Patch by Benjamin Fogle.
 
diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c
--- a/Modules/_io/textio.c
+++ b/Modules/_io/textio.c
@@ -1103,7 +1103,7 @@
 }
 
 static int
-_textiowrapper_clear(textio *self)
+textiowrapper_clear(textio *self)
 {
     self->ok = 0;
     Py_CLEAR(self->buffer);
@@ -1116,6 +1116,8 @@
     Py_CLEAR(self->snapshot);
     Py_CLEAR(self->errors);
     Py_CLEAR(self->raw);
+
+    Py_CLEAR(self->dict);
     return 0;
 }
 
@@ -1125,11 +1127,11 @@
     self->finalizing = 1;
     if (_PyIOBase_finalize((PyObject *) self) < 0)
         return;
-    _textiowrapper_clear(self);
+    self->ok = 0;
     _PyObject_GC_UNTRACK(self);
     if (self->weakreflist != NULL)
         PyObject_ClearWeakRefs((PyObject *)self);
-    Py_CLEAR(self->dict);
+    textiowrapper_clear(self);
     Py_TYPE(self)->tp_free((PyObject *)self);
 }
 
@@ -1151,15 +1153,6 @@
     return 0;
 }
 
-static int
-textiowrapper_clear(textio *self)
-{
-    if (_textiowrapper_clear(self) < 0)
-        return -1;
-    Py_CLEAR(self->dict);
-    return 0;
-}
-
 static PyObject *
 textiowrapper_closed_get(textio *self, void *context);
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov  3 09:39:24 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Thu, 03 Nov 2016 13:39:24 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4Mzg3?=
 =?utf-8?q?=3A_Fixed_possible_crash_in_=5Fio=2ETextIOWrapper_deallocator_w?=
 =?utf-8?q?hen?=
Message-ID: <20161103133918.34612.99735.9AF2EBF0@psf.io>

https://hg.python.org/cpython/rev/91f024fc9b3a
changeset:   104886:91f024fc9b3a
branch:      2.7
parent:      104883:c0a0abce38f2
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Thu Nov 03 15:36:50 2016 +0200
summary:
  Issue #28387: Fixed possible crash in _io.TextIOWrapper deallocator when
the garbage collector is invoked in other thread.
Based on patch by Sebastian Cufre.

files:
  Misc/ACKS            |   1 +
  Misc/NEWS            |   4 ++++
  Modules/_io/textio.c |  15 +++++++--------
  3 files changed, 12 insertions(+), 8 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -296,6 +296,7 @@
 Felipe Cruz
 Drew Csillag
 Joaquin Cuenca Abela
+Sebastian Cufre
 John Cugini
 Tom Culliton
 Antonio Cuni
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -60,6 +60,10 @@
 Library
 -------
 
+- Issue #28387: Fixed possible crash in _io.TextIOWrapper deallocator when
+  the garbage collector is invoked in other thread.  Based on patch by
+  Sebastian Cufre.
+
 - Issue #28449: tarfile.open() with mode "r" or "r:" now tries to open a tar
   file with compression before trying to open it without compression.  Otherwise
   it had 50% chance failed with ignore_zeros=True.
diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c
--- a/Modules/_io/textio.c
+++ b/Modules/_io/textio.c
@@ -1071,11 +1071,9 @@
     return -1;
 }
 
-static int
+static void
 _textiowrapper_clear(textio *self)
 {
-    if (self->ok && _PyIOBase_finalize((PyObject *) self) < 0)
-        return -1;
     self->ok = 0;
     Py_CLEAR(self->buffer);
     Py_CLEAR(self->encoding);
@@ -1087,18 +1085,19 @@
     Py_CLEAR(self->snapshot);
     Py_CLEAR(self->errors);
     Py_CLEAR(self->raw);
-    return 0;
+
+    Py_CLEAR(self->dict);
 }
 
 static void
 textiowrapper_dealloc(textio *self)
 {
-    if (_textiowrapper_clear(self) < 0)
+    if (self->ok && _PyIOBase_finalize((PyObject *) self) < 0)
         return;
     _PyObject_GC_UNTRACK(self);
     if (self->weakreflist != NULL)
         PyObject_ClearWeakRefs((PyObject *)self);
-    Py_CLEAR(self->dict);
+    _textiowrapper_clear(self);
     Py_TYPE(self)->tp_free((PyObject *)self);
 }
 
@@ -1123,9 +1122,9 @@
 static int
 textiowrapper_clear(textio *self)
 {
-    if (_textiowrapper_clear(self) < 0)
+    if (self->ok && _PyIOBase_finalize((PyObject *) self) < 0)
         return -1;
-    Py_CLEAR(self->dict);
+    _textiowrapper_clear(self);
     return 0;
 }
 

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Thu Nov  3 11:41:49 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Thu, 3 Nov 2016 15:41:49 +0000
Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7
 2016-11-03
Message-ID: <508d0782-5c5e-4bb1-824f-b519424d7bca@irsmsx103.ger.corp.intel.com>

Results for project Python 2.7, build date 2016-11-03 03:48:30 +0000
commit:		c0a0abce38f2
previous commit:	8f9c54a75c3d
revision date:	2016-11-02 20:20:20 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.18%          0.06%          5.59%             6.45%
:-)             pybench      0.11%         -0.01%          5.80%             4.37%
:-(            regex_v8      0.64%         -0.21%         -2.29%            11.10%
:-)               nbody      0.06%          0.01%          7.94%             4.04%
:-)        json_dump_v2      0.31%          0.48%          2.65%            10.10%
:-|      normal_startup      0.55%         -0.46%         -0.77%             2.84%
:-)             ssbench      0.17%          0.44%          2.49%             1.76%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-11-03/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Thu Nov  3 11:43:02 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Thu, 3 Nov 2016 15:43:02 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python Default
 2016-11-03
Message-ID: <a6b17cde-f857-4a5c-bdab-19ca394394b2@irsmsx103.ger.corp.intel.com>

Results for project Python default, build date 2016-11-03 03:01:50 +0000
commit:		b74897b69184
previous commit:	a6e59a2e880e
revision date:	2016-11-03 00:33:07 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.23%          0.29%          4.65%            16.57%
:-)             pybench      0.08%          0.40%          5.40%             4.99%
:-|            regex_v8      3.83%          1.59%         -1.99%             2.38%
:-)               nbody      0.25%         -0.47%          3.09%             1.95%
:-(        json_dump_v2      0.24%          0.09%        -11.73%            16.06%
:-|      normal_startup      0.71%          0.07%         -0.59%             6.50%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-11-03/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Thu Nov  3 17:18:47 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Thu, 03 Nov 2016 21:18:47 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2OTgw?=
 =?utf-8?q?=3A_Improve_docs_for_create=5Funix=5Fconnection=28=29=2E_By_Mar?=
 =?utf-8?q?iatta=2E?=
Message-ID: <20161103211847.22539.15868.500D2D2F@psf.io>

https://hg.python.org/cpython/rev/b97b0201c2f4
changeset:   104890:b97b0201c2f4
branch:      3.5
parent:      104887:89f7386104e2
user:        Guido van Rossum <guido at python.org>
date:        Thu Nov 03 14:17:25 2016 -0700
summary:
  Issue #26980: Improve docs for create_unix_connection(). By Mariatta.

files:
  Doc/library/asyncio-eventloop.rst |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst
--- a/Doc/library/asyncio-eventloop.rst
+++ b/Doc/library/asyncio-eventloop.rst
@@ -380,6 +380,10 @@
    establish the connection in the background.  When successful, the
    coroutine returns a ``(transport, protocol)`` pair.
 
+   *path* is the name of a UNIX domain socket, and is required unless a *sock*
+   parameter is specified.  Abstract UNIX sockets, :class:`str`, and
+   :class:`bytes` paths are supported.
+
    See the :meth:`AbstractEventLoop.create_connection` method for parameters.
 
    Availability: UNIX.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov  3 17:18:47 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Thu, 03 Nov 2016 21:18:47 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2326980=3A_Improve_docs_for_create=5Funix=5Fconne?=
 =?utf-8?b?Y3Rpb24oKS4gQnkgTWFyaWF0dGEuICgzLjYtPjMuNyk=?=
Message-ID: <20161103211847.7144.77615.2512F9F7@psf.io>

https://hg.python.org/cpython/rev/d6f4c1b864e6
changeset:   104892:d6f4c1b864e6
parent:      104889:36af3566b67a
parent:      104891:ddbba4739ef4
user:        Guido van Rossum <guido at python.org>
date:        Thu Nov 03 14:18:32 2016 -0700
summary:
  Issue #26980: Improve docs for create_unix_connection(). By Mariatta. (3.6->3.7)

files:
  Doc/library/asyncio-eventloop.rst |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst
--- a/Doc/library/asyncio-eventloop.rst
+++ b/Doc/library/asyncio-eventloop.rst
@@ -381,6 +381,10 @@
    establish the connection in the background.  When successful, the
    coroutine returns a ``(transport, protocol)`` pair.
 
+   *path* is the name of a UNIX domain socket, and is required unless a *sock*
+   parameter is specified.  Abstract UNIX sockets, :class:`str`, and
+   :class:`bytes` paths are supported.
+
    See the :meth:`AbstractEventLoop.create_connection` method for parameters.
 
    Availability: UNIX.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov  3 17:18:48 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Thu, 03 Nov 2016 21:18:48 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2326980=3A_Improve_docs_for_create=5Funix=5Fconnection?=
 =?utf-8?b?KCkuIEJ5IE1hcmlhdHRhLiAoMy41LT4zLjYp?=
Message-ID: <20161103211847.25402.26101.05119AE8@psf.io>

https://hg.python.org/cpython/rev/ddbba4739ef4
changeset:   104891:ddbba4739ef4
branch:      3.6
parent:      104888:c4319c0d0131
parent:      104890:b97b0201c2f4
user:        Guido van Rossum <guido at python.org>
date:        Thu Nov 03 14:18:04 2016 -0700
summary:
  Issue #26980: Improve docs for create_unix_connection(). By Mariatta. (3.5->3.6)

files:
  Doc/library/asyncio-eventloop.rst |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst
--- a/Doc/library/asyncio-eventloop.rst
+++ b/Doc/library/asyncio-eventloop.rst
@@ -381,6 +381,10 @@
    establish the connection in the background.  When successful, the
    coroutine returns a ``(transport, protocol)`` pair.
 
+   *path* is the name of a UNIX domain socket, and is required unless a *sock*
+   parameter is specified.  Abstract UNIX sockets, :class:`str`, and
+   :class:`bytes` paths are supported.
+
    See the :meth:`AbstractEventLoop.create_connection` method for parameters.
 
    Availability: UNIX.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov  3 18:11:02 2016
From: python-checkins at python.org (yury.selivanov)
Date: Thu, 03 Nov 2016 22:11:02 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg2MDAp?=
Message-ID: <20161103221102.34973.50372.80E9D01A@psf.io>

https://hg.python.org/cpython/rev/46c3eede41a6
changeset:   104895:46c3eede41a6
parent:      104892:d6f4c1b864e6
parent:      104894:4f570a612aec
user:        Yury Selivanov <yury at magic.io>
date:        Thu Nov 03 15:10:48 2016 -0700
summary:
  Merge 3.6 (issue #28600)

files:
  Lib/asyncio/base_events.py                |  40 +++++-----
  Lib/asyncio/events.py                     |   1 -
  Lib/test/test_asyncio/test_base_events.py |  36 ++-------
  Lib/test/test_asyncio/test_events.py      |   7 -
  4 files changed, 27 insertions(+), 57 deletions(-)


diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -532,12 +532,10 @@
 
         Absolute time corresponds to the event loop's time() method.
         """
-        if (coroutines.iscoroutine(callback)
-        or coroutines.iscoroutinefunction(callback)):
-            raise TypeError("coroutines cannot be used with call_at()")
         self._check_closed()
         if self._debug:
             self._check_thread()
+            self._check_callback(callback, 'call_at')
         timer = events.TimerHandle(when, callback, args, self)
         if timer._source_traceback:
             del timer._source_traceback[-1]
@@ -555,18 +553,27 @@
         Any positional arguments after the callback will be passed to
         the callback when it is called.
         """
+        self._check_closed()
         if self._debug:
             self._check_thread()
+            self._check_callback(callback, 'call_soon')
         handle = self._call_soon(callback, args)
         if handle._source_traceback:
             del handle._source_traceback[-1]
         return handle
 
+    def _check_callback(self, callback, method):
+        if (coroutines.iscoroutine(callback) or
+                coroutines.iscoroutinefunction(callback)):
+            raise TypeError(
+                "coroutines cannot be used with {}()".format(method))
+        if not callable(callback):
+            raise TypeError(
+                'a callable object was expected by {}(), got {!r}'.format(
+                    method, callback))
+
+
     def _call_soon(self, callback, args):
-        if (coroutines.iscoroutine(callback)
-        or coroutines.iscoroutinefunction(callback)):
-            raise TypeError("coroutines cannot be used with call_soon()")
-        self._check_closed()
         handle = events.Handle(callback, args, self)
         if handle._source_traceback:
             del handle._source_traceback[-1]
@@ -592,6 +599,9 @@
 
     def call_soon_threadsafe(self, callback, *args):
         """Like call_soon(), but thread-safe."""
+        self._check_closed()
+        if self._debug:
+            self._check_callback(callback, 'call_soon_threadsafe')
         handle = self._call_soon(callback, args)
         if handle._source_traceback:
             del handle._source_traceback[-1]
@@ -599,21 +609,9 @@
         return handle
 
     def run_in_executor(self, executor, func, *args):
-        if (coroutines.iscoroutine(func)
-        or coroutines.iscoroutinefunction(func)):
-            raise TypeError("coroutines cannot be used with run_in_executor()")
         self._check_closed()
-        if isinstance(func, events.Handle):
-            assert not args
-            assert not isinstance(func, events.TimerHandle)
-            warnings.warn(
-                "Passing Handle to loop.run_in_executor() is deprecated",
-                DeprecationWarning)
-            if func._cancelled:
-                f = self.create_future()
-                f.set_result(None)
-                return f
-            func, args = func._callback, func._args
+        if self._debug:
+            self._check_callback(func, 'run_in_executor')
         if executor is None:
             executor = self._default_executor
             if executor is None:
diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py
--- a/Lib/asyncio/events.py
+++ b/Lib/asyncio/events.py
@@ -82,7 +82,6 @@
                  '_source_traceback', '_repr', '__weakref__')
 
     def __init__(self, callback, args, loop):
-        assert not isinstance(callback, Handle), 'A Handle is not a callback'
         self._loop = loop
         self._callback = callback
         self._args = args
diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py
--- a/Lib/test/test_asyncio/test_base_events.py
+++ b/Lib/test/test_asyncio/test_base_events.py
@@ -235,6 +235,11 @@
         self.assertIsInstance(h, asyncio.Handle)
         self.assertIn(h, self.loop._ready)
 
+    def test_call_soon_non_callable(self):
+        self.loop.set_debug(True)
+        with self.assertRaisesRegex(TypeError, 'a callable object'):
+            self.loop.call_soon(1)
+
     def test_call_later(self):
         def cb():
             pass
@@ -341,47 +346,21 @@
         # check disabled if debug mode is disabled
         test_thread(self.loop, False, create_loop=True)
 
-    def test_run_once_in_executor_handle(self):
-        def cb():
-            pass
-
-        self.assertRaises(
-            AssertionError, self.loop.run_in_executor,
-            None, asyncio.Handle(cb, (), self.loop), ('',))
-        self.assertRaises(
-            AssertionError, self.loop.run_in_executor,
-            None, asyncio.TimerHandle(10, cb, (), self.loop))
-
-    def test_run_once_in_executor_cancelled(self):
-        def cb():
-            pass
-        h = asyncio.Handle(cb, (), self.loop)
-        h.cancel()
-
-        with self.assertWarnsRegex(DeprecationWarning, "Passing Handle"):
-            f = self.loop.run_in_executor(None, h)
-        self.assertIsInstance(f, asyncio.Future)
-        self.assertTrue(f.done())
-        self.assertIsNone(f.result())
-
     def test_run_once_in_executor_plain(self):
         def cb():
             pass
-        h = asyncio.Handle(cb, (), self.loop)
         f = asyncio.Future(loop=self.loop)
         executor = mock.Mock()
         executor.submit.return_value = f
 
         self.loop.set_default_executor(executor)
 
-        with self.assertWarnsRegex(DeprecationWarning, "Passing Handle"):
-            res = self.loop.run_in_executor(None, h)
+        res = self.loop.run_in_executor(None, cb)
         self.assertIs(f, res)
 
         executor = mock.Mock()
         executor.submit.return_value = f
-        with self.assertWarnsRegex(DeprecationWarning, "Passing Handle"):
-            res = self.loop.run_in_executor(executor, h)
+        res = self.loop.run_in_executor(executor, cb)
         self.assertIs(f, res)
         self.assertTrue(executor.submit.called)
 
@@ -1666,6 +1645,7 @@
         def simple_coroutine():
             pass
 
+        self.loop.set_debug(True)
         coro_func = simple_coroutine
         coro_obj = coro_func()
         self.addCleanup(coro_obj.close)
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -2249,13 +2249,6 @@
         h.cancel()
         self.assertTrue(h._cancelled)
 
-    def test_handle_from_handle(self):
-        def callback(*args):
-            return args
-        h1 = asyncio.Handle(callback, (), loop=self.loop)
-        self.assertRaises(
-            AssertionError, asyncio.Handle, h1, (), self.loop)
-
     def test_callback_with_exception(self):
         def callback():
             raise ValueError()

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov  3 18:11:03 2016
From: python-checkins at python.org (yury.selivanov)
Date: Thu, 03 Nov 2016 22:11:03 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjAw?=
 =?utf-8?q?=3A_Optimize_loop=2Ecall=5Fsoon=28=29=2E?=
Message-ID: <20161103221101.7662.71326.F0C982E1@psf.io>

https://hg.python.org/cpython/rev/128ffe3c3eb9
changeset:   104893:128ffe3c3eb9
branch:      3.5
parent:      104890:b97b0201c2f4
user:        Yury Selivanov <yury at magic.io>
date:        Thu Nov 03 15:09:24 2016 -0700
summary:
  Issue #28600: Optimize loop.call_soon().

Run expensive type checks only in debug mode.  In addition, stop
supporting passing handles to loop.run_in_executor.

files:
  Lib/asyncio/base_events.py                |  40 +++++-----
  Lib/asyncio/events.py                     |   1 -
  Lib/test/test_asyncio/test_base_events.py |  36 ++-------
  Lib/test/test_asyncio/test_events.py      |   7 -
  Misc/NEWS                                 |   2 +
  5 files changed, 29 insertions(+), 57 deletions(-)


diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -528,12 +528,10 @@
 
         Absolute time corresponds to the event loop's time() method.
         """
-        if (coroutines.iscoroutine(callback)
-        or coroutines.iscoroutinefunction(callback)):
-            raise TypeError("coroutines cannot be used with call_at()")
         self._check_closed()
         if self._debug:
             self._check_thread()
+            self._check_callback(callback, 'call_at')
         timer = events.TimerHandle(when, callback, args, self)
         if timer._source_traceback:
             del timer._source_traceback[-1]
@@ -551,18 +549,27 @@
         Any positional arguments after the callback will be passed to
         the callback when it is called.
         """
+        self._check_closed()
         if self._debug:
             self._check_thread()
+            self._check_callback(callback, 'call_soon')
         handle = self._call_soon(callback, args)
         if handle._source_traceback:
             del handle._source_traceback[-1]
         return handle
 
+    def _check_callback(self, callback, method):
+        if (coroutines.iscoroutine(callback) or
+                coroutines.iscoroutinefunction(callback)):
+            raise TypeError(
+                "coroutines cannot be used with {}()".format(method))
+        if not callable(callback):
+            raise TypeError(
+                'a callable object was expected by {}(), got {!r}'.format(
+                    method, callback))
+
+
     def _call_soon(self, callback, args):
-        if (coroutines.iscoroutine(callback)
-        or coroutines.iscoroutinefunction(callback)):
-            raise TypeError("coroutines cannot be used with call_soon()")
-        self._check_closed()
         handle = events.Handle(callback, args, self)
         if handle._source_traceback:
             del handle._source_traceback[-1]
@@ -588,6 +595,9 @@
 
     def call_soon_threadsafe(self, callback, *args):
         """Like call_soon(), but thread-safe."""
+        self._check_closed()
+        if self._debug:
+            self._check_callback(callback, 'call_soon_threadsafe')
         handle = self._call_soon(callback, args)
         if handle._source_traceback:
             del handle._source_traceback[-1]
@@ -595,21 +605,9 @@
         return handle
 
     def run_in_executor(self, executor, func, *args):
-        if (coroutines.iscoroutine(func)
-        or coroutines.iscoroutinefunction(func)):
-            raise TypeError("coroutines cannot be used with run_in_executor()")
         self._check_closed()
-        if isinstance(func, events.Handle):
-            assert not args
-            assert not isinstance(func, events.TimerHandle)
-            warnings.warn(
-                "Passing Handle to loop.run_in_executor() is deprecated",
-                DeprecationWarning)
-            if func._cancelled:
-                f = self.create_future()
-                f.set_result(None)
-                return f
-            func, args = func._callback, func._args
+        if self._debug:
+            self._check_callback(func, 'run_in_executor')
         if executor is None:
             executor = self._default_executor
             if executor is None:
diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py
--- a/Lib/asyncio/events.py
+++ b/Lib/asyncio/events.py
@@ -82,7 +82,6 @@
                  '_source_traceback', '_repr', '__weakref__')
 
     def __init__(self, callback, args, loop):
-        assert not isinstance(callback, Handle), 'A Handle is not a callback'
         self._loop = loop
         self._callback = callback
         self._args = args
diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py
--- a/Lib/test/test_asyncio/test_base_events.py
+++ b/Lib/test/test_asyncio/test_base_events.py
@@ -235,6 +235,11 @@
         self.assertIsInstance(h, asyncio.Handle)
         self.assertIn(h, self.loop._ready)
 
+    def test_call_soon_non_callable(self):
+        self.loop.set_debug(True)
+        with self.assertRaisesRegex(TypeError, 'a callable object'):
+            self.loop.call_soon(1)
+
     def test_call_later(self):
         def cb():
             pass
@@ -341,47 +346,21 @@
         # check disabled if debug mode is disabled
         test_thread(self.loop, False, create_loop=True)
 
-    def test_run_once_in_executor_handle(self):
-        def cb():
-            pass
-
-        self.assertRaises(
-            AssertionError, self.loop.run_in_executor,
-            None, asyncio.Handle(cb, (), self.loop), ('',))
-        self.assertRaises(
-            AssertionError, self.loop.run_in_executor,
-            None, asyncio.TimerHandle(10, cb, (), self.loop))
-
-    def test_run_once_in_executor_cancelled(self):
-        def cb():
-            pass
-        h = asyncio.Handle(cb, (), self.loop)
-        h.cancel()
-
-        with self.assertWarnsRegex(DeprecationWarning, "Passing Handle"):
-            f = self.loop.run_in_executor(None, h)
-        self.assertIsInstance(f, asyncio.Future)
-        self.assertTrue(f.done())
-        self.assertIsNone(f.result())
-
     def test_run_once_in_executor_plain(self):
         def cb():
             pass
-        h = asyncio.Handle(cb, (), self.loop)
         f = asyncio.Future(loop=self.loop)
         executor = mock.Mock()
         executor.submit.return_value = f
 
         self.loop.set_default_executor(executor)
 
-        with self.assertWarnsRegex(DeprecationWarning, "Passing Handle"):
-            res = self.loop.run_in_executor(None, h)
+        res = self.loop.run_in_executor(None, cb)
         self.assertIs(f, res)
 
         executor = mock.Mock()
         executor.submit.return_value = f
-        with self.assertWarnsRegex(DeprecationWarning, "Passing Handle"):
-            res = self.loop.run_in_executor(executor, h)
+        res = self.loop.run_in_executor(executor, cb)
         self.assertIs(f, res)
         self.assertTrue(executor.submit.called)
 
@@ -1666,6 +1645,7 @@
         def simple_coroutine():
             pass
 
+        self.loop.set_debug(True)
         coro_func = simple_coroutine
         coro_obj = coro_func()
         self.addCleanup(coro_obj.close)
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -2249,13 +2249,6 @@
         h.cancel()
         self.assertTrue(h._cancelled)
 
-    def test_handle_from_handle(self):
-        def callback(*args):
-            return args
-        h1 = asyncio.Handle(callback, (), loop=self.loop)
-        self.assertRaises(
-            AssertionError, asyncio.Handle, h1, (), self.loop)
-
     def test_callback_with_exception(self):
         def callback():
             raise ValueError()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -441,6 +441,8 @@
   threadpool executor.
   Initial patch by Hans Lawrenz.
 
+- Issue #28600: Optimize loop.call_soon().
+
 IDLE
 ----
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov  3 18:11:03 2016
From: python-checkins at python.org (yury.selivanov)
Date: Thu, 03 Nov 2016 22:11:03 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5_=28issue_=2328600=29?=
Message-ID: <20161103221101.22391.28967.F63DBCD6@psf.io>

https://hg.python.org/cpython/rev/4f570a612aec
changeset:   104894:4f570a612aec
branch:      3.6
parent:      104891:ddbba4739ef4
parent:      104893:128ffe3c3eb9
user:        Yury Selivanov <yury at magic.io>
date:        Thu Nov 03 15:10:11 2016 -0700
summary:
  Merge 3.5 (issue #28600)

files:
  Lib/asyncio/base_events.py                |  40 +++++-----
  Lib/asyncio/events.py                     |   1 -
  Lib/test/test_asyncio/test_base_events.py |  36 ++-------
  Lib/test/test_asyncio/test_events.py      |   7 -
  Misc/NEWS                                 |   2 +
  5 files changed, 29 insertions(+), 57 deletions(-)


diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -532,12 +532,10 @@
 
         Absolute time corresponds to the event loop's time() method.
         """
-        if (coroutines.iscoroutine(callback)
-        or coroutines.iscoroutinefunction(callback)):
-            raise TypeError("coroutines cannot be used with call_at()")
         self._check_closed()
         if self._debug:
             self._check_thread()
+            self._check_callback(callback, 'call_at')
         timer = events.TimerHandle(when, callback, args, self)
         if timer._source_traceback:
             del timer._source_traceback[-1]
@@ -555,18 +553,27 @@
         Any positional arguments after the callback will be passed to
         the callback when it is called.
         """
+        self._check_closed()
         if self._debug:
             self._check_thread()
+            self._check_callback(callback, 'call_soon')
         handle = self._call_soon(callback, args)
         if handle._source_traceback:
             del handle._source_traceback[-1]
         return handle
 
+    def _check_callback(self, callback, method):
+        if (coroutines.iscoroutine(callback) or
+                coroutines.iscoroutinefunction(callback)):
+            raise TypeError(
+                "coroutines cannot be used with {}()".format(method))
+        if not callable(callback):
+            raise TypeError(
+                'a callable object was expected by {}(), got {!r}'.format(
+                    method, callback))
+
+
     def _call_soon(self, callback, args):
-        if (coroutines.iscoroutine(callback)
-        or coroutines.iscoroutinefunction(callback)):
-            raise TypeError("coroutines cannot be used with call_soon()")
-        self._check_closed()
         handle = events.Handle(callback, args, self)
         if handle._source_traceback:
             del handle._source_traceback[-1]
@@ -592,6 +599,9 @@
 
     def call_soon_threadsafe(self, callback, *args):
         """Like call_soon(), but thread-safe."""
+        self._check_closed()
+        if self._debug:
+            self._check_callback(callback, 'call_soon_threadsafe')
         handle = self._call_soon(callback, args)
         if handle._source_traceback:
             del handle._source_traceback[-1]
@@ -599,21 +609,9 @@
         return handle
 
     def run_in_executor(self, executor, func, *args):
-        if (coroutines.iscoroutine(func)
-        or coroutines.iscoroutinefunction(func)):
-            raise TypeError("coroutines cannot be used with run_in_executor()")
         self._check_closed()
-        if isinstance(func, events.Handle):
-            assert not args
-            assert not isinstance(func, events.TimerHandle)
-            warnings.warn(
-                "Passing Handle to loop.run_in_executor() is deprecated",
-                DeprecationWarning)
-            if func._cancelled:
-                f = self.create_future()
-                f.set_result(None)
-                return f
-            func, args = func._callback, func._args
+        if self._debug:
+            self._check_callback(func, 'run_in_executor')
         if executor is None:
             executor = self._default_executor
             if executor is None:
diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py
--- a/Lib/asyncio/events.py
+++ b/Lib/asyncio/events.py
@@ -82,7 +82,6 @@
                  '_source_traceback', '_repr', '__weakref__')
 
     def __init__(self, callback, args, loop):
-        assert not isinstance(callback, Handle), 'A Handle is not a callback'
         self._loop = loop
         self._callback = callback
         self._args = args
diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py
--- a/Lib/test/test_asyncio/test_base_events.py
+++ b/Lib/test/test_asyncio/test_base_events.py
@@ -235,6 +235,11 @@
         self.assertIsInstance(h, asyncio.Handle)
         self.assertIn(h, self.loop._ready)
 
+    def test_call_soon_non_callable(self):
+        self.loop.set_debug(True)
+        with self.assertRaisesRegex(TypeError, 'a callable object'):
+            self.loop.call_soon(1)
+
     def test_call_later(self):
         def cb():
             pass
@@ -341,47 +346,21 @@
         # check disabled if debug mode is disabled
         test_thread(self.loop, False, create_loop=True)
 
-    def test_run_once_in_executor_handle(self):
-        def cb():
-            pass
-
-        self.assertRaises(
-            AssertionError, self.loop.run_in_executor,
-            None, asyncio.Handle(cb, (), self.loop), ('',))
-        self.assertRaises(
-            AssertionError, self.loop.run_in_executor,
-            None, asyncio.TimerHandle(10, cb, (), self.loop))
-
-    def test_run_once_in_executor_cancelled(self):
-        def cb():
-            pass
-        h = asyncio.Handle(cb, (), self.loop)
-        h.cancel()
-
-        with self.assertWarnsRegex(DeprecationWarning, "Passing Handle"):
-            f = self.loop.run_in_executor(None, h)
-        self.assertIsInstance(f, asyncio.Future)
-        self.assertTrue(f.done())
-        self.assertIsNone(f.result())
-
     def test_run_once_in_executor_plain(self):
         def cb():
             pass
-        h = asyncio.Handle(cb, (), self.loop)
         f = asyncio.Future(loop=self.loop)
         executor = mock.Mock()
         executor.submit.return_value = f
 
         self.loop.set_default_executor(executor)
 
-        with self.assertWarnsRegex(DeprecationWarning, "Passing Handle"):
-            res = self.loop.run_in_executor(None, h)
+        res = self.loop.run_in_executor(None, cb)
         self.assertIs(f, res)
 
         executor = mock.Mock()
         executor.submit.return_value = f
-        with self.assertWarnsRegex(DeprecationWarning, "Passing Handle"):
-            res = self.loop.run_in_executor(executor, h)
+        res = self.loop.run_in_executor(executor, cb)
         self.assertIs(f, res)
         self.assertTrue(executor.submit.called)
 
@@ -1666,6 +1645,7 @@
         def simple_coroutine():
             pass
 
+        self.loop.set_debug(True)
         coro_func = simple_coroutine
         coro_obj = coro_func()
         self.addCleanup(coro_obj.close)
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -2249,13 +2249,6 @@
         h.cancel()
         self.assertTrue(h._cancelled)
 
-    def test_handle_from_handle(self):
-        def callback(*args):
-            return args
-        h1 = asyncio.Handle(callback, (), loop=self.loop)
-        self.assertRaises(
-            AssertionError, asyncio.Handle, h1, (), self.loop)
-
     def test_callback_with_exception(self):
         def callback():
             raise ValueError()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -20,6 +20,8 @@
   the garbage collector is invoked in other thread.  Based on patch by
   Sebastian Cufre.
 
+- Issue #28600: Optimize loop.call_soon.
+
 Documentation
 -------------
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov  3 18:36:14 2016
From: python-checkins at python.org (yury.selivanov)
Date: Thu, 03 Nov 2016 22:36:14 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogYXN5bmNpbzogU3lu?=
 =?utf-8?q?c_with_upstream?=
Message-ID: <20161103223613.17076.59241.9BE76DFA@psf.io>

https://hg.python.org/cpython/rev/c28c053c607a
changeset:   104896:c28c053c607a
branch:      3.5
parent:      104893:128ffe3c3eb9
user:        Yury Selivanov <yury at magic.io>
date:        Thu Nov 03 15:35:23 2016 -0700
summary:
  asyncio: Sync with upstream

files:
  Lib/asyncio/base_events.py |  3 +++
  1 files changed, 3 insertions(+), 0 deletions(-)


diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -348,6 +348,9 @@
         self._asyncgens.discard(agen)
         if not self.is_closed():
             self.create_task(agen.aclose())
+            # Wake up the loop if the finalizer was called from
+            # a different thread.
+            self._write_to_self()
 
     def _asyncgen_firstiter_hook(self, agen):
         if self._asyncgens_shutdown_called:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov  3 18:36:15 2016
From: python-checkins at python.org (yury.selivanov)
Date: Thu, 03 Nov 2016 22:36:15 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5_=28asyncio=29?=
Message-ID: <20161103223613.30814.29039.7D75AC85@psf.io>

https://hg.python.org/cpython/rev/5af19a28c76e
changeset:   104897:5af19a28c76e
branch:      3.6
parent:      104894:4f570a612aec
parent:      104896:c28c053c607a
user:        Yury Selivanov <yury at magic.io>
date:        Thu Nov 03 15:35:41 2016 -0700
summary:
  Merge 3.5 (asyncio)

files:



-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov  3 18:36:15 2016
From: python-checkins at python.org (yury.selivanov)
Date: Thu, 03 Nov 2016 22:36:15 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_3=2E6_=28asyncio=29?=
Message-ID: <20161103223613.31751.15272.D56279FA@psf.io>

https://hg.python.org/cpython/rev/b143ca90a1af
changeset:   104898:b143ca90a1af
parent:      104895:46c3eede41a6
parent:      104897:5af19a28c76e
user:        Yury Selivanov <yury at magic.io>
date:        Thu Nov 03 15:36:02 2016 -0700
summary:
  Merge 3.6 (asyncio)

files:



-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov  3 19:21:16 2016
From: python-checkins at python.org (brett.cannon)
Date: Thu, 03 Nov 2016 23:21:16 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NjA1?=
 =?utf-8?q?=3A_Fix_the_help_and_What=27s_New_entry_for_--with-optimization?=
 =?utf-8?q?s=2E?=
Message-ID: <20161103232116.31709.25106.7E7A6B74@psf.io>

https://hg.python.org/cpython/rev/1f750fff788e
changeset:   104899:1f750fff788e
branch:      3.6
parent:      104897:5af19a28c76e
user:        Brett Cannon <brett at python.org>
date:        Thu Nov 03 16:20:00 2016 -0700
summary:
  Issue #28605: Fix the help and What's New entry for --with-optimizations.

files:
  Doc/whatsnew/3.6.rst |  2 +-
  configure            |  6 +++---
  configure.ac         |  4 ++--
  3 files changed, 6 insertions(+), 6 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -1176,7 +1176,7 @@
   with only about 16 tests failures. See the Android meta-issue :issue:`26865`.
 
 * The ``--with-optimizations`` configure flag has been added. Turning it on
-  will activate LTO and PGO build support (when available).
+  will activate expensive optimizations like PGO.
   (Original patch by Alecsandru Patrascu of Intel in :issue:`26539`.)
 
 * New :c:func:`Py_FinalizeEx` API which indicates if flushing buffered data
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -1499,8 +1499,8 @@
                           compiler
   --with-suffix=.exe      set executable suffix
   --with-pydebug          build with Py_DEBUG defined
-  --with-optimizations    Enable expensive optimizations (PGO, maybe LTO,
-                          etc). Disabled by default.
+  --with-optimizations    Enable expensive optimizations (PGO, etc). Disabled
+                          by default.
   --with-lto              Enable Link Time Optimization in PGO builds.
                           Disabled by default.
   --with-hash-algorithm=[fnv|siphash24]
@@ -6548,7 +6548,7 @@
 if test "$Py_OPT" = 'true' ; then
   # Intentionally not forcing Py_LTO='true' here.  Too many toolchains do not
   # compile working code using it and both test_distutils and test_gdb are
-  # broken when you do managed to get a toolchain that works with it.  People
+  # broken when you do manage to get a toolchain that works with it.  People
   # who want LTO need to use --with-lto themselves.
   DEF_MAKE_ALL_RULE="profile-opt"
   REQUIRE_PGO="yes"
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -1282,7 +1282,7 @@
 AC_SUBST(DEF_MAKE_RULE)
 Py_OPT='false'
 AC_MSG_CHECKING(for --with-optimizations)
-AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable expensive optimizations (PGO, maybe LTO, etc).  Disabled by default.]),
+AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable expensive optimizations (PGO, etc).  Disabled by default.]),
 [
 if test "$withval" != no
 then
@@ -1296,7 +1296,7 @@
 if test "$Py_OPT" = 'true' ; then
   # Intentionally not forcing Py_LTO='true' here.  Too many toolchains do not
   # compile working code using it and both test_distutils and test_gdb are
-  # broken when you do managed to get a toolchain that works with it.  People
+  # broken when you do manage to get a toolchain that works with it.  People
   # who want LTO need to use --with-lto themselves.
   DEF_MAKE_ALL_RULE="profile-opt"
   REQUIRE_PGO="yes"

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov  3 19:21:17 2016
From: python-checkins at python.org (brett.cannon)
Date: Thu, 03 Nov 2016 23:21:17 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_for_issue_=2328605?=
Message-ID: <20161103232116.17403.88008.D58BACF2@psf.io>

https://hg.python.org/cpython/rev/4000de2dcd24
changeset:   104900:4000de2dcd24
parent:      104898:b143ca90a1af
parent:      104899:1f750fff788e
user:        Brett Cannon <brett at python.org>
date:        Thu Nov 03 16:21:11 2016 -0700
summary:
  Merge for issue #28605

files:
  Doc/whatsnew/3.6.rst |   2 +-
  configure            |  20 ++++----------------
  configure.ac         |   4 ++--
  3 files changed, 7 insertions(+), 19 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -1177,7 +1177,7 @@
   with only about 16 tests failures. See the Android meta-issue :issue:`26865`.
 
 * The ``--with-optimizations`` configure flag has been added. Turning it on
-  will activate LTO and PGO build support (when available).
+  will activate expensive optimizations like PGO.
   (Original patch by Alecsandru Patrascu of Intel in :issue:`26539`.)
 
 * New :c:func:`Py_FinalizeEx` API which indicates if flushing buffered data
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -784,7 +784,6 @@
 docdir
 oldincludedir
 includedir
-runstatedir
 localstatedir
 sharedstatedir
 sysconfdir
@@ -895,7 +894,6 @@
 sysconfdir='${prefix}/etc'
 sharedstatedir='${prefix}/com'
 localstatedir='${prefix}/var'
-runstatedir='${localstatedir}/run'
 includedir='${prefix}/include'
 oldincludedir='/usr/include'
 docdir='${datarootdir}/doc/${PACKAGE_TARNAME}'
@@ -1148,15 +1146,6 @@
   | -silent | --silent | --silen | --sile | --sil)
     silent=yes ;;
 
-  -runstatedir | --runstatedir | --runstatedi | --runstated \
-  | --runstate | --runstat | --runsta | --runst | --runs \
-  | --run | --ru | --r)
-    ac_prev=runstatedir ;;
-  -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \
-  | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \
-  | --run=* | --ru=* | --r=*)
-    runstatedir=$ac_optarg ;;
-
   -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb)
     ac_prev=sbindir ;;
   -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \
@@ -1294,7 +1283,7 @@
 for ac_var in	exec_prefix prefix bindir sbindir libexecdir datarootdir \
 		datadir sysconfdir sharedstatedir localstatedir includedir \
 		oldincludedir docdir infodir htmldir dvidir pdfdir psdir \
-		libdir localedir mandir runstatedir
+		libdir localedir mandir
 do
   eval ac_val=\$$ac_var
   # Remove trailing slashes.
@@ -1447,7 +1436,6 @@
   --sysconfdir=DIR        read-only single-machine data [PREFIX/etc]
   --sharedstatedir=DIR    modifiable architecture-independent data [PREFIX/com]
   --localstatedir=DIR     modifiable single-machine data [PREFIX/var]
-  --runstatedir=DIR       modifiable per-process data [LOCALSTATEDIR/run]
   --libdir=DIR            object code libraries [EPREFIX/lib]
   --includedir=DIR        C header files [PREFIX/include]
   --oldincludedir=DIR     C header files for non-gcc [/usr/include]
@@ -1511,8 +1499,8 @@
                           compiler
   --with-suffix=.exe      set executable suffix
   --with-pydebug          build with Py_DEBUG defined
-  --with-optimizations    Enable expensive optimizations (PGO, maybe LTO,
-                          etc). Disabled by default.
+  --with-optimizations    Enable expensive optimizations (PGO, etc). Disabled
+                          by default.
   --with-lto              Enable Link Time Optimization in PGO builds.
                           Disabled by default.
   --with-hash-algorithm=[fnv|siphash24]
@@ -6560,7 +6548,7 @@
 if test "$Py_OPT" = 'true' ; then
   # Intentionally not forcing Py_LTO='true' here.  Too many toolchains do not
   # compile working code using it and both test_distutils and test_gdb are
-  # broken when you do managed to get a toolchain that works with it.  People
+  # broken when you do manage to get a toolchain that works with it.  People
   # who want LTO need to use --with-lto themselves.
   DEF_MAKE_ALL_RULE="profile-opt"
   REQUIRE_PGO="yes"
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -1282,7 +1282,7 @@
 AC_SUBST(DEF_MAKE_RULE)
 Py_OPT='false'
 AC_MSG_CHECKING(for --with-optimizations)
-AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable expensive optimizations (PGO, maybe LTO, etc).  Disabled by default.]),
+AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable expensive optimizations (PGO, etc).  Disabled by default.]),
 [
 if test "$withval" != no
 then
@@ -1296,7 +1296,7 @@
 if test "$Py_OPT" = 'true' ; then
   # Intentionally not forcing Py_LTO='true' here.  Too many toolchains do not
   # compile working code using it and both test_distutils and test_gdb are
-  # broken when you do managed to get a toolchain that works with it.  People
+  # broken when you do manage to get a toolchain that works with it.  People
   # who want LTO need to use --with-lto themselves.
   DEF_MAKE_ALL_RULE="profile-opt"
   REQUIRE_PGO="yes"

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 03:35:39 2016
From: python-checkins at python.org (inada.naoki)
Date: Fri, 04 Nov 2016 07:35:39 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328088=3A_Document_Transport=2Eset=5Fprotocol_an?=
 =?utf-8?q?d_get=5Fprotocol=2E?=
Message-ID: <20161104073539.25593.87291.4853DE44@psf.io>

https://hg.python.org/cpython/rev/a0299574a733
changeset:   104903:a0299574a733
parent:      104900:4000de2dcd24
parent:      104902:a5e52b7be71f
user:        INADA Naoki <songofacandy at gmail.com>
date:        Fri Nov 04 16:35:31 2016 +0900
summary:
  Issue #28088: Document Transport.set_protocol and get_protocol.

Patch by Mariatta Wijaya.

files:
  Doc/library/asyncio-protocol.rst |  13 +++++++++++++
  1 files changed, 13 insertions(+), 0 deletions(-)


diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst
--- a/Doc/library/asyncio-protocol.rst
+++ b/Doc/library/asyncio-protocol.rst
@@ -87,6 +87,19 @@
 
         - ``'subprocess'``: :class:`subprocess.Popen` instance
 
+   .. method:: set_protocol(self, protocol)
+
+      Set a new protocol.  Switching protocol should only be done when both
+      protocols are documented to support the switch.
+
+      .. versionadded:: 3.6.0
+
+   .. method:: get_protocol(self)
+
+      Return the current protocol.
+
+      .. versionadded:: 3.6.0
+
    .. versionchanged:: 3.5.1
       ``'ssl_object'`` info was added to SSL sockets.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 03:35:39 2016
From: python-checkins at python.org (inada.naoki)
Date: Fri, 04 Nov 2016 07:35:39 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328088=3A_Document_Transport=2Eset=5Fprotocol_and_get?=
 =?utf-8?q?=5Fprotocol=2E?=
Message-ID: <20161104073539.45129.43252.F067B211@psf.io>

https://hg.python.org/cpython/rev/a5e52b7be71f
changeset:   104902:a5e52b7be71f
branch:      3.6
parent:      104899:1f750fff788e
parent:      104901:3f5af4a25995
user:        INADA Naoki <songofacandy at gmail.com>
date:        Fri Nov 04 16:34:46 2016 +0900
summary:
  Issue #28088: Document Transport.set_protocol and get_protocol.

Patch by Mariatta Wijaya.

files:
  Doc/library/asyncio-protocol.rst |  13 +++++++++++++
  1 files changed, 13 insertions(+), 0 deletions(-)


diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst
--- a/Doc/library/asyncio-protocol.rst
+++ b/Doc/library/asyncio-protocol.rst
@@ -87,6 +87,19 @@
 
         - ``'subprocess'``: :class:`subprocess.Popen` instance
 
+   .. method:: set_protocol(self, protocol)
+
+      Set a new protocol.  Switching protocol should only be done when both
+      protocols are documented to support the switch.
+
+      .. versionadded:: 3.6.0
+
+   .. method:: get_protocol(self)
+
+      Return the current protocol.
+
+      .. versionadded:: 3.6.0
+
    .. versionchanged:: 3.5.1
       ``'ssl_object'`` info was added to SSL sockets.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 03:35:43 2016
From: python-checkins at python.org (inada.naoki)
Date: Fri, 04 Nov 2016 07:35:43 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4MDg4?=
 =?utf-8?q?=3A_Document_Transport=2Eset=5Fprotocol_and_get=5Fprotocol?=
Message-ID: <20161104073538.22391.76393.BBA59518@psf.io>

https://hg.python.org/cpython/rev/3f5af4a25995
changeset:   104901:3f5af4a25995
branch:      3.5
parent:      104896:c28c053c607a
user:        INADA Naoki <songofacandy at gmail.com>
date:        Fri Nov 04 16:33:47 2016 +0900
summary:
  Issue #28088: Document Transport.set_protocol and get_protocol

files:
  Doc/library/asyncio-protocol.rst |  13 +++++++++++++
  1 files changed, 13 insertions(+), 0 deletions(-)


diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst
--- a/Doc/library/asyncio-protocol.rst
+++ b/Doc/library/asyncio-protocol.rst
@@ -87,6 +87,19 @@
 
         - ``'subprocess'``: :class:`subprocess.Popen` instance
 
+   .. method:: set_protocol(self, protocol)
+
+      Set a new protocol.  Switching protocol should only be done when both
+      protocols are documented to support the switch.
+
+      .. versionadded:: 3.6.0
+
+   .. method:: get_protocol(self)
+
+      Return the current protocol.
+
+      .. versionadded:: 3.6.0
+
    .. versionchanged:: 3.5.1
       ``'ssl_object'`` info was added to SSL sockets.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 03:59:23 2016
From: python-checkins at python.org (inada.naoki)
Date: Fri, 04 Nov 2016 07:59:23 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328580=3A_Optimize?=
 =?utf-8?q?_iterating_split_table_values=2E?=
Message-ID: <20161104075923.16893.70595.80564C04@psf.io>

https://hg.python.org/cpython/rev/3904194d06e6
changeset:   104904:3904194d06e6
user:        INADA Naoki <songofacandy at gmail.com>
date:        Fri Nov 04 16:59:10 2016 +0900
summary:
  Issue #28580: Optimize iterating split table values.

Patch by Xiang Zhang.

files:
  Misc/NEWS            |   3 +
  Objects/dictobject.c |  62 ++++++++++++-------------------
  2 files changed, 28 insertions(+), 37 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #28580: Optimize iterating split table values.
+  Patch by Xiang Zhang.
+
 - Issue #28583: PyDict_SetDefault didn't combine split table when needed.
   Patch by Xiang Zhang.
 
diff --git a/Objects/dictobject.c b/Objects/dictobject.c
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -1700,7 +1700,7 @@
 _PyDict_Next(PyObject *op, Py_ssize_t *ppos, PyObject **pkey,
              PyObject **pvalue, Py_hash_t *phash)
 {
-    Py_ssize_t i, n;
+    Py_ssize_t i;
     PyDictObject *mp;
     PyDictKeyEntry *entry_ptr;
     PyObject *value;
@@ -1709,21 +1709,18 @@
         return 0;
     mp = (PyDictObject *)op;
     i = *ppos;
-    n = mp->ma_keys->dk_nentries;
-    if ((size_t)i >= (size_t)n)
-        return 0;
     if (mp->ma_values) {
-        PyObject **value_ptr = &mp->ma_values[i];
-        while (i < n && *value_ptr == NULL) {
-            value_ptr++;
-            i++;
-        }
-        if (i >= n)
+        if (i < 0 || i >= mp->ma_used)
             return 0;
+        /* values of split table is always dense */
         entry_ptr = &DK_ENTRIES(mp->ma_keys)[i];
-        value = *value_ptr;
+        value = mp->ma_values[i];
+        assert(value != NULL);
     }
     else {
+        Py_ssize_t n = mp->ma_keys->dk_nentries;
+        if (i < 0 || i >= n)
+            return 0;
         entry_ptr = &DK_ENTRIES(mp->ma_keys)[i];
         while (i < n && entry_ptr->me_value == NULL) {
             entry_ptr++;
@@ -3380,7 +3377,7 @@
 dictiter_iternextkey(dictiterobject *di)
 {
     PyObject *key;
-    Py_ssize_t i, n;
+    Py_ssize_t i;
     PyDictKeysObject *k;
     PyDictObject *d = di->di_dict;
 
@@ -3397,18 +3394,15 @@
 
     i = di->di_pos;
     k = d->ma_keys;
-    n = k->dk_nentries;
+    assert(i >= 0);
     if (d->ma_values) {
-        PyObject **value_ptr = &d->ma_values[i];
-        while (i < n && *value_ptr == NULL) {
-            value_ptr++;
-            i++;
-        }
-        if (i >= n)
+        if (i >= d->ma_used)
             goto fail;
         key = DK_ENTRIES(k)[i].me_key;
+        assert(d->ma_values[i] != NULL);
     }
     else {
+        Py_ssize_t n = k->dk_nentries;
         PyDictKeyEntry *entry_ptr = &DK_ENTRIES(k)[i];
         while (i < n && entry_ptr->me_value == NULL) {
             entry_ptr++;
@@ -3466,7 +3460,7 @@
 dictiter_iternextvalue(dictiterobject *di)
 {
     PyObject *value;
-    Py_ssize_t i, n;
+    Py_ssize_t i;
     PyDictObject *d = di->di_dict;
 
     if (d == NULL)
@@ -3481,18 +3475,15 @@
     }
 
     i = di->di_pos;
-    n = d->ma_keys->dk_nentries;
+    assert(i >= 0);
     if (d->ma_values) {
-        PyObject **value_ptr = &d->ma_values[i];
-        while (i < n && *value_ptr == NULL) {
-            value_ptr++;
-            i++;
-        }
-        if (i >= n)
+        if (i >= d->ma_used)
             goto fail;
-        value = *value_ptr;
+        value = d->ma_values[i];
+        assert(value != NULL);
     }
     else {
+        Py_ssize_t n = d->ma_keys->dk_nentries;
         PyDictKeyEntry *entry_ptr = &DK_ENTRIES(d->ma_keys)[i];
         while (i < n && entry_ptr->me_value == NULL) {
             entry_ptr++;
@@ -3550,7 +3541,7 @@
 dictiter_iternextitem(dictiterobject *di)
 {
     PyObject *key, *value, *result = di->di_result;
-    Py_ssize_t i, n;
+    Py_ssize_t i;
     PyDictObject *d = di->di_dict;
 
     if (d == NULL)
@@ -3565,19 +3556,16 @@
     }
 
     i = di->di_pos;
-    n = d->ma_keys->dk_nentries;
+    assert(i >= 0);
     if (d->ma_values) {
-        PyObject **value_ptr = &d->ma_values[i];
-        while (i < n && *value_ptr == NULL) {
-            value_ptr++;
-            i++;
-        }
-        if (i >= n)
+        if (i >= d->ma_used)
             goto fail;
         key = DK_ENTRIES(d->ma_keys)[i].me_key;
-        value = *value_ptr;
+        value = d->ma_values[i];
+        assert(value != NULL);
     }
     else {
+        Py_ssize_t n = d->ma_keys->dk_nentries;
         PyDictKeyEntry *entry_ptr = &DK_ENTRIES(d->ma_keys)[i];
         while (i < n && entry_ptr->me_value == NULL) {
             entry_ptr++;

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Fri Nov  4 11:07:24 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Fri, 4 Nov 2016 15:07:24 +0000
Subject: [Python-checkins] NEUTRAL Benchmark Results for Python Default
 2016-11-04
Message-ID: <0616571a-959a-42d6-ac9d-57e47bbab832@irsmsx151.ger.corp.intel.com>

Results for project Python default, build date 2016-11-04 03:01:50 +0000
commit:		4000de2dcd24
previous commit:	b74897b69184
revision date:	2016-11-03 23:21:11 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.21%         -0.06%          4.59%            15.77%
:-)             pybench      0.10%          0.02%          5.42%             4.25%
:-(            regex_v8      3.78%         -0.47%         -2.47%             3.04%
:-)               nbody      0.21%         -0.24%          2.85%             2.63%
:-(        json_dump_v2      0.20%          0.70%        -10.95%            15.57%
:-|      normal_startup      1.14%          0.66%         -0.59%             6.24%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-default-2016-11-04/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Fri Nov  4 11:06:53 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Fri, 4 Nov 2016 15:06:53 +0000
Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7
 2016-11-04
Message-ID: <6e0c76cf-5727-497d-ab88-0a4dac838470@irsmsx151.ger.corp.intel.com>

Results for project Python 2.7, build date 2016-11-04 03:48:28 +0000
commit:		91f024fc9b3a
previous commit:	c0a0abce38f2
revision date:	2016-11-03 13:36:50 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.17%         -0.89%          4.75%             6.09%
:-)             pybench      0.15%         -0.11%          5.70%             2.91%
:-(            regex_v8      0.68%         -0.09%         -2.37%            11.06%
:-)               nbody      0.07%          0.03%          7.96%             0.42%
:-)        json_dump_v2      0.27%         -0.08%          2.57%             9.69%
:-|      normal_startup      0.56%          0.26%         -0.51%             1.87%
:-)             ssbench      0.25%          0.06%          2.55%             2.06%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-11-04/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Fri Nov  4 14:30:48 2016
From: python-checkins at python.org (yury.selivanov)
Date: Fri, 04 Nov 2016 18:30:48 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjEz?=
 =?utf-8?q?=3A_Fix_get=5Fevent=5Floop=28=29_to_return_the_current_loop?=
Message-ID: <20161104183048.22950.39119.AFE7BC6F@psf.io>

https://hg.python.org/cpython/rev/aa37f3859462
changeset:   104905:aa37f3859462
branch:      3.5
parent:      104901:3f5af4a25995
user:        Yury Selivanov <yury at magic.io>
date:        Fri Nov 04 14:29:28 2016 -0400
summary:
  Issue #28613: Fix get_event_loop() to return the current loop

when called from coroutines or callbacks.

files:
  Lib/asyncio/base_events.py                    |   7 +-
  Lib/asyncio/events.py                         |  36 +++++++++-
  Lib/asyncio/test_utils.py                     |   6 +
  Lib/test/test_asyncio/test_base_events.py     |  20 +++++
  Lib/test/test_asyncio/test_events.py          |  23 ++++++
  Lib/test/test_asyncio/test_futures.py         |   3 +
  Lib/test/test_asyncio/test_locks.py           |   4 +
  Lib/test/test_asyncio/test_pep492.py          |   1 +
  Lib/test/test_asyncio/test_proactor_events.py |   3 +
  Lib/test/test_asyncio/test_queues.py          |   1 +
  Lib/test/test_asyncio/test_selector_events.py |   5 +
  Lib/test/test_asyncio/test_sslproto.py        |   1 +
  Lib/test/test_asyncio/test_streams.py         |   1 +
  Lib/test/test_asyncio/test_subprocess.py      |   3 +
  Lib/test/test_asyncio/test_tasks.py           |   5 +
  Lib/test/test_asyncio/test_unix_events.py     |   5 +
  Lib/test/test_asyncio/test_windows_events.py  |   1 +
  Misc/NEWS                                     |   3 +
  18 files changed, 126 insertions(+), 2 deletions(-)


diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -393,7 +393,10 @@
         """Run until stop() is called."""
         self._check_closed()
         if self.is_running():
-            raise RuntimeError('Event loop is running.')
+            raise RuntimeError('This event loop is already running')
+        if events._get_running_loop() is not None:
+            raise RuntimeError(
+                'Cannot run the event loop while another loop is running')
         self._set_coroutine_wrapper(self._debug)
         self._thread_id = threading.get_ident()
         if self._asyncgens is not None:
@@ -401,6 +404,7 @@
             sys.set_asyncgen_hooks(firstiter=self._asyncgen_firstiter_hook,
                                    finalizer=self._asyncgen_finalizer_hook)
         try:
+            events._set_running_loop(self)
             while True:
                 self._run_once()
                 if self._stopping:
@@ -408,6 +412,7 @@
         finally:
             self._stopping = False
             self._thread_id = None
+            events._set_running_loop(None)
             self._set_coroutine_wrapper(False)
             if self._asyncgens is not None:
                 sys.set_asyncgen_hooks(*old_agen_hooks)
diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py
--- a/Lib/asyncio/events.py
+++ b/Lib/asyncio/events.py
@@ -607,6 +607,30 @@
 _lock = threading.Lock()
 
 
+# A TLS for the running event loop, used by _get_running_loop.
+class _RunningLoop(threading.local):
+    _loop = None
+_running_loop = _RunningLoop()
+
+
+def _get_running_loop():
+    """Return the running event loop or None.
+
+    This is a low-level function intended to be used by event loops.
+    This function is thread-specific.
+    """
+    return _running_loop._loop
+
+
+def _set_running_loop(loop):
+    """Set the running event loop.
+
+    This is a low-level function intended to be used by event loops.
+    This function is thread-specific.
+    """
+    _running_loop._loop = loop
+
+
 def _init_event_loop_policy():
     global _event_loop_policy
     with _lock:
@@ -632,7 +656,17 @@
 
 
 def get_event_loop():
-    """Equivalent to calling get_event_loop_policy().get_event_loop()."""
+    """Return an asyncio event loop.
+
+    When called from a coroutine or a callback (e.g. scheduled with call_soon
+    or similar API), this function will always return the running event loop.
+
+    If there is no running event loop set, the function will return
+    the result of `get_event_loop_policy().get_event_loop()` call.
+    """
+    current_loop = _get_running_loop()
+    if current_loop is not None:
+        return current_loop
     return get_event_loop_policy().get_event_loop()
 
 
diff --git a/Lib/asyncio/test_utils.py b/Lib/asyncio/test_utils.py
--- a/Lib/asyncio/test_utils.py
+++ b/Lib/asyncio/test_utils.py
@@ -449,7 +449,13 @@
         self.set_event_loop(loop)
         return loop
 
+    def setUp(self):
+        self._get_running_loop = events._get_running_loop
+        events._get_running_loop = lambda: None
+
     def tearDown(self):
+        events._get_running_loop = self._get_running_loop
+
         events.set_event_loop(None)
 
         # Detect CPython bug #23353: ensure that yield/yield-from is not used
diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py
--- a/Lib/test/test_asyncio/test_base_events.py
+++ b/Lib/test/test_asyncio/test_base_events.py
@@ -154,6 +154,7 @@
 class BaseEventLoopTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = base_events.BaseEventLoop()
         self.loop._selector = mock.Mock()
         self.loop._selector.select.return_value = ()
@@ -976,6 +977,7 @@
 class BaseEventLoopWithSelectorTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.new_event_loop()
         self.set_event_loop(self.loop)
 
@@ -1692,5 +1694,23 @@
                          "took .* seconds$")
 
 
+class RunningLoopTests(unittest.TestCase):
+
+    def test_running_loop_within_a_loop(self):
+        @asyncio.coroutine
+        def runner(loop):
+            loop.run_forever()
+
+        loop = asyncio.new_event_loop()
+        outer_loop = asyncio.new_event_loop()
+        try:
+            with self.assertRaisesRegex(RuntimeError,
+                                        'while another loop is running'):
+                outer_loop.run_until_complete(runner(loop))
+        finally:
+            loop.close()
+            outer_loop.close()
+
+
 if __name__ == '__main__':
     unittest.main()
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -2233,6 +2233,7 @@
 class HandleTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = mock.Mock()
         self.loop.get_debug.return_value = True
 
@@ -2411,6 +2412,7 @@
 class TimerTests(unittest.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = mock.Mock()
 
     def test_hash(self):
@@ -2719,6 +2721,27 @@
         self.assertIs(policy, asyncio.get_event_loop_policy())
         self.assertIsNot(policy, old_policy)
 
+    def test_get_event_loop_returns_running_loop(self):
+        class Policy(asyncio.DefaultEventLoopPolicy):
+            def get_event_loop(self):
+                raise NotImplementedError
+
+        loop = None
+
+        old_policy = asyncio.get_event_loop_policy()
+        try:
+            asyncio.set_event_loop_policy(Policy())
+            loop = asyncio.new_event_loop()
+
+            async def func():
+                self.assertIs(asyncio.get_event_loop(), loop)
+
+            loop.run_until_complete(func())
+        finally:
+            asyncio.set_event_loop_policy(old_policy)
+            if loop is not None:
+                loop.close()
+
 
 if __name__ == '__main__':
     unittest.main()
diff --git a/Lib/test/test_asyncio/test_futures.py b/Lib/test/test_asyncio/test_futures.py
--- a/Lib/test/test_asyncio/test_futures.py
+++ b/Lib/test/test_asyncio/test_futures.py
@@ -79,6 +79,7 @@
 class DuckTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.addCleanup(self.loop.close)
 
@@ -96,6 +97,7 @@
 class FutureTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.addCleanup(self.loop.close)
 
@@ -468,6 +470,7 @@
 class FutureDoneCallbackTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def run_briefly(self):
diff --git a/Lib/test/test_asyncio/test_locks.py b/Lib/test/test_asyncio/test_locks.py
--- a/Lib/test/test_asyncio/test_locks.py
+++ b/Lib/test/test_asyncio/test_locks.py
@@ -19,6 +19,7 @@
 class LockTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def test_ctor_loop(self):
@@ -235,6 +236,7 @@
 class EventTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def test_ctor_loop(self):
@@ -364,6 +366,7 @@
 class ConditionTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def test_ctor_loop(self):
@@ -699,6 +702,7 @@
 class SemaphoreTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def test_ctor_loop(self):
diff --git a/Lib/test/test_asyncio/test_pep492.py b/Lib/test/test_asyncio/test_pep492.py
--- a/Lib/test/test_asyncio/test_pep492.py
+++ b/Lib/test/test_asyncio/test_pep492.py
@@ -17,6 +17,7 @@
 class BaseTest(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.BaseEventLoop()
         self.loop._process_events = mock.Mock()
         self.loop._selector = mock.Mock()
diff --git a/Lib/test/test_asyncio/test_proactor_events.py b/Lib/test/test_asyncio/test_proactor_events.py
--- a/Lib/test/test_asyncio/test_proactor_events.py
+++ b/Lib/test/test_asyncio/test_proactor_events.py
@@ -24,6 +24,7 @@
 class ProactorSocketTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.addCleanup(self.loop.close)
         self.proactor = mock.Mock()
@@ -436,6 +437,8 @@
 class BaseProactorEventLoopTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
+
         self.sock = test_utils.mock_nonblocking_socket()
         self.proactor = mock.Mock()
 
diff --git a/Lib/test/test_asyncio/test_queues.py b/Lib/test/test_asyncio/test_queues.py
--- a/Lib/test/test_asyncio/test_queues.py
+++ b/Lib/test/test_asyncio/test_queues.py
@@ -10,6 +10,7 @@
 class _QueueTestBase(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
 
diff --git a/Lib/test/test_asyncio/test_selector_events.py b/Lib/test/test_asyncio/test_selector_events.py
--- a/Lib/test/test_asyncio/test_selector_events.py
+++ b/Lib/test/test_asyncio/test_selector_events.py
@@ -51,6 +51,7 @@
 class BaseSelectorEventLoopTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.selector = mock.Mock()
         self.selector.select.return_value = []
         self.loop = TestBaseSelectorEventLoop(self.selector)
@@ -698,6 +699,7 @@
 class SelectorTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.Protocol)
         self.sock = mock.Mock(socket.socket)
@@ -793,6 +795,7 @@
 class SelectorSocketTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.Protocol)
         self.sock = mock.Mock(socket.socket)
@@ -1141,6 +1144,7 @@
 class SelectorSslTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.Protocol)
         self.sock = mock.Mock(socket.socket)
@@ -1501,6 +1505,7 @@
 class SelectorDatagramTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.DatagramProtocol)
         self.sock = mock.Mock(spec_set=socket.socket)
diff --git a/Lib/test/test_asyncio/test_sslproto.py b/Lib/test/test_asyncio/test_sslproto.py
--- a/Lib/test/test_asyncio/test_sslproto.py
+++ b/Lib/test/test_asyncio/test_sslproto.py
@@ -18,6 +18,7 @@
 class SslProtoHandshakeTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.new_event_loop()
         self.set_event_loop(self.loop)
 
diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py
--- a/Lib/test/test_asyncio/test_streams.py
+++ b/Lib/test/test_asyncio/test_streams.py
@@ -22,6 +22,7 @@
     DATA = b'line1\nline2\nline3\n'
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.new_event_loop()
         self.set_event_loop(self.loop)
 
diff --git a/Lib/test/test_asyncio/test_subprocess.py b/Lib/test/test_asyncio/test_subprocess.py
--- a/Lib/test/test_asyncio/test_subprocess.py
+++ b/Lib/test/test_asyncio/test_subprocess.py
@@ -35,6 +35,7 @@
 
 class SubprocessTransportTests(test_utils.TestCase):
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.set_event_loop(self.loop)
 
@@ -466,6 +467,7 @@
         Watcher = None
 
         def setUp(self):
+            super().setUp()
             policy = asyncio.get_event_loop_policy()
             self.loop = policy.new_event_loop()
             self.set_event_loop(self.loop)
@@ -490,6 +492,7 @@
     class SubprocessProactorTests(SubprocessMixin, test_utils.TestCase):
 
         def setUp(self):
+            super().setUp()
             self.loop = asyncio.ProactorEventLoop()
             self.set_event_loop(self.loop)
 
diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py
--- a/Lib/test/test_asyncio/test_tasks.py
+++ b/Lib/test/test_asyncio/test_tasks.py
@@ -75,6 +75,7 @@
 class TaskTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def test_other_loop_future(self):
@@ -1933,6 +1934,7 @@
 class GatherTestsBase:
 
     def setUp(self):
+        super().setUp()
         self.one_loop = self.new_test_loop()
         self.other_loop = self.new_test_loop()
         self.set_event_loop(self.one_loop, cleanup=False)
@@ -2216,6 +2218,7 @@
     """Test case for asyncio.run_coroutine_threadsafe."""
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.new_event_loop()
         self.set_event_loop(self.loop) # Will cleanup properly
 
@@ -2306,12 +2309,14 @@
 
 class SleepTests(test_utils.TestCase):
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.new_event_loop()
         asyncio.set_event_loop(None)
 
     def tearDown(self):
         self.loop.close()
         self.loop = None
+        super().tearDown()
 
     def test_sleep_zero(self):
         result = 0
diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py
--- a/Lib/test/test_asyncio/test_unix_events.py
+++ b/Lib/test/test_asyncio/test_unix_events.py
@@ -40,6 +40,7 @@
 class SelectorEventLoopSignalTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.SelectorEventLoop()
         self.set_event_loop(self.loop)
 
@@ -234,6 +235,7 @@
 class SelectorEventLoopUnixSocketTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.SelectorEventLoop()
         self.set_event_loop(self.loop)
 
@@ -338,6 +340,7 @@
 class UnixReadPipeTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.Protocol)
         self.pipe = mock.Mock(spec_set=io.RawIOBase)
@@ -487,6 +490,7 @@
 class UnixWritePipeTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.BaseProtocol)
         self.pipe = mock.Mock(spec_set=io.RawIOBase)
@@ -805,6 +809,7 @@
     ignore_warnings = mock.patch.object(log.logger, "warning")
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.running = False
         self.zombies = {}
diff --git a/Lib/test/test_asyncio/test_windows_events.py b/Lib/test/test_asyncio/test_windows_events.py
--- a/Lib/test/test_asyncio/test_windows_events.py
+++ b/Lib/test/test_asyncio/test_windows_events.py
@@ -31,6 +31,7 @@
 class ProactorTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.ProactorEventLoop()
         self.set_event_loop(self.loop)
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -443,6 +443,9 @@
 
 - Issue #28600: Optimize loop.call_soon().
 
+- Issue #28613: Fix get_event_loop() return the current loop if 
+  called from coroutines/callbacks.
+
 IDLE
 ----
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 14:30:49 2016
From: python-checkins at python.org (yury.selivanov)
Date: Fri, 04 Nov 2016 18:30:49 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5_=28issue_=2328613=29?=
Message-ID: <20161104183049.62824.59349.FF44EEDD@psf.io>

https://hg.python.org/cpython/rev/1473b9a17a91
changeset:   104906:1473b9a17a91
branch:      3.6
parent:      104902:a5e52b7be71f
parent:      104905:aa37f3859462
user:        Yury Selivanov <yury at magic.io>
date:        Fri Nov 04 14:30:11 2016 -0400
summary:
  Merge 3.5 (issue #28613)

files:
  Lib/asyncio/base_events.py                    |   7 +-
  Lib/asyncio/events.py                         |  36 +++++++++-
  Lib/asyncio/test_utils.py                     |   6 +
  Lib/test/test_asyncio/test_base_events.py     |  20 +++++
  Lib/test/test_asyncio/test_events.py          |  23 ++++++
  Lib/test/test_asyncio/test_futures.py         |   3 +
  Lib/test/test_asyncio/test_locks.py           |   4 +
  Lib/test/test_asyncio/test_pep492.py          |   1 +
  Lib/test/test_asyncio/test_proactor_events.py |   3 +
  Lib/test/test_asyncio/test_queues.py          |   1 +
  Lib/test/test_asyncio/test_selector_events.py |   5 +
  Lib/test/test_asyncio/test_sslproto.py        |   1 +
  Lib/test/test_asyncio/test_streams.py         |   1 +
  Lib/test/test_asyncio/test_subprocess.py      |   3 +
  Lib/test/test_asyncio/test_tasks.py           |   5 +
  Lib/test/test_asyncio/test_unix_events.py     |   5 +
  Lib/test/test_asyncio/test_windows_events.py  |   1 +
  Misc/NEWS                                     |   3 +
  18 files changed, 126 insertions(+), 2 deletions(-)


diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -393,7 +393,10 @@
         """Run until stop() is called."""
         self._check_closed()
         if self.is_running():
-            raise RuntimeError('Event loop is running.')
+            raise RuntimeError('This event loop is already running')
+        if events._get_running_loop() is not None:
+            raise RuntimeError(
+                'Cannot run the event loop while another loop is running')
         self._set_coroutine_wrapper(self._debug)
         self._thread_id = threading.get_ident()
         if self._asyncgens is not None:
@@ -401,6 +404,7 @@
             sys.set_asyncgen_hooks(firstiter=self._asyncgen_firstiter_hook,
                                    finalizer=self._asyncgen_finalizer_hook)
         try:
+            events._set_running_loop(self)
             while True:
                 self._run_once()
                 if self._stopping:
@@ -408,6 +412,7 @@
         finally:
             self._stopping = False
             self._thread_id = None
+            events._set_running_loop(None)
             self._set_coroutine_wrapper(False)
             if self._asyncgens is not None:
                 sys.set_asyncgen_hooks(*old_agen_hooks)
diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py
--- a/Lib/asyncio/events.py
+++ b/Lib/asyncio/events.py
@@ -607,6 +607,30 @@
 _lock = threading.Lock()
 
 
+# A TLS for the running event loop, used by _get_running_loop.
+class _RunningLoop(threading.local):
+    _loop = None
+_running_loop = _RunningLoop()
+
+
+def _get_running_loop():
+    """Return the running event loop or None.
+
+    This is a low-level function intended to be used by event loops.
+    This function is thread-specific.
+    """
+    return _running_loop._loop
+
+
+def _set_running_loop(loop):
+    """Set the running event loop.
+
+    This is a low-level function intended to be used by event loops.
+    This function is thread-specific.
+    """
+    _running_loop._loop = loop
+
+
 def _init_event_loop_policy():
     global _event_loop_policy
     with _lock:
@@ -632,7 +656,17 @@
 
 
 def get_event_loop():
-    """Equivalent to calling get_event_loop_policy().get_event_loop()."""
+    """Return an asyncio event loop.
+
+    When called from a coroutine or a callback (e.g. scheduled with call_soon
+    or similar API), this function will always return the running event loop.
+
+    If there is no running event loop set, the function will return
+    the result of `get_event_loop_policy().get_event_loop()` call.
+    """
+    current_loop = _get_running_loop()
+    if current_loop is not None:
+        return current_loop
     return get_event_loop_policy().get_event_loop()
 
 
diff --git a/Lib/asyncio/test_utils.py b/Lib/asyncio/test_utils.py
--- a/Lib/asyncio/test_utils.py
+++ b/Lib/asyncio/test_utils.py
@@ -449,7 +449,13 @@
         self.set_event_loop(loop)
         return loop
 
+    def setUp(self):
+        self._get_running_loop = events._get_running_loop
+        events._get_running_loop = lambda: None
+
     def tearDown(self):
+        events._get_running_loop = self._get_running_loop
+
         events.set_event_loop(None)
 
         # Detect CPython bug #23353: ensure that yield/yield-from is not used
diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py
--- a/Lib/test/test_asyncio/test_base_events.py
+++ b/Lib/test/test_asyncio/test_base_events.py
@@ -154,6 +154,7 @@
 class BaseEventLoopTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = base_events.BaseEventLoop()
         self.loop._selector = mock.Mock()
         self.loop._selector.select.return_value = ()
@@ -976,6 +977,7 @@
 class BaseEventLoopWithSelectorTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.new_event_loop()
         self.set_event_loop(self.loop)
 
@@ -1692,5 +1694,23 @@
                          "took .* seconds$")
 
 
+class RunningLoopTests(unittest.TestCase):
+
+    def test_running_loop_within_a_loop(self):
+        @asyncio.coroutine
+        def runner(loop):
+            loop.run_forever()
+
+        loop = asyncio.new_event_loop()
+        outer_loop = asyncio.new_event_loop()
+        try:
+            with self.assertRaisesRegex(RuntimeError,
+                                        'while another loop is running'):
+                outer_loop.run_until_complete(runner(loop))
+        finally:
+            loop.close()
+            outer_loop.close()
+
+
 if __name__ == '__main__':
     unittest.main()
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -2233,6 +2233,7 @@
 class HandleTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = mock.Mock()
         self.loop.get_debug.return_value = True
 
@@ -2411,6 +2412,7 @@
 class TimerTests(unittest.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = mock.Mock()
 
     def test_hash(self):
@@ -2719,6 +2721,27 @@
         self.assertIs(policy, asyncio.get_event_loop_policy())
         self.assertIsNot(policy, old_policy)
 
+    def test_get_event_loop_returns_running_loop(self):
+        class Policy(asyncio.DefaultEventLoopPolicy):
+            def get_event_loop(self):
+                raise NotImplementedError
+
+        loop = None
+
+        old_policy = asyncio.get_event_loop_policy()
+        try:
+            asyncio.set_event_loop_policy(Policy())
+            loop = asyncio.new_event_loop()
+
+            async def func():
+                self.assertIs(asyncio.get_event_loop(), loop)
+
+            loop.run_until_complete(func())
+        finally:
+            asyncio.set_event_loop_policy(old_policy)
+            if loop is not None:
+                loop.close()
+
 
 if __name__ == '__main__':
     unittest.main()
diff --git a/Lib/test/test_asyncio/test_futures.py b/Lib/test/test_asyncio/test_futures.py
--- a/Lib/test/test_asyncio/test_futures.py
+++ b/Lib/test/test_asyncio/test_futures.py
@@ -80,6 +80,7 @@
 class DuckTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.addCleanup(self.loop.close)
 
@@ -100,6 +101,7 @@
         raise NotImplementedError
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.addCleanup(self.loop.close)
 
@@ -459,6 +461,7 @@
 class BaseFutureDoneCallbackTests():
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def run_briefly(self):
diff --git a/Lib/test/test_asyncio/test_locks.py b/Lib/test/test_asyncio/test_locks.py
--- a/Lib/test/test_asyncio/test_locks.py
+++ b/Lib/test/test_asyncio/test_locks.py
@@ -19,6 +19,7 @@
 class LockTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def test_ctor_loop(self):
@@ -235,6 +236,7 @@
 class EventTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def test_ctor_loop(self):
@@ -364,6 +366,7 @@
 class ConditionTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def test_ctor_loop(self):
@@ -699,6 +702,7 @@
 class SemaphoreTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def test_ctor_loop(self):
diff --git a/Lib/test/test_asyncio/test_pep492.py b/Lib/test/test_asyncio/test_pep492.py
--- a/Lib/test/test_asyncio/test_pep492.py
+++ b/Lib/test/test_asyncio/test_pep492.py
@@ -17,6 +17,7 @@
 class BaseTest(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.BaseEventLoop()
         self.loop._process_events = mock.Mock()
         self.loop._selector = mock.Mock()
diff --git a/Lib/test/test_asyncio/test_proactor_events.py b/Lib/test/test_asyncio/test_proactor_events.py
--- a/Lib/test/test_asyncio/test_proactor_events.py
+++ b/Lib/test/test_asyncio/test_proactor_events.py
@@ -24,6 +24,7 @@
 class ProactorSocketTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.addCleanup(self.loop.close)
         self.proactor = mock.Mock()
@@ -436,6 +437,8 @@
 class BaseProactorEventLoopTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
+
         self.sock = test_utils.mock_nonblocking_socket()
         self.proactor = mock.Mock()
 
diff --git a/Lib/test/test_asyncio/test_queues.py b/Lib/test/test_asyncio/test_queues.py
--- a/Lib/test/test_asyncio/test_queues.py
+++ b/Lib/test/test_asyncio/test_queues.py
@@ -10,6 +10,7 @@
 class _QueueTestBase(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
 
diff --git a/Lib/test/test_asyncio/test_selector_events.py b/Lib/test/test_asyncio/test_selector_events.py
--- a/Lib/test/test_asyncio/test_selector_events.py
+++ b/Lib/test/test_asyncio/test_selector_events.py
@@ -51,6 +51,7 @@
 class BaseSelectorEventLoopTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.selector = mock.Mock()
         self.selector.select.return_value = []
         self.loop = TestBaseSelectorEventLoop(self.selector)
@@ -698,6 +699,7 @@
 class SelectorTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.Protocol)
         self.sock = mock.Mock(socket.socket)
@@ -793,6 +795,7 @@
 class SelectorSocketTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.Protocol)
         self.sock = mock.Mock(socket.socket)
@@ -1141,6 +1144,7 @@
 class SelectorSslTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.Protocol)
         self.sock = mock.Mock(socket.socket)
@@ -1501,6 +1505,7 @@
 class SelectorDatagramTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.DatagramProtocol)
         self.sock = mock.Mock(spec_set=socket.socket)
diff --git a/Lib/test/test_asyncio/test_sslproto.py b/Lib/test/test_asyncio/test_sslproto.py
--- a/Lib/test/test_asyncio/test_sslproto.py
+++ b/Lib/test/test_asyncio/test_sslproto.py
@@ -18,6 +18,7 @@
 class SslProtoHandshakeTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.new_event_loop()
         self.set_event_loop(self.loop)
 
diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py
--- a/Lib/test/test_asyncio/test_streams.py
+++ b/Lib/test/test_asyncio/test_streams.py
@@ -22,6 +22,7 @@
     DATA = b'line1\nline2\nline3\n'
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.new_event_loop()
         self.set_event_loop(self.loop)
 
diff --git a/Lib/test/test_asyncio/test_subprocess.py b/Lib/test/test_asyncio/test_subprocess.py
--- a/Lib/test/test_asyncio/test_subprocess.py
+++ b/Lib/test/test_asyncio/test_subprocess.py
@@ -35,6 +35,7 @@
 
 class SubprocessTransportTests(test_utils.TestCase):
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.set_event_loop(self.loop)
 
@@ -466,6 +467,7 @@
         Watcher = None
 
         def setUp(self):
+            super().setUp()
             policy = asyncio.get_event_loop_policy()
             self.loop = policy.new_event_loop()
             self.set_event_loop(self.loop)
@@ -490,6 +492,7 @@
     class SubprocessProactorTests(SubprocessMixin, test_utils.TestCase):
 
         def setUp(self):
+            super().setUp()
             self.loop = asyncio.ProactorEventLoop()
             self.set_event_loop(self.loop)
 
diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py
--- a/Lib/test/test_asyncio/test_tasks.py
+++ b/Lib/test/test_asyncio/test_tasks.py
@@ -87,6 +87,7 @@
         return self.__class__.Future(loop=loop)
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.loop.set_task_factory(self.new_task)
         self.loop.create_future = lambda: self.new_future(self.loop)
@@ -2104,6 +2105,7 @@
 class GatherTestsBase:
 
     def setUp(self):
+        super().setUp()
         self.one_loop = self.new_test_loop()
         self.other_loop = self.new_test_loop()
         self.set_event_loop(self.one_loop, cleanup=False)
@@ -2387,6 +2389,7 @@
     """Test case for asyncio.run_coroutine_threadsafe."""
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.new_event_loop()
         self.set_event_loop(self.loop) # Will cleanup properly
 
@@ -2477,12 +2480,14 @@
 
 class SleepTests(test_utils.TestCase):
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.new_event_loop()
         asyncio.set_event_loop(None)
 
     def tearDown(self):
         self.loop.close()
         self.loop = None
+        super().tearDown()
 
     def test_sleep_zero(self):
         result = 0
diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py
--- a/Lib/test/test_asyncio/test_unix_events.py
+++ b/Lib/test/test_asyncio/test_unix_events.py
@@ -40,6 +40,7 @@
 class SelectorEventLoopSignalTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.SelectorEventLoop()
         self.set_event_loop(self.loop)
 
@@ -234,6 +235,7 @@
 class SelectorEventLoopUnixSocketTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.SelectorEventLoop()
         self.set_event_loop(self.loop)
 
@@ -338,6 +340,7 @@
 class UnixReadPipeTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.Protocol)
         self.pipe = mock.Mock(spec_set=io.RawIOBase)
@@ -487,6 +490,7 @@
 class UnixWritePipeTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.BaseProtocol)
         self.pipe = mock.Mock(spec_set=io.RawIOBase)
@@ -805,6 +809,7 @@
     ignore_warnings = mock.patch.object(log.logger, "warning")
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.running = False
         self.zombies = {}
diff --git a/Lib/test/test_asyncio/test_windows_events.py b/Lib/test/test_asyncio/test_windows_events.py
--- a/Lib/test/test_asyncio/test_windows_events.py
+++ b/Lib/test/test_asyncio/test_windows_events.py
@@ -31,6 +31,7 @@
 class ProactorTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.ProactorEventLoop()
         self.set_event_loop(self.loop)
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -22,6 +22,9 @@
 
 - Issue #28600: Optimize loop.call_soon.
 
+- Issue #28613: Fix get_event_loop() return the current loop if
+  called from coroutines/callbacks.
+
 Documentation
 -------------
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 14:30:49 2016
From: python-checkins at python.org (yury.selivanov)
Date: Fri, 04 Nov 2016 18:30:49 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg2MTMp?=
Message-ID: <20161104183049.3194.13953.F6BE9585@psf.io>

https://hg.python.org/cpython/rev/af4ac4e4b188
changeset:   104907:af4ac4e4b188
parent:      104904:3904194d06e6
parent:      104906:1473b9a17a91
user:        Yury Selivanov <yury at magic.io>
date:        Fri Nov 04 14:30:41 2016 -0400
summary:
  Merge 3.6 (issue #28613)

files:
  Lib/asyncio/base_events.py                    |   7 +-
  Lib/asyncio/events.py                         |  36 +++++++++-
  Lib/asyncio/test_utils.py                     |   6 +
  Lib/test/test_asyncio/test_base_events.py     |  20 +++++
  Lib/test/test_asyncio/test_events.py          |  23 ++++++
  Lib/test/test_asyncio/test_futures.py         |   3 +
  Lib/test/test_asyncio/test_locks.py           |   4 +
  Lib/test/test_asyncio/test_pep492.py          |   1 +
  Lib/test/test_asyncio/test_proactor_events.py |   3 +
  Lib/test/test_asyncio/test_queues.py          |   1 +
  Lib/test/test_asyncio/test_selector_events.py |   5 +
  Lib/test/test_asyncio/test_sslproto.py        |   1 +
  Lib/test/test_asyncio/test_streams.py         |   1 +
  Lib/test/test_asyncio/test_subprocess.py      |   3 +
  Lib/test/test_asyncio/test_tasks.py           |   5 +
  Lib/test/test_asyncio/test_unix_events.py     |   5 +
  Lib/test/test_asyncio/test_windows_events.py  |   1 +
  17 files changed, 123 insertions(+), 2 deletions(-)


diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -393,7 +393,10 @@
         """Run until stop() is called."""
         self._check_closed()
         if self.is_running():
-            raise RuntimeError('Event loop is running.')
+            raise RuntimeError('This event loop is already running')
+        if events._get_running_loop() is not None:
+            raise RuntimeError(
+                'Cannot run the event loop while another loop is running')
         self._set_coroutine_wrapper(self._debug)
         self._thread_id = threading.get_ident()
         if self._asyncgens is not None:
@@ -401,6 +404,7 @@
             sys.set_asyncgen_hooks(firstiter=self._asyncgen_firstiter_hook,
                                    finalizer=self._asyncgen_finalizer_hook)
         try:
+            events._set_running_loop(self)
             while True:
                 self._run_once()
                 if self._stopping:
@@ -408,6 +412,7 @@
         finally:
             self._stopping = False
             self._thread_id = None
+            events._set_running_loop(None)
             self._set_coroutine_wrapper(False)
             if self._asyncgens is not None:
                 sys.set_asyncgen_hooks(*old_agen_hooks)
diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py
--- a/Lib/asyncio/events.py
+++ b/Lib/asyncio/events.py
@@ -607,6 +607,30 @@
 _lock = threading.Lock()
 
 
+# A TLS for the running event loop, used by _get_running_loop.
+class _RunningLoop(threading.local):
+    _loop = None
+_running_loop = _RunningLoop()
+
+
+def _get_running_loop():
+    """Return the running event loop or None.
+
+    This is a low-level function intended to be used by event loops.
+    This function is thread-specific.
+    """
+    return _running_loop._loop
+
+
+def _set_running_loop(loop):
+    """Set the running event loop.
+
+    This is a low-level function intended to be used by event loops.
+    This function is thread-specific.
+    """
+    _running_loop._loop = loop
+
+
 def _init_event_loop_policy():
     global _event_loop_policy
     with _lock:
@@ -632,7 +656,17 @@
 
 
 def get_event_loop():
-    """Equivalent to calling get_event_loop_policy().get_event_loop()."""
+    """Return an asyncio event loop.
+
+    When called from a coroutine or a callback (e.g. scheduled with call_soon
+    or similar API), this function will always return the running event loop.
+
+    If there is no running event loop set, the function will return
+    the result of `get_event_loop_policy().get_event_loop()` call.
+    """
+    current_loop = _get_running_loop()
+    if current_loop is not None:
+        return current_loop
     return get_event_loop_policy().get_event_loop()
 
 
diff --git a/Lib/asyncio/test_utils.py b/Lib/asyncio/test_utils.py
--- a/Lib/asyncio/test_utils.py
+++ b/Lib/asyncio/test_utils.py
@@ -449,7 +449,13 @@
         self.set_event_loop(loop)
         return loop
 
+    def setUp(self):
+        self._get_running_loop = events._get_running_loop
+        events._get_running_loop = lambda: None
+
     def tearDown(self):
+        events._get_running_loop = self._get_running_loop
+
         events.set_event_loop(None)
 
         # Detect CPython bug #23353: ensure that yield/yield-from is not used
diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py
--- a/Lib/test/test_asyncio/test_base_events.py
+++ b/Lib/test/test_asyncio/test_base_events.py
@@ -154,6 +154,7 @@
 class BaseEventLoopTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = base_events.BaseEventLoop()
         self.loop._selector = mock.Mock()
         self.loop._selector.select.return_value = ()
@@ -976,6 +977,7 @@
 class BaseEventLoopWithSelectorTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.new_event_loop()
         self.set_event_loop(self.loop)
 
@@ -1692,5 +1694,23 @@
                          "took .* seconds$")
 
 
+class RunningLoopTests(unittest.TestCase):
+
+    def test_running_loop_within_a_loop(self):
+        @asyncio.coroutine
+        def runner(loop):
+            loop.run_forever()
+
+        loop = asyncio.new_event_loop()
+        outer_loop = asyncio.new_event_loop()
+        try:
+            with self.assertRaisesRegex(RuntimeError,
+                                        'while another loop is running'):
+                outer_loop.run_until_complete(runner(loop))
+        finally:
+            loop.close()
+            outer_loop.close()
+
+
 if __name__ == '__main__':
     unittest.main()
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -2233,6 +2233,7 @@
 class HandleTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = mock.Mock()
         self.loop.get_debug.return_value = True
 
@@ -2411,6 +2412,7 @@
 class TimerTests(unittest.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = mock.Mock()
 
     def test_hash(self):
@@ -2719,6 +2721,27 @@
         self.assertIs(policy, asyncio.get_event_loop_policy())
         self.assertIsNot(policy, old_policy)
 
+    def test_get_event_loop_returns_running_loop(self):
+        class Policy(asyncio.DefaultEventLoopPolicy):
+            def get_event_loop(self):
+                raise NotImplementedError
+
+        loop = None
+
+        old_policy = asyncio.get_event_loop_policy()
+        try:
+            asyncio.set_event_loop_policy(Policy())
+            loop = asyncio.new_event_loop()
+
+            async def func():
+                self.assertIs(asyncio.get_event_loop(), loop)
+
+            loop.run_until_complete(func())
+        finally:
+            asyncio.set_event_loop_policy(old_policy)
+            if loop is not None:
+                loop.close()
+
 
 if __name__ == '__main__':
     unittest.main()
diff --git a/Lib/test/test_asyncio/test_futures.py b/Lib/test/test_asyncio/test_futures.py
--- a/Lib/test/test_asyncio/test_futures.py
+++ b/Lib/test/test_asyncio/test_futures.py
@@ -80,6 +80,7 @@
 class DuckTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.addCleanup(self.loop.close)
 
@@ -100,6 +101,7 @@
         raise NotImplementedError
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.addCleanup(self.loop.close)
 
@@ -459,6 +461,7 @@
 class BaseFutureDoneCallbackTests():
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def run_briefly(self):
diff --git a/Lib/test/test_asyncio/test_locks.py b/Lib/test/test_asyncio/test_locks.py
--- a/Lib/test/test_asyncio/test_locks.py
+++ b/Lib/test/test_asyncio/test_locks.py
@@ -19,6 +19,7 @@
 class LockTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def test_ctor_loop(self):
@@ -235,6 +236,7 @@
 class EventTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def test_ctor_loop(self):
@@ -364,6 +366,7 @@
 class ConditionTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def test_ctor_loop(self):
@@ -699,6 +702,7 @@
 class SemaphoreTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
     def test_ctor_loop(self):
diff --git a/Lib/test/test_asyncio/test_pep492.py b/Lib/test/test_asyncio/test_pep492.py
--- a/Lib/test/test_asyncio/test_pep492.py
+++ b/Lib/test/test_asyncio/test_pep492.py
@@ -17,6 +17,7 @@
 class BaseTest(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.BaseEventLoop()
         self.loop._process_events = mock.Mock()
         self.loop._selector = mock.Mock()
diff --git a/Lib/test/test_asyncio/test_proactor_events.py b/Lib/test/test_asyncio/test_proactor_events.py
--- a/Lib/test/test_asyncio/test_proactor_events.py
+++ b/Lib/test/test_asyncio/test_proactor_events.py
@@ -24,6 +24,7 @@
 class ProactorSocketTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.addCleanup(self.loop.close)
         self.proactor = mock.Mock()
@@ -436,6 +437,8 @@
 class BaseProactorEventLoopTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
+
         self.sock = test_utils.mock_nonblocking_socket()
         self.proactor = mock.Mock()
 
diff --git a/Lib/test/test_asyncio/test_queues.py b/Lib/test/test_asyncio/test_queues.py
--- a/Lib/test/test_asyncio/test_queues.py
+++ b/Lib/test/test_asyncio/test_queues.py
@@ -10,6 +10,7 @@
 class _QueueTestBase(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
 
 
diff --git a/Lib/test/test_asyncio/test_selector_events.py b/Lib/test/test_asyncio/test_selector_events.py
--- a/Lib/test/test_asyncio/test_selector_events.py
+++ b/Lib/test/test_asyncio/test_selector_events.py
@@ -51,6 +51,7 @@
 class BaseSelectorEventLoopTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.selector = mock.Mock()
         self.selector.select.return_value = []
         self.loop = TestBaseSelectorEventLoop(self.selector)
@@ -698,6 +699,7 @@
 class SelectorTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.Protocol)
         self.sock = mock.Mock(socket.socket)
@@ -793,6 +795,7 @@
 class SelectorSocketTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.Protocol)
         self.sock = mock.Mock(socket.socket)
@@ -1141,6 +1144,7 @@
 class SelectorSslTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.Protocol)
         self.sock = mock.Mock(socket.socket)
@@ -1501,6 +1505,7 @@
 class SelectorDatagramTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.DatagramProtocol)
         self.sock = mock.Mock(spec_set=socket.socket)
diff --git a/Lib/test/test_asyncio/test_sslproto.py b/Lib/test/test_asyncio/test_sslproto.py
--- a/Lib/test/test_asyncio/test_sslproto.py
+++ b/Lib/test/test_asyncio/test_sslproto.py
@@ -18,6 +18,7 @@
 class SslProtoHandshakeTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.new_event_loop()
         self.set_event_loop(self.loop)
 
diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py
--- a/Lib/test/test_asyncio/test_streams.py
+++ b/Lib/test/test_asyncio/test_streams.py
@@ -22,6 +22,7 @@
     DATA = b'line1\nline2\nline3\n'
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.new_event_loop()
         self.set_event_loop(self.loop)
 
diff --git a/Lib/test/test_asyncio/test_subprocess.py b/Lib/test/test_asyncio/test_subprocess.py
--- a/Lib/test/test_asyncio/test_subprocess.py
+++ b/Lib/test/test_asyncio/test_subprocess.py
@@ -35,6 +35,7 @@
 
 class SubprocessTransportTests(test_utils.TestCase):
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.set_event_loop(self.loop)
 
@@ -466,6 +467,7 @@
         Watcher = None
 
         def setUp(self):
+            super().setUp()
             policy = asyncio.get_event_loop_policy()
             self.loop = policy.new_event_loop()
             self.set_event_loop(self.loop)
@@ -490,6 +492,7 @@
     class SubprocessProactorTests(SubprocessMixin, test_utils.TestCase):
 
         def setUp(self):
+            super().setUp()
             self.loop = asyncio.ProactorEventLoop()
             self.set_event_loop(self.loop)
 
diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py
--- a/Lib/test/test_asyncio/test_tasks.py
+++ b/Lib/test/test_asyncio/test_tasks.py
@@ -87,6 +87,7 @@
         return self.__class__.Future(loop=loop)
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.loop.set_task_factory(self.new_task)
         self.loop.create_future = lambda: self.new_future(self.loop)
@@ -2104,6 +2105,7 @@
 class GatherTestsBase:
 
     def setUp(self):
+        super().setUp()
         self.one_loop = self.new_test_loop()
         self.other_loop = self.new_test_loop()
         self.set_event_loop(self.one_loop, cleanup=False)
@@ -2387,6 +2389,7 @@
     """Test case for asyncio.run_coroutine_threadsafe."""
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.new_event_loop()
         self.set_event_loop(self.loop) # Will cleanup properly
 
@@ -2477,12 +2480,14 @@
 
 class SleepTests(test_utils.TestCase):
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.new_event_loop()
         asyncio.set_event_loop(None)
 
     def tearDown(self):
         self.loop.close()
         self.loop = None
+        super().tearDown()
 
     def test_sleep_zero(self):
         result = 0
diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py
--- a/Lib/test/test_asyncio/test_unix_events.py
+++ b/Lib/test/test_asyncio/test_unix_events.py
@@ -40,6 +40,7 @@
 class SelectorEventLoopSignalTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.SelectorEventLoop()
         self.set_event_loop(self.loop)
 
@@ -234,6 +235,7 @@
 class SelectorEventLoopUnixSocketTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.SelectorEventLoop()
         self.set_event_loop(self.loop)
 
@@ -338,6 +340,7 @@
 class UnixReadPipeTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.Protocol)
         self.pipe = mock.Mock(spec_set=io.RawIOBase)
@@ -487,6 +490,7 @@
 class UnixWritePipeTransportTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.protocol = test_utils.make_test_protocol(asyncio.BaseProtocol)
         self.pipe = mock.Mock(spec_set=io.RawIOBase)
@@ -805,6 +809,7 @@
     ignore_warnings = mock.patch.object(log.logger, "warning")
 
     def setUp(self):
+        super().setUp()
         self.loop = self.new_test_loop()
         self.running = False
         self.zombies = {}
diff --git a/Lib/test/test_asyncio/test_windows_events.py b/Lib/test/test_asyncio/test_windows_events.py
--- a/Lib/test/test_asyncio/test_windows_events.py
+++ b/Lib/test/test_asyncio/test_windows_events.py
@@ -31,6 +31,7 @@
 class ProactorTests(test_utils.TestCase):
 
     def setUp(self):
+        super().setUp()
         self.loop = asyncio.ProactorEventLoop()
         self.set_event_loop(self.loop)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 17:08:42 2016
From: python-checkins at python.org (ned.deily)
Date: Fri, 04 Nov 2016 21:08:42 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328616=3A_merge_from_3=2E5?=
Message-ID: <20161104210842.111593.15469.91687D8F@psf.io>

https://hg.python.org/cpython/rev/048870daf397
changeset:   104910:048870daf397
branch:      3.6
parent:      104906:1473b9a17a91
parent:      104909:1390bde4b768
user:        Ned Deily <nad at python.org>
date:        Fri Nov 04 17:07:06 2016 -0400
summary:
  Issue #28616: merge from 3.5

files:
  Misc/ACKS          |  1 +
  Python/sysmodule.c |  2 +-
  2 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -1486,6 +1486,7 @@
 Indra Talip
 Neil Tallim
 Geoff Talvola
+Anish Tambe
 Musashi Tamura
 William Tanksley
 Christian Tanzer
diff --git a/Python/sysmodule.c b/Python/sysmodule.c
--- a/Python/sysmodule.c
+++ b/Python/sysmodule.c
@@ -1739,7 +1739,7 @@
     {"major", "Major release number"},
     {"minor", "Minor release number"},
     {"micro", "Patch release number"},
-    {"releaselevel", "'alpha', 'beta', 'candidate', or 'release'"},
+    {"releaselevel", "'alpha', 'beta', 'candidate', or 'final'"},
     {"serial", "Serial release number"},
     {0}
 };

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 17:08:42 2016
From: python-checkins at python.org (ned.deily)
Date: Fri, 04 Nov 2016 21:08:42 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4NjE2?=
 =?utf-8?q?=3A_Correct_help_for_sys=2Eversion=5Finfo_releaselevel_componen?=
 =?utf-8?q?t=2E?=
Message-ID: <20161104210841.111534.45466.7F8CDF1F@psf.io>

https://hg.python.org/cpython/rev/0b4bcd954554
changeset:   104908:0b4bcd954554
branch:      2.7
parent:      104886:91f024fc9b3a
user:        Ned Deily <nad at python.org>
date:        Fri Nov 04 17:05:59 2016 -0400
summary:
  Issue #28616: Correct help for sys.version_info releaselevel component.
Patch by Anish Tambe.

files:
  Misc/ACKS          |  1 +
  Python/sysmodule.c |  2 +-
  2 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -1371,6 +1371,7 @@
 Indra Talip
 Neil Tallim
 Geoff Talvola
+Anish Tambe
 Musashi Tamura
 William Tanksley
 Christian Tanzer
diff --git a/Python/sysmodule.c b/Python/sysmodule.c
--- a/Python/sysmodule.c
+++ b/Python/sysmodule.c
@@ -1295,7 +1295,7 @@
     {"major", "Major release number"},
     {"minor", "Minor release number"},
     {"micro", "Patch release number"},
-    {"releaselevel", "'alpha', 'beta', 'candidate', or 'release'"},
+    {"releaselevel", "'alpha', 'beta', 'candidate', or 'final'"},
     {"serial", "Serial release number"},
     {0}
 };

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 17:08:42 2016
From: python-checkins at python.org (ned.deily)
Date: Fri, 04 Nov 2016 21:08:42 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjE2?=
 =?utf-8?q?=3A_Correct_help_for_sys=2Eversion=5Finfo_releaselevel_componen?=
 =?utf-8?q?t=2E?=
Message-ID: <20161104210841.31709.52992.12F3A32F@psf.io>

https://hg.python.org/cpython/rev/1390bde4b768
changeset:   104909:1390bde4b768
branch:      3.5
parent:      104905:aa37f3859462
user:        Ned Deily <nad at python.org>
date:        Fri Nov 04 17:03:34 2016 -0400
summary:
  Issue #28616: Correct help for sys.version_info releaselevel component.
Patch by Anish Tambe.

files:
  Misc/ACKS          |  1 +
  Python/sysmodule.c |  2 +-
  2 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -1462,6 +1462,7 @@
 Indra Talip
 Neil Tallim
 Geoff Talvola
+Anish Tambe
 Musashi Tamura
 William Tanksley
 Christian Tanzer
diff --git a/Python/sysmodule.c b/Python/sysmodule.c
--- a/Python/sysmodule.c
+++ b/Python/sysmodule.c
@@ -1619,7 +1619,7 @@
     {"major", "Major release number"},
     {"minor", "Minor release number"},
     {"micro", "Patch release number"},
-    {"releaselevel", "'alpha', 'beta', 'candidate', or 'release'"},
+    {"releaselevel", "'alpha', 'beta', 'candidate', or 'final'"},
     {"serial", "Serial release number"},
     {0}
 };

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 17:08:42 2016
From: python-checkins at python.org (ned.deily)
Date: Fri, 04 Nov 2016 21:08:42 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328616=3A_merge_from_3=2E6?=
Message-ID: <20161104210842.7195.73343.2AECBBC2@psf.io>

https://hg.python.org/cpython/rev/87d76ce01217
changeset:   104911:87d76ce01217
parent:      104907:af4ac4e4b188
parent:      104910:048870daf397
user:        Ned Deily <nad at python.org>
date:        Fri Nov 04 17:07:47 2016 -0400
summary:
  Issue #28616: merge from 3.6

files:
  Misc/ACKS          |  1 +
  Python/sysmodule.c |  2 +-
  2 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -1488,6 +1488,7 @@
 Indra Talip
 Neil Tallim
 Geoff Talvola
+Anish Tambe
 Musashi Tamura
 William Tanksley
 Christian Tanzer
diff --git a/Python/sysmodule.c b/Python/sysmodule.c
--- a/Python/sysmodule.c
+++ b/Python/sysmodule.c
@@ -1739,7 +1739,7 @@
     {"major", "Major release number"},
     {"minor", "Minor release number"},
     {"micro", "Patch release number"},
-    {"releaselevel", "'alpha', 'beta', 'candidate', or 'release'"},
+    {"releaselevel", "'alpha', 'beta', 'candidate', or 'final'"},
     {"serial", "Serial release number"},
     {0}
 };

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 22:23:43 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 05 Nov 2016 02:23:43 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328485=3A_Merge_single-threading_fix_from_3=2E5_into_3?=
 =?utf-8?q?=2E6?=
Message-ID: <20161105022343.22687.91915.9E2F1AF7@psf.io>

https://hg.python.org/cpython/rev/a7c76c3843af
changeset:   104913:a7c76c3843af
branch:      3.6
parent:      104910:048870daf397
parent:      104912:532b0b9f41e0
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sat Nov 05 01:35:25 2016 +0000
summary:
  Issue #28485: Merge single-threading fix from 3.5 into 3.6

files:
  Lib/compileall.py |  6 +++---
  Misc/NEWS         |  4 ++++
  2 files changed, 7 insertions(+), 3 deletions(-)


diff --git a/Lib/compileall.py b/Lib/compileall.py
--- a/Lib/compileall.py
+++ b/Lib/compileall.py
@@ -68,13 +68,13 @@
     optimize:  optimization level or -1 for level of the interpreter
     workers:   maximum number of parallel workers
     """
+    if workers is not None and workers < 0:
+        raise ValueError('workers must be greater or equal to 0')
+
     files = _walk_dir(dir, quiet=quiet, maxlevels=maxlevels,
                       ddir=ddir)
     success = True
     if workers is not None and workers != 1 and ProcessPoolExecutor is not None:
-        if workers < 0:
-            raise ValueError('workers must be greater or equal to 0')
-
         workers = workers or None
         with ProcessPoolExecutor(max_workers=workers) as executor:
             results = executor.map(partial(compile_file,
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -573,6 +573,10 @@
 Library
 -------
 
+- Issue #28485: Always raise ValueError for negative
+  compileall.compile_dir(workers=...) parameter, even when multithreading is
+  unavailable.
+
 - Issue #28037: Use sqlite3_get_autocommit() instead of setting
   Connection->inTransaction manually.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 22:23:43 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 05 Nov 2016 02:23:43 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NDg1?=
 =?utf-8?q?=3A_Check_for_negative_workers_even_without_ProcessPoolExecutor?=
Message-ID: <20161105022342.3385.8392.D9DDCA0E@psf.io>

https://hg.python.org/cpython/rev/532b0b9f41e0
changeset:   104912:532b0b9f41e0
branch:      3.5
parent:      104909:1390bde4b768
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sat Nov 05 01:11:36 2016 +0000
summary:
  Issue #28485: Check for negative workers even without ProcessPoolExecutor

This matches the documentation, and passes the test suite when multithreading
is disabled.

files:
  Lib/compileall.py |  6 +++---
  Misc/NEWS         |  4 ++++
  2 files changed, 7 insertions(+), 3 deletions(-)


diff --git a/Lib/compileall.py b/Lib/compileall.py
--- a/Lib/compileall.py
+++ b/Lib/compileall.py
@@ -66,13 +66,13 @@
     optimize:  optimization level or -1 for level of the interpreter
     workers:   maximum number of parallel workers
     """
+    if workers is not None and workers < 0:
+        raise ValueError('workers must be greater or equal to 0')
+
     files = _walk_dir(dir, quiet=quiet, maxlevels=maxlevels,
                       ddir=ddir)
     success = 1
     if workers is not None and workers != 1 and ProcessPoolExecutor is not None:
-        if workers < 0:
-            raise ValueError('workers must be greater or equal to 0')
-
         workers = workers or None
         with ProcessPoolExecutor(max_workers=workers) as executor:
             results = executor.map(partial(compile_file,
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -113,6 +113,10 @@
 Library
 -------
 
+- Issue #28485: Always raise ValueError for negative
+  compileall.compile_dir(workers=...) parameter, even when multithreading is
+  unavailable.
+
 - Issue #28387: Fixed possible crash in _io.TextIOWrapper deallocator when
   the garbage collector is invoked in other thread.  Based on patch by
   Sebastian Cufre.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 22:23:43 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 05 Nov 2016 02:23:43 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328485=3A_Merge_single-threading_fix_from_3=2E6?=
Message-ID: <20161105022343.3656.78451.A476C009@psf.io>

https://hg.python.org/cpython/rev/7d9885fd6777
changeset:   104914:7d9885fd6777
parent:      104911:87d76ce01217
parent:      104913:a7c76c3843af
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sat Nov 05 01:56:58 2016 +0000
summary:
  Issue #28485: Merge single-threading fix from 3.6

files:
  Lib/compileall.py |  6 +++---
  Misc/NEWS         |  4 ++++
  2 files changed, 7 insertions(+), 3 deletions(-)


diff --git a/Lib/compileall.py b/Lib/compileall.py
--- a/Lib/compileall.py
+++ b/Lib/compileall.py
@@ -68,13 +68,13 @@
     optimize:  optimization level or -1 for level of the interpreter
     workers:   maximum number of parallel workers
     """
+    if workers is not None and workers < 0:
+        raise ValueError('workers must be greater or equal to 0')
+
     files = _walk_dir(dir, quiet=quiet, maxlevels=maxlevels,
                       ddir=ddir)
     success = True
     if workers is not None and workers != 1 and ProcessPoolExecutor is not None:
-        if workers < 0:
-            raise ValueError('workers must be greater or equal to 0')
-
         workers = workers or None
         with ProcessPoolExecutor(max_workers=workers) as executor:
             results = executor.map(partial(compile_file,
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -582,6 +582,10 @@
 Library
 -------
 
+- Issue #28485: Always raise ValueError for negative
+  compileall.compile_dir(workers=...) parameter, even when multithreading is
+  unavailable.
+
 - Issue #28037: Use sqlite3_get_autocommit() instead of setting
   Connection->inTransaction manually.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 23:50:32 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 05 Nov 2016 03:50:32 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_spacing_af?=
 =?utf-8?q?ter_C++_in_documentation?=
Message-ID: <20161105035032.102211.63209.9B07CD99@psf.io>

https://hg.python.org/cpython/rev/2b58cde6a7f2
changeset:   104919:2b58cde6a7f2
branch:      2.7
parent:      104908:0b4bcd954554
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sat Nov 05 03:15:20 2016 +0000
summary:
  Fix spacing after C++ in documentation

files:
  Doc/distutils/apiref.rst      |  2 +-
  Doc/distutils/setupscript.rst |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst
--- a/Doc/distutils/apiref.rst
+++ b/Doc/distutils/apiref.rst
@@ -166,7 +166,7 @@
 
 .. class:: Extension
 
-   The Extension class describes a single C or C++extension module in a setup
+   The Extension class describes a single C or C++ extension module in a setup
    script. It accepts the following keyword arguments in its constructor
 
    .. tabularcolumns:: |l|L|l|
diff --git a/Doc/distutils/setupscript.rst b/Doc/distutils/setupscript.rst
--- a/Doc/distutils/setupscript.rst
+++ b/Doc/distutils/setupscript.rst
@@ -201,7 +201,7 @@
 a list of source
 files.  Since the Distutils currently only support C, C++, and Objective-C
 extensions, these are normally C/C++/Objective-C source files.  (Be sure to use
-appropriate extensions to distinguish C++\ source files: :file:`.cc` and
+appropriate extensions to distinguish C++ source files: :file:`.cc` and
 :file:`.cpp` seem to be recognized by both Unix and Windows compilers.)
 
 However, you can also include SWIG interface (:file:`.i`) files in the list; the

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 23:50:32 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 05 Nov 2016 03:50:32 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_doc_fixup_from_3=2E5_into_3=2E6?=
Message-ID: <20161105035032.83682.94883.989021B4@psf.io>

https://hg.python.org/cpython/rev/361409a8961e
changeset:   104916:361409a8961e
branch:      3.6
parent:      104913:a7c76c3843af
parent:      104915:8448152f64d7
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sat Nov 05 02:46:30 2016 +0000
summary:
  Merge doc fixup from 3.5 into 3.6

files:
  Doc/distutils/apiref.rst      |  2 +-
  Doc/distutils/setupscript.rst |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst
--- a/Doc/distutils/apiref.rst
+++ b/Doc/distutils/apiref.rst
@@ -166,7 +166,7 @@
 
 .. class:: Extension
 
-   The Extension class describes a single C or C++extension module in a setup
+   The Extension class describes a single C or C++ extension module in a setup
    script. It accepts the following keyword arguments in its constructor:
 
    .. tabularcolumns:: |l|L|l|
diff --git a/Doc/distutils/setupscript.rst b/Doc/distutils/setupscript.rst
--- a/Doc/distutils/setupscript.rst
+++ b/Doc/distutils/setupscript.rst
@@ -201,7 +201,7 @@
 a list of source
 files.  Since the Distutils currently only support C, C++, and Objective-C
 extensions, these are normally C/C++/Objective-C source files.  (Be sure to use
-appropriate extensions to distinguish C++\ source files: :file:`.cc` and
+appropriate extensions to distinguish C++ source files: :file:`.cc` and
 :file:`.cpp` seem to be recognized by both Unix and Windows compilers.)
 
 However, you can also include SWIG interface (:file:`.i`) files in the list; the

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 23:50:31 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 05 Nov 2016 03:50:31 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_spacing_af?=
 =?utf-8?q?ter_C++_in_documentation?=
Message-ID: <20161105035031.102467.58606.BDEB5407@psf.io>

https://hg.python.org/cpython/rev/8448152f64d7
changeset:   104915:8448152f64d7
branch:      3.5
parent:      104912:532b0b9f41e0
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sat Nov 05 02:40:31 2016 +0000
summary:
  Fix spacing after C++ in documentation

files:
  Doc/distutils/apiref.rst      |  2 +-
  Doc/distutils/setupscript.rst |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst
--- a/Doc/distutils/apiref.rst
+++ b/Doc/distutils/apiref.rst
@@ -166,7 +166,7 @@
 
 .. class:: Extension
 
-   The Extension class describes a single C or C++extension module in a setup
+   The Extension class describes a single C or C++ extension module in a setup
    script. It accepts the following keyword arguments in its constructor:
 
    .. tabularcolumns:: |l|L|l|
diff --git a/Doc/distutils/setupscript.rst b/Doc/distutils/setupscript.rst
--- a/Doc/distutils/setupscript.rst
+++ b/Doc/distutils/setupscript.rst
@@ -201,7 +201,7 @@
 a list of source
 files.  Since the Distutils currently only support C, C++, and Objective-C
 extensions, these are normally C/C++/Objective-C source files.  (Be sure to use
-appropriate extensions to distinguish C++\ source files: :file:`.cc` and
+appropriate extensions to distinguish C++ source files: :file:`.cc` and
 :file:`.cpp` seem to be recognized by both Unix and Windows compilers.)
 
 However, you can also include SWIG interface (:file:`.i`) files in the list; the

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 23:50:32 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 05 Nov 2016 03:50:32 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_disabled_test_code?=
Message-ID: <20161105035032.38742.41759.F86BF702@psf.io>

https://hg.python.org/cpython/rev/54087e63280a
changeset:   104918:54087e63280a
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sat Nov 05 02:49:20 2016 +0000
summary:
  Remove disabled test code

Loading Unix libraries via getattr() was disabled before ctypes was added to
Python.

files:
  Lib/ctypes/util.py |  4 ----
  1 files changed, 0 insertions(+), 4 deletions(-)


diff --git a/Lib/ctypes/util.py b/Lib/ctypes/util.py
--- a/Lib/ctypes/util.py
+++ b/Lib/ctypes/util.py
@@ -318,10 +318,6 @@
         print(find_library("c"))
         print(find_library("bz2"))
 
-        # getattr
-##        print cdll.m
-##        print cdll.bz2
-
         # load
         if sys.platform == "darwin":
             print(cdll.LoadLibrary("libm.dylib"))

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov  4 23:50:32 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 05 Nov 2016 03:50:32 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_doc_fixup_from_3=2E6?=
Message-ID: <20161105035032.22214.68841.362C5A62@psf.io>

https://hg.python.org/cpython/rev/5823096c4b78
changeset:   104917:5823096c4b78
parent:      104914:7d9885fd6777
parent:      104916:361409a8961e
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sat Nov 05 02:49:11 2016 +0000
summary:
  Merge doc fixup from 3.6

files:
  Doc/distutils/apiref.rst      |  2 +-
  Doc/distutils/setupscript.rst |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst
--- a/Doc/distutils/apiref.rst
+++ b/Doc/distutils/apiref.rst
@@ -166,7 +166,7 @@
 
 .. class:: Extension
 
-   The Extension class describes a single C or C++extension module in a setup
+   The Extension class describes a single C or C++ extension module in a setup
    script. It accepts the following keyword arguments in its constructor:
 
    .. tabularcolumns:: |l|L|l|
diff --git a/Doc/distutils/setupscript.rst b/Doc/distutils/setupscript.rst
--- a/Doc/distutils/setupscript.rst
+++ b/Doc/distutils/setupscript.rst
@@ -201,7 +201,7 @@
 a list of source
 files.  Since the Distutils currently only support C, C++, and Objective-C
 extensions, these are normally C/C++/Objective-C source files.  (Be sure to use
-appropriate extensions to distinguish C++\ source files: :file:`.cc` and
+appropriate extensions to distinguish C++ source files: :file:`.cc` and
 :file:`.cpp` seem to be recognized by both Unix and Windows compilers.)
 
 However, you can also include SWIG interface (:file:`.i`) files in the list; the

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 06:19:57 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 06 Nov 2016 11:19:57 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4MTIz?=
 =?utf-8?q?=3A_=5FPyDict=5FGetItem=5FKnownHash=28=29_now_can_raise_an_exce?=
 =?utf-8?q?ption_as?=
Message-ID: <20161106111956.38649.19321.E2EF04FC@psf.io>

https://hg.python.org/cpython/rev/d06a6b0fd992
changeset:   104920:d06a6b0fd992
branch:      3.6
parent:      104916:361409a8961e
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 06 13:18:24 2016 +0200
summary:
  Issue #28123: _PyDict_GetItem_KnownHash() now can raise an exception as
PyDict_GetItemWithError().  Patch by Xiang Zhang.

files:
  Lib/test/test_dict.py        |  31 +++++++++++++++
  Modules/_collectionsmodule.c |   2 +
  Modules/_testcapimodule.c    |  21 ++++++++++
  Objects/dictobject.c         |  48 +++++++++++-------------
  4 files changed, 76 insertions(+), 26 deletions(-)


diff --git a/Lib/test/test_dict.py b/Lib/test/test_dict.py
--- a/Lib/test/test_dict.py
+++ b/Lib/test/test_dict.py
@@ -1055,6 +1055,37 @@
         support.check_free_after_iterating(self, lambda d: iter(d.values()), dict)
         support.check_free_after_iterating(self, lambda d: iter(d.items()), dict)
 
+
+class CAPITest(unittest.TestCase):
+
+    # Test _PyDict_GetItem_KnownHash()
+    @support.cpython_only
+    def test_getitem_knownhash(self):
+        from _testcapi import dict_getitem_knownhash
+
+        d = {'x': 1, 'y': 2, 'z': 3}
+        self.assertEqual(dict_getitem_knownhash(d, 'x', hash('x')), 1)
+        self.assertEqual(dict_getitem_knownhash(d, 'y', hash('y')), 2)
+        self.assertEqual(dict_getitem_knownhash(d, 'z', hash('z')), 3)
+
+        # not a dict
+        self.assertRaises(SystemError, dict_getitem_knownhash, [], 1, hash(1))
+        # key does not exist
+        self.assertRaises(KeyError, dict_getitem_knownhash, {}, 1, hash(1))
+
+        class Exc(Exception): pass
+        class BadEq:
+            def __eq__(self, other):
+                raise Exc
+            def __hash__(self):
+                return 7
+
+        k1, k2 = BadEq(), BadEq()
+        d = {k1: 1}
+        self.assertEqual(dict_getitem_knownhash(d, k1, hash(k1)), 1)
+        self.assertRaises(Exc, dict_getitem_knownhash, d, k2, hash(k2))
+
+
 from test import mapping_tests
 
 class GeneralMappingTests(mapping_tests.BasicTestMappingProtocol):
diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c
--- a/Modules/_collectionsmodule.c
+++ b/Modules/_collectionsmodule.c
@@ -2300,6 +2300,8 @@
 
             oldval = _PyDict_GetItem_KnownHash(mapping, key, hash);
             if (oldval == NULL) {
+                if (PyErr_Occurred())
+                    goto done;
                 if (_PyDict_SetItem_KnownHash(mapping, key, one, hash) < 0)
                     goto done;
             } else {
diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c
--- a/Modules/_testcapimodule.c
+++ b/Modules/_testcapimodule.c
@@ -238,6 +238,26 @@
     return Py_None;
 }
 
+static PyObject*
+dict_getitem_knownhash(PyObject *self, PyObject *args)
+{
+    PyObject *mp, *key, *result;
+    Py_ssize_t hash;
+
+    if (!PyArg_ParseTuple(args, "OOn:dict_getitem_knownhash",
+                          &mp, &key, &hash)) {
+        return NULL;
+    }
+
+    result = _PyDict_GetItem_KnownHash(mp, key, (Py_hash_t)hash);
+    if (result == NULL && !PyErr_Occurred()) {
+        _PyErr_SetKeyError(key);
+        return NULL;
+    }
+
+    Py_XINCREF(result);
+    return result;
+}
 
 /* Issue #4701: Check that PyObject_Hash implicitly calls
  *   PyType_Ready if it hasn't already been called
@@ -4003,6 +4023,7 @@
     {"test_datetime_capi",  test_datetime_capi,              METH_NOARGS},
     {"test_list_api",           (PyCFunction)test_list_api,      METH_NOARGS},
     {"test_dict_iteration",     (PyCFunction)test_dict_iteration,METH_NOARGS},
+    {"dict_getitem_knownhash",  dict_getitem_knownhash,          METH_VARARGS},
     {"test_lazy_hash_inheritance",      (PyCFunction)test_lazy_hash_inheritance,METH_NOARGS},
     {"test_long_api",           (PyCFunction)test_long_api,      METH_NOARGS},
     {"test_xincref_doesnt_leak",(PyCFunction)test_xincref_doesnt_leak,      METH_NOARGS},
diff --git a/Objects/dictobject.c b/Objects/dictobject.c
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -723,8 +723,10 @@
             Py_INCREF(startkey);
             cmp = PyObject_RichCompareBool(startkey, key, Py_EQ);
             Py_DECREF(startkey);
-            if (cmp < 0)
+            if (cmp < 0) {
+                *value_addr = NULL;
                 return DKIX_ERROR;
+            }
             if (dk == mp->ma_keys && ep->me_key == startkey) {
                 if (cmp > 0) {
                     *value_addr = &ep->me_value;
@@ -1424,39 +1426,25 @@
     return *value_addr;
 }
 
+/* Same as PyDict_GetItemWithError() but with hash supplied by caller.
+   This returns NULL *with* an exception set if an exception occurred.
+   It returns NULL *without* an exception set if the key wasn't present.
+*/
 PyObject *
 _PyDict_GetItem_KnownHash(PyObject *op, PyObject *key, Py_hash_t hash)
 {
     Py_ssize_t ix;
     PyDictObject *mp = (PyDictObject *)op;
-    PyThreadState *tstate;
     PyObject **value_addr;
 
-    if (!PyDict_Check(op))
+    if (!PyDict_Check(op)) {
+        PyErr_BadInternalCall();
         return NULL;
-
-    /* We can arrive here with a NULL tstate during initialization: try
-       running "python -Wi" for an example related to string interning.
-       Let's just hope that no exception occurs then...  This must be
-       _PyThreadState_Current and not PyThreadState_GET() because in debug
-       mode, the latter complains if tstate is NULL. */
-    tstate = _PyThreadState_UncheckedGet();
-    if (tstate != NULL && tstate->curexc_type != NULL) {
-        /* preserve the existing exception */
-        PyObject *err_type, *err_value, *err_tb;
-        PyErr_Fetch(&err_type, &err_value, &err_tb);
-        ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL);
-        /* ignore errors */
-        PyErr_Restore(err_type, err_value, err_tb);
-        if (ix == DKIX_EMPTY)
-            return NULL;
     }
-    else {
-        ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL);
-        if (ix == DKIX_EMPTY) {
-            PyErr_Clear();
-            return NULL;
-        }
+
+    ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL);
+    if (ix < 0) {
+        return NULL;
     }
     return *value_addr;
 }
@@ -2431,8 +2419,16 @@
                 int err = 0;
                 Py_INCREF(key);
                 Py_INCREF(value);
-                if (override == 1 || _PyDict_GetItem_KnownHash(a, key, hash) == NULL)
+                if (override == 1)
                     err = insertdict(mp, key, hash, value);
+                else if (_PyDict_GetItem_KnownHash(a, key, hash) == NULL) {
+                    if (PyErr_Occurred()) {
+                        Py_DECREF(value);
+                        Py_DECREF(key);
+                        return -1;
+                    }
+                    err = insertdict(mp, key, hash, value);
+                }
                 else if (override != 0) {
                     _PyErr_SetKeyError(key);
                     Py_DECREF(value);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 06:20:03 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 06 Nov 2016 11:20:03 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogSXNzdWUgIzI4MTIzOiBfUHlEaWN0X0dldEl0ZW1fS25vd25IYXNoKCkg?=
 =?utf-8?q?now_can_raise_an_exception_as?=
Message-ID: <20161106111957.31271.35570.7972F045@psf.io>

https://hg.python.org/cpython/rev/805467de22fc
changeset:   104921:805467de22fc
parent:      104918:54087e63280a
parent:      104920:d06a6b0fd992
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 06 13:19:38 2016 +0200
summary:
  Issue #28123: _PyDict_GetItem_KnownHash() now can raise an exception as
PyDict_GetItemWithError().  Patch by Xiang Zhang.

files:
  Lib/test/test_dict.py        |  31 +++++++++++++++
  Modules/_collectionsmodule.c |   2 +
  Modules/_testcapimodule.c    |  21 ++++++++++
  Objects/dictobject.c         |  48 +++++++++++-------------
  4 files changed, 76 insertions(+), 26 deletions(-)


diff --git a/Lib/test/test_dict.py b/Lib/test/test_dict.py
--- a/Lib/test/test_dict.py
+++ b/Lib/test/test_dict.py
@@ -1055,6 +1055,37 @@
         support.check_free_after_iterating(self, lambda d: iter(d.values()), dict)
         support.check_free_after_iterating(self, lambda d: iter(d.items()), dict)
 
+
+class CAPITest(unittest.TestCase):
+
+    # Test _PyDict_GetItem_KnownHash()
+    @support.cpython_only
+    def test_getitem_knownhash(self):
+        from _testcapi import dict_getitem_knownhash
+
+        d = {'x': 1, 'y': 2, 'z': 3}
+        self.assertEqual(dict_getitem_knownhash(d, 'x', hash('x')), 1)
+        self.assertEqual(dict_getitem_knownhash(d, 'y', hash('y')), 2)
+        self.assertEqual(dict_getitem_knownhash(d, 'z', hash('z')), 3)
+
+        # not a dict
+        self.assertRaises(SystemError, dict_getitem_knownhash, [], 1, hash(1))
+        # key does not exist
+        self.assertRaises(KeyError, dict_getitem_knownhash, {}, 1, hash(1))
+
+        class Exc(Exception): pass
+        class BadEq:
+            def __eq__(self, other):
+                raise Exc
+            def __hash__(self):
+                return 7
+
+        k1, k2 = BadEq(), BadEq()
+        d = {k1: 1}
+        self.assertEqual(dict_getitem_knownhash(d, k1, hash(k1)), 1)
+        self.assertRaises(Exc, dict_getitem_knownhash, d, k2, hash(k2))
+
+
 from test import mapping_tests
 
 class GeneralMappingTests(mapping_tests.BasicTestMappingProtocol):
diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c
--- a/Modules/_collectionsmodule.c
+++ b/Modules/_collectionsmodule.c
@@ -2300,6 +2300,8 @@
 
             oldval = _PyDict_GetItem_KnownHash(mapping, key, hash);
             if (oldval == NULL) {
+                if (PyErr_Occurred())
+                    goto done;
                 if (_PyDict_SetItem_KnownHash(mapping, key, one, hash) < 0)
                     goto done;
             } else {
diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c
--- a/Modules/_testcapimodule.c
+++ b/Modules/_testcapimodule.c
@@ -238,6 +238,26 @@
     return Py_None;
 }
 
+static PyObject*
+dict_getitem_knownhash(PyObject *self, PyObject *args)
+{
+    PyObject *mp, *key, *result;
+    Py_ssize_t hash;
+
+    if (!PyArg_ParseTuple(args, "OOn:dict_getitem_knownhash",
+                          &mp, &key, &hash)) {
+        return NULL;
+    }
+
+    result = _PyDict_GetItem_KnownHash(mp, key, (Py_hash_t)hash);
+    if (result == NULL && !PyErr_Occurred()) {
+        _PyErr_SetKeyError(key);
+        return NULL;
+    }
+
+    Py_XINCREF(result);
+    return result;
+}
 
 /* Issue #4701: Check that PyObject_Hash implicitly calls
  *   PyType_Ready if it hasn't already been called
@@ -3999,6 +4019,7 @@
     {"test_datetime_capi",  test_datetime_capi,              METH_NOARGS},
     {"test_list_api",           (PyCFunction)test_list_api,      METH_NOARGS},
     {"test_dict_iteration",     (PyCFunction)test_dict_iteration,METH_NOARGS},
+    {"dict_getitem_knownhash",  dict_getitem_knownhash,          METH_VARARGS},
     {"test_lazy_hash_inheritance",      (PyCFunction)test_lazy_hash_inheritance,METH_NOARGS},
     {"test_long_api",           (PyCFunction)test_long_api,      METH_NOARGS},
     {"test_xincref_doesnt_leak",(PyCFunction)test_xincref_doesnt_leak,      METH_NOARGS},
diff --git a/Objects/dictobject.c b/Objects/dictobject.c
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -724,8 +724,10 @@
             Py_INCREF(startkey);
             cmp = PyObject_RichCompareBool(startkey, key, Py_EQ);
             Py_DECREF(startkey);
-            if (cmp < 0)
+            if (cmp < 0) {
+                *value_addr = NULL;
                 return DKIX_ERROR;
+            }
             if (dk == mp->ma_keys && ep->me_key == startkey) {
                 if (cmp > 0) {
                     *value_addr = &ep->me_value;
@@ -1425,39 +1427,25 @@
     return *value_addr;
 }
 
+/* Same as PyDict_GetItemWithError() but with hash supplied by caller.
+   This returns NULL *with* an exception set if an exception occurred.
+   It returns NULL *without* an exception set if the key wasn't present.
+*/
 PyObject *
 _PyDict_GetItem_KnownHash(PyObject *op, PyObject *key, Py_hash_t hash)
 {
     Py_ssize_t ix;
     PyDictObject *mp = (PyDictObject *)op;
-    PyThreadState *tstate;
     PyObject **value_addr;
 
-    if (!PyDict_Check(op))
+    if (!PyDict_Check(op)) {
+        PyErr_BadInternalCall();
         return NULL;
-
-    /* We can arrive here with a NULL tstate during initialization: try
-       running "python -Wi" for an example related to string interning.
-       Let's just hope that no exception occurs then...  This must be
-       _PyThreadState_Current and not PyThreadState_GET() because in debug
-       mode, the latter complains if tstate is NULL. */
-    tstate = _PyThreadState_UncheckedGet();
-    if (tstate != NULL && tstate->curexc_type != NULL) {
-        /* preserve the existing exception */
-        PyObject *err_type, *err_value, *err_tb;
-        PyErr_Fetch(&err_type, &err_value, &err_tb);
-        ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL);
-        /* ignore errors */
-        PyErr_Restore(err_type, err_value, err_tb);
-        if (ix == DKIX_EMPTY)
-            return NULL;
     }
-    else {
-        ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL);
-        if (ix == DKIX_EMPTY) {
-            PyErr_Clear();
-            return NULL;
-        }
+
+    ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL);
+    if (ix < 0) {
+        return NULL;
     }
     return *value_addr;
 }
@@ -2429,8 +2417,16 @@
                 int err = 0;
                 Py_INCREF(key);
                 Py_INCREF(value);
-                if (override == 1 || _PyDict_GetItem_KnownHash(a, key, hash) == NULL)
+                if (override == 1)
                     err = insertdict(mp, key, hash, value);
+                else if (_PyDict_GetItem_KnownHash(a, key, hash) == NULL) {
+                    if (PyErr_Occurred()) {
+                        Py_DECREF(value);
+                        Py_DECREF(key);
+                        return -1;
+                    }
+                    err = insertdict(mp, key, hash, value);
+                }
                 else if (override != 0) {
                     _PyErr_SetKeyError(key);
                     Py_DECREF(value);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 06:45:54 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 06 Nov 2016 11:45:54 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328586=3A_Converte?=
 =?utf-8?q?d_os=2Escandir=28=29_to_Argument_Clinic=2E?=
Message-ID: <20161106114554.111529.68412.0BFAF61C@psf.io>

https://hg.python.org/cpython/rev/0590b9c5a18e
changeset:   104922:0590b9c5a18e
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 06 13:45:33 2016 +0200
summary:
  Issue #28586: Converted os.scandir() to Argument Clinic.

files:
  Modules/clinic/posixmodule.c.h |  205 +++++++++++++++++++-
  Modules/posixmodule.c          |  219 ++++++++++----------
  2 files changed, 309 insertions(+), 115 deletions(-)


diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h
--- a/Modules/clinic/posixmodule.c.h
+++ b/Modules/clinic/posixmodule.c.h
@@ -5596,6 +5596,209 @@
 
 #endif /* defined(MS_WINDOWS) */
 
+PyDoc_STRVAR(os_DirEntry_is_symlink__doc__,
+"is_symlink($self, /)\n"
+"--\n"
+"\n"
+"Return True if the entry is a symbolic link; cached per entry.");
+
+#define OS_DIRENTRY_IS_SYMLINK_METHODDEF    \
+    {"is_symlink", (PyCFunction)os_DirEntry_is_symlink, METH_NOARGS, os_DirEntry_is_symlink__doc__},
+
+static int
+os_DirEntry_is_symlink_impl(DirEntry *self);
+
+static PyObject *
+os_DirEntry_is_symlink(DirEntry *self, PyObject *Py_UNUSED(ignored))
+{
+    PyObject *return_value = NULL;
+    int _return_value;
+
+    _return_value = os_DirEntry_is_symlink_impl(self);
+    if ((_return_value == -1) && PyErr_Occurred()) {
+        goto exit;
+    }
+    return_value = PyBool_FromLong((long)_return_value);
+
+exit:
+    return return_value;
+}
+
+PyDoc_STRVAR(os_DirEntry_stat__doc__,
+"stat($self, /, *, follow_symlinks=True)\n"
+"--\n"
+"\n"
+"Return stat_result object for the entry; cached per entry.");
+
+#define OS_DIRENTRY_STAT_METHODDEF    \
+    {"stat", (PyCFunction)os_DirEntry_stat, METH_FASTCALL, os_DirEntry_stat__doc__},
+
+static PyObject *
+os_DirEntry_stat_impl(DirEntry *self, int follow_symlinks);
+
+static PyObject *
+os_DirEntry_stat(DirEntry *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames)
+{
+    PyObject *return_value = NULL;
+    static const char * const _keywords[] = {"follow_symlinks", NULL};
+    static _PyArg_Parser _parser = {"|$p:stat", _keywords, 0};
+    int follow_symlinks = 1;
+
+    if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser,
+        &follow_symlinks)) {
+        goto exit;
+    }
+    return_value = os_DirEntry_stat_impl(self, follow_symlinks);
+
+exit:
+    return return_value;
+}
+
+PyDoc_STRVAR(os_DirEntry_is_dir__doc__,
+"is_dir($self, /, *, follow_symlinks=True)\n"
+"--\n"
+"\n"
+"Return True if the entry is a directory; cached per entry.");
+
+#define OS_DIRENTRY_IS_DIR_METHODDEF    \
+    {"is_dir", (PyCFunction)os_DirEntry_is_dir, METH_FASTCALL, os_DirEntry_is_dir__doc__},
+
+static int
+os_DirEntry_is_dir_impl(DirEntry *self, int follow_symlinks);
+
+static PyObject *
+os_DirEntry_is_dir(DirEntry *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames)
+{
+    PyObject *return_value = NULL;
+    static const char * const _keywords[] = {"follow_symlinks", NULL};
+    static _PyArg_Parser _parser = {"|$p:is_dir", _keywords, 0};
+    int follow_symlinks = 1;
+    int _return_value;
+
+    if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser,
+        &follow_symlinks)) {
+        goto exit;
+    }
+    _return_value = os_DirEntry_is_dir_impl(self, follow_symlinks);
+    if ((_return_value == -1) && PyErr_Occurred()) {
+        goto exit;
+    }
+    return_value = PyBool_FromLong((long)_return_value);
+
+exit:
+    return return_value;
+}
+
+PyDoc_STRVAR(os_DirEntry_is_file__doc__,
+"is_file($self, /, *, follow_symlinks=True)\n"
+"--\n"
+"\n"
+"Return True if the entry is a file; cached per entry.");
+
+#define OS_DIRENTRY_IS_FILE_METHODDEF    \
+    {"is_file", (PyCFunction)os_DirEntry_is_file, METH_FASTCALL, os_DirEntry_is_file__doc__},
+
+static int
+os_DirEntry_is_file_impl(DirEntry *self, int follow_symlinks);
+
+static PyObject *
+os_DirEntry_is_file(DirEntry *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames)
+{
+    PyObject *return_value = NULL;
+    static const char * const _keywords[] = {"follow_symlinks", NULL};
+    static _PyArg_Parser _parser = {"|$p:is_file", _keywords, 0};
+    int follow_symlinks = 1;
+    int _return_value;
+
+    if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser,
+        &follow_symlinks)) {
+        goto exit;
+    }
+    _return_value = os_DirEntry_is_file_impl(self, follow_symlinks);
+    if ((_return_value == -1) && PyErr_Occurred()) {
+        goto exit;
+    }
+    return_value = PyBool_FromLong((long)_return_value);
+
+exit:
+    return return_value;
+}
+
+PyDoc_STRVAR(os_DirEntry_inode__doc__,
+"inode($self, /)\n"
+"--\n"
+"\n"
+"Return inode of the entry; cached per entry.");
+
+#define OS_DIRENTRY_INODE_METHODDEF    \
+    {"inode", (PyCFunction)os_DirEntry_inode, METH_NOARGS, os_DirEntry_inode__doc__},
+
+static PyObject *
+os_DirEntry_inode_impl(DirEntry *self);
+
+static PyObject *
+os_DirEntry_inode(DirEntry *self, PyObject *Py_UNUSED(ignored))
+{
+    return os_DirEntry_inode_impl(self);
+}
+
+PyDoc_STRVAR(os_DirEntry___fspath____doc__,
+"__fspath__($self, /)\n"
+"--\n"
+"\n"
+"Returns the path for the entry.");
+
+#define OS_DIRENTRY___FSPATH___METHODDEF    \
+    {"__fspath__", (PyCFunction)os_DirEntry___fspath__, METH_NOARGS, os_DirEntry___fspath____doc__},
+
+static PyObject *
+os_DirEntry___fspath___impl(DirEntry *self);
+
+static PyObject *
+os_DirEntry___fspath__(DirEntry *self, PyObject *Py_UNUSED(ignored))
+{
+    return os_DirEntry___fspath___impl(self);
+}
+
+PyDoc_STRVAR(os_scandir__doc__,
+"scandir($module, /, path=None)\n"
+"--\n"
+"\n"
+"Return an iterator of DirEntry objects for given path.\n"
+"\n"
+"path can be specified as either str, bytes or path-like object.  If path\n"
+"is bytes, the names of yielded DirEntry objects will also be bytes; in\n"
+"all other circumstances they will be str.\n"
+"\n"
+"If path is None, uses the path=\'.\'.");
+
+#define OS_SCANDIR_METHODDEF    \
+    {"scandir", (PyCFunction)os_scandir, METH_FASTCALL, os_scandir__doc__},
+
+static PyObject *
+os_scandir_impl(PyObject *module, path_t *path);
+
+static PyObject *
+os_scandir(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames)
+{
+    PyObject *return_value = NULL;
+    static const char * const _keywords[] = {"path", NULL};
+    static _PyArg_Parser _parser = {"|O&:scandir", _keywords, 0};
+    path_t path = PATH_T_INITIALIZE("scandir", "path", 1, 0);
+
+    if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser,
+        path_converter, &path)) {
+        goto exit;
+    }
+    return_value = os_scandir_impl(module, &path);
+
+exit:
+    /* Cleanup for path */
+    path_cleanup(&path);
+
+    return return_value;
+}
+
 PyDoc_STRVAR(os_fspath__doc__,
 "fspath($module, /, path)\n"
 "--\n"
@@ -6148,4 +6351,4 @@
 #ifndef OS_GETRANDOM_METHODDEF
     #define OS_GETRANDOM_METHODDEF
 #endif /* !defined(OS_GETRANDOM_METHODDEF) */
-/*[clinic end generated code: output=b9ed5703d2feb0d9 input=a9049054013a1b77]*/
+/*[clinic end generated code: output=e4a3bd36c7bb8356 input=a9049054013a1b77]*/
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -11092,10 +11092,10 @@
 #endif   /* !MS_WINDOWS */
 
 
-PyDoc_STRVAR(posix_scandir__doc__,
-"scandir(path='.') -> iterator of DirEntry objects for given path");
-
-static char *follow_symlinks_keywords[] = {"follow_symlinks", NULL};
+/*[clinic input]
+class os.DirEntry "DirEntry *" "&DirEntryType"
+[clinic start generated code]*/
+/*[clinic end generated code: output=da39a3ee5e6b4b0d input=3138f09f7c683f1d]*/
 
 typedef struct {
     PyObject_HEAD
@@ -11129,9 +11129,15 @@
 static int
 DirEntry_test_mode(DirEntry *self, int follow_symlinks, unsigned short mode_bits);
 
-/* Set exception and return -1 on error, 0 for False, 1 for True */
+/*[clinic input]
+os.DirEntry.is_symlink -> bool
+
+Return True if the entry is a symbolic link; cached per entry.
+[clinic start generated code]*/
+
 static int
-DirEntry_is_symlink(DirEntry *self)
+os_DirEntry_is_symlink_impl(DirEntry *self)
+/*[clinic end generated code: output=42244667d7bcfc25 input=1605a1b4b96976c3]*/
 {
 #ifdef MS_WINDOWS
     return (self->win32_lstat.st_mode & S_IFMT) == S_IFLNK;
@@ -11148,17 +11154,6 @@
 }
 
 static PyObject *
-DirEntry_py_is_symlink(DirEntry *self)
-{
-    int result;
-
-    result = DirEntry_is_symlink(self);
-    if (result == -1)
-        return NULL;
-    return PyBool_FromLong(result);
-}
-
-static PyObject *
 DirEntry_fetch_stat(DirEntry *self, int follow_symlinks)
 {
     int result;
@@ -11200,14 +11195,23 @@
     return self->lstat;
 }
 
-static PyObject *
-DirEntry_get_stat(DirEntry *self, int follow_symlinks)
+/*[clinic input]
+os.DirEntry.stat
+    *
+    follow_symlinks: bool = True
+
+Return stat_result object for the entry; cached per entry.
+[clinic start generated code]*/
+
+static PyObject *
+os_DirEntry_stat_impl(DirEntry *self, int follow_symlinks)
+/*[clinic end generated code: output=008593b3a6d01305 input=280d14c1d6f1d00d]*/
 {
     if (!follow_symlinks)
         return DirEntry_get_lstat(self);
 
     if (!self->stat) {
-        int result = DirEntry_is_symlink(self);
+        int result = os_DirEntry_is_symlink_impl(self);
         if (result == -1)
             return NULL;
         else if (result)
@@ -11220,18 +11224,6 @@
     return self->stat;
 }
 
-static PyObject *
-DirEntry_stat(DirEntry *self, PyObject *args, PyObject *kwargs)
-{
-    int follow_symlinks = 1;
-
-    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|$p:DirEntry.stat",
-                                     follow_symlinks_keywords, &follow_symlinks))
-        return NULL;
-
-    return DirEntry_get_stat(self, follow_symlinks);
-}
-
 /* Set exception and return -1 on error, 0 for False, 1 for True */
 static int
 DirEntry_test_mode(DirEntry *self, int follow_symlinks, unsigned short mode_bits)
@@ -11260,7 +11252,7 @@
 #if defined(MS_WINDOWS) || defined(HAVE_DIRENT_D_TYPE)
     if (need_stat) {
 #endif
-        stat = DirEntry_get_stat(self, follow_symlinks);
+        stat = os_DirEntry_stat_impl(self, follow_symlinks);
         if (!stat) {
             if (PyErr_ExceptionMatches(PyExc_FileNotFoundError)) {
                 /* If file doesn't exist (anymore), then return False
@@ -11311,43 +11303,45 @@
     return -1;
 }
 
-static PyObject *
-DirEntry_py_test_mode(DirEntry *self, int follow_symlinks, unsigned short mode_bits)
-{
-    int result;
-
-    result = DirEntry_test_mode(self, follow_symlinks, mode_bits);
-    if (result == -1)
-        return NULL;
-    return PyBool_FromLong(result);
-}
-
-static PyObject *
-DirEntry_is_dir(DirEntry *self, PyObject *args, PyObject *kwargs)
-{
-    int follow_symlinks = 1;
-
-    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|$p:DirEntry.is_dir",
-                                     follow_symlinks_keywords, &follow_symlinks))
-        return NULL;
-
-    return DirEntry_py_test_mode(self, follow_symlinks, S_IFDIR);
-}
-
-static PyObject *
-DirEntry_is_file(DirEntry *self, PyObject *args, PyObject *kwargs)
-{
-    int follow_symlinks = 1;
-
-    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|$p:DirEntry.is_file",
-                                     follow_symlinks_keywords, &follow_symlinks))
-        return NULL;
-
-    return DirEntry_py_test_mode(self, follow_symlinks, S_IFREG);
-}
-
-static PyObject *
-DirEntry_inode(DirEntry *self)
+/*[clinic input]
+os.DirEntry.is_dir -> bool
+    *
+    follow_symlinks: bool = True
+
+Return True if the entry is a directory; cached per entry.
+[clinic start generated code]*/
+
+static int
+os_DirEntry_is_dir_impl(DirEntry *self, int follow_symlinks)
+/*[clinic end generated code: output=ad2e8d54365da287 input=0135232766f53f58]*/
+{
+    return DirEntry_test_mode(self, follow_symlinks, S_IFDIR);
+}
+
+/*[clinic input]
+os.DirEntry.is_file -> bool
+    *
+    follow_symlinks: bool = True
+
+Return True if the entry is a file; cached per entry.
+[clinic start generated code]*/
+
+static int
+os_DirEntry_is_file_impl(DirEntry *self, int follow_symlinks)
+/*[clinic end generated code: output=8462ade481d8a476 input=0dc90be168b041ee]*/
+{
+    return DirEntry_test_mode(self, follow_symlinks, S_IFREG);
+}
+
+/*[clinic input]
+os.DirEntry.inode
+
+Return inode of the entry; cached per entry.
+[clinic start generated code]*/
+
+static PyObject *
+os_DirEntry_inode_impl(DirEntry *self)
+/*[clinic end generated code: output=156bb3a72162440e input=3ee7b872ae8649f0]*/
 {
 #ifdef MS_WINDOWS
     if (!self->got_file_index) {
@@ -11384,8 +11378,15 @@
     return PyUnicode_FromFormat("<DirEntry %R>", self->name);
 }
 
-static PyObject *
-DirEntry_fspath(DirEntry *self)
+/*[clinic input]
+os.DirEntry.__fspath__
+
+Returns the path for the entry.
+[clinic start generated code]*/
+
+static PyObject *
+os_DirEntry___fspath___impl(DirEntry *self)
+/*[clinic end generated code: output=6dd7f7ef752e6f4f input=3c49d0cf38df4fac]*/
 {
     Py_INCREF(self->path);
     return self->path;
@@ -11399,25 +11400,15 @@
     {NULL}
 };
 
+#include "clinic/posixmodule.c.h"
+
 static PyMethodDef DirEntry_methods[] = {
-    {"is_dir", (PyCFunction)DirEntry_is_dir, METH_VARARGS | METH_KEYWORDS,
-     "return True if the entry is a directory; cached per entry"
-    },
-    {"is_file", (PyCFunction)DirEntry_is_file, METH_VARARGS | METH_KEYWORDS,
-     "return True if the entry is a file; cached per entry"
-    },
-    {"is_symlink", (PyCFunction)DirEntry_py_is_symlink, METH_NOARGS,
-     "return True if the entry is a symbolic link; cached per entry"
-    },
-    {"stat", (PyCFunction)DirEntry_stat, METH_VARARGS | METH_KEYWORDS,
-     "return stat_result object for the entry; cached per entry"
-    },
-    {"inode", (PyCFunction)DirEntry_inode, METH_NOARGS,
-     "return inode of the entry; cached per entry",
-    },
-    {"__fspath__", (PyCFunction)DirEntry_fspath, METH_NOARGS,
-     "returns the path for the entry",
-    },
+    OS_DIRENTRY_IS_DIR_METHODDEF
+    OS_DIRENTRY_IS_FILE_METHODDEF
+    OS_DIRENTRY_IS_SYMLINK_METHODDEF
+    OS_DIRENTRY_STAT_METHODDEF
+    OS_DIRENTRY_INODE_METHODDEF
+    OS_DIRENTRY___FSPATH___METHODDEF
     {NULL}
 };
 
@@ -11890,23 +11881,34 @@
     (destructor)ScandirIterator_finalize,   /* tp_finalize */
 };
 
-static PyObject *
-posix_scandir(PyObject *self, PyObject *args, PyObject *kwargs)
+/*[clinic input]
+os.scandir
+
+    path : path_t(nullable=True) = None
+
+Return an iterator of DirEntry objects for given path.
+
+path can be specified as either str, bytes or path-like object.  If path
+is bytes, the names of yielded DirEntry objects will also be bytes; in
+all other circumstances they will be str.
+
+If path is None, uses the path='.'.
+[clinic start generated code]*/
+
+static PyObject *
+os_scandir_impl(PyObject *module, path_t *path)
+/*[clinic end generated code: output=6eb2668b675ca89e input=e62b08b3cd41f604]*/
 {
     ScandirIterator *iterator;
-    static char *keywords[] = {"path", NULL};
 #ifdef MS_WINDOWS
     wchar_t *path_strW;
 #else
-    const char *path;
+    const char *path_str;
 #endif
 
     iterator = PyObject_New(ScandirIterator, &ScandirIteratorType);
     if (!iterator)
         return NULL;
-    memset(&iterator->path, 0, sizeof(path_t));
-    iterator->path.function_name = "scandir";
-    iterator->path.nullable = 1;
 
 #ifdef MS_WINDOWS
     iterator->handle = INVALID_HANDLE_VALUE;
@@ -11914,15 +11916,13 @@
     iterator->dirp = NULL;
 #endif
 
-    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O&:scandir", keywords,
-                                     path_converter, &iterator->path))
-        goto error;
-
+    memcpy(&iterator->path, path, sizeof(path_t));
     /* path_converter doesn't keep path.object around, so do it
        manually for the lifetime of the iterator here (the refcount
        is decremented in ScandirIterator_dealloc)
     */
     Py_XINCREF(iterator->path.object);
+    Py_XINCREF(iterator->path.cleanup);
 
 #ifdef MS_WINDOWS
     iterator->first_time = 1;
@@ -11943,13 +11943,13 @@
     }
 #else /* POSIX */
     if (iterator->path.narrow)
-        path = iterator->path.narrow;
+        path_str = iterator->path.narrow;
     else
-        path = ".";
+        path_str = ".";
 
     errno = 0;
     Py_BEGIN_ALLOW_THREADS
-    iterator->dirp = opendir(path);
+    iterator->dirp = opendir(path_str);
     Py_END_ALLOW_THREADS
 
     if (!iterator->dirp) {
@@ -12092,13 +12092,6 @@
 }
 #endif   /* HAVE_GETRANDOM_SYSCALL */
 
-#include "clinic/posixmodule.c.h"
-
-/*[clinic input]
-dump buffer
-[clinic start generated code]*/
-/*[clinic end generated code: output=da39a3ee5e6b4b0d input=524ce2e021e4eba6]*/
-
 
 static PyMethodDef posix_methods[] = {
 
@@ -12288,9 +12281,7 @@
     {"get_blocking", posix_get_blocking, METH_VARARGS, get_blocking__doc__},
     {"set_blocking", posix_set_blocking, METH_VARARGS, set_blocking__doc__},
 #endif
-    {"scandir",         (PyCFunction)posix_scandir,
-                        METH_VARARGS | METH_KEYWORDS,
-                        posix_scandir__doc__},
+    OS_SCANDIR_METHODDEF
     OS_FSPATH_METHODDEF
     OS_GETRANDOM_METHODDEF
     {NULL,              NULL}            /* Sentinel */

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 10:24:06 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 06 Nov 2016 15:24:06 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328199=3A_Microopt?=
 =?utf-8?q?imized_dict_resizing=2E__Based_on_patch_by_Naoki_Inada=2E?=
Message-ID: <20161106152405.30849.46194.F01ABD25@psf.io>

https://hg.python.org/cpython/rev/39f33c15243b
changeset:   104923:39f33c15243b
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sat Oct 29 10:50:00 2016 +0300
summary:
  Issue #28199: Microoptimized dict resizing.  Based on patch by Naoki Inada.

files:
  Objects/dictobject.c |  123 +++++++++++++++---------------
  1 files changed, 63 insertions(+), 60 deletions(-)


diff --git a/Objects/dictobject.c b/Objects/dictobject.c
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -1198,41 +1198,21 @@
 }
 
 /*
-Internal routine used by dictresize() to insert an item which is
-known to be absent from the dict.  This routine also assumes that
-the dict contains no deleted entries.  Besides the performance benefit,
-using insertdict() in dictresize() is dangerous (SF bug #1456209).
-Note that no refcounts are changed by this routine; if needed, the caller
-is responsible for incref'ing `key` and `value`.
-Neither mp->ma_used nor k->dk_usable are modified by this routine; the caller
-must set them correctly
+Internal routine used by dictresize() to buid a hashtable of entries.
 */
 static void
-insertdict_clean(PyDictObject *mp, PyObject *key, Py_hash_t hash,
-                 PyObject *value)
+build_indices(PyDictKeysObject *keys, PyDictKeyEntry *ep, Py_ssize_t n)
 {
-    size_t i;
-    PyDictKeysObject *k = mp->ma_keys;
-    size_t mask = (size_t)DK_SIZE(k)-1;
-    PyDictKeyEntry *ep0 = DK_ENTRIES(mp->ma_keys);
-    PyDictKeyEntry *ep;
-
-    assert(k->dk_lookup != NULL);
-    assert(value != NULL);
-    assert(key != NULL);
-    assert(PyUnicode_CheckExact(key) || k->dk_lookup == lookdict);
-    i = hash & mask;
-    for (size_t perturb = hash; dk_get_index(k, i) != DKIX_EMPTY;) {
-        perturb >>= PERTURB_SHIFT;
-        i = mask & ((i << 2) + i + perturb + 1);
+    size_t mask = (size_t)DK_SIZE(keys) - 1;
+    for (Py_ssize_t ix = 0; ix != n; ix++, ep++) {
+        Py_hash_t hash = ep->me_hash;
+        size_t i = hash & mask;
+        for (size_t perturb = hash; dk_get_index(keys, i) != DKIX_EMPTY;) {
+            perturb >>= PERTURB_SHIFT;
+            i = mask & ((i << 2) + i + perturb + 1);
+        }
+        dk_set_index(keys, i, ix);
     }
-    ep = &ep0[k->dk_nentries];
-    assert(ep->me_value == NULL);
-    dk_set_index(k, i, k->dk_nentries);
-    k->dk_nentries++;
-    ep->me_key = key;
-    ep->me_hash = hash;
-    ep->me_value = value;
 }
 
 /*
@@ -1248,10 +1228,10 @@
 static int
 dictresize(PyDictObject *mp, Py_ssize_t minused)
 {
-    Py_ssize_t i, newsize;
+    Py_ssize_t newsize, numentries;
     PyDictKeysObject *oldkeys;
     PyObject **oldvalues;
-    PyDictKeyEntry *ep0;
+    PyDictKeyEntry *oldentries, *newentries;
 
     /* Find the smallest table size > minused. */
     for (newsize = PyDict_MINSIZE;
@@ -1262,8 +1242,14 @@
         PyErr_NoMemory();
         return -1;
     }
+
     oldkeys = mp->ma_keys;
-    oldvalues = mp->ma_values;
+
+    /* NOTE: Current odict checks mp->ma_keys to detect resize happen.
+     * So we can't reuse oldkeys even if oldkeys->dk_size == newsize.
+     * TODO: Try reusing oldkeys when reimplement odict.
+     */
+
     /* Allocate a new table. */
     mp->ma_keys = new_keys_object(newsize);
     if (mp->ma_keys == NULL) {
@@ -1272,42 +1258,59 @@
     }
     if (oldkeys->dk_lookup == lookdict)
         mp->ma_keys->dk_lookup = lookdict;
-    mp->ma_values = NULL;
-    ep0 = DK_ENTRIES(oldkeys);
-    /* Main loop below assumes we can transfer refcount to new keys
-     * and that value is stored in me_value.
-     * Increment ref-counts and copy values here to compensate
-     * This (resizing a split table) should be relatively rare */
+
+    numentries = mp->ma_used;
+    oldentries = DK_ENTRIES(oldkeys);
+    newentries = DK_ENTRIES(mp->ma_keys);
+    oldvalues = mp->ma_values;
     if (oldvalues != NULL) {
-        for (i = 0; i < oldkeys->dk_nentries; i++) {
-            if (oldvalues[i] != NULL) {
-                Py_INCREF(ep0[i].me_key);
-                ep0[i].me_value = oldvalues[i];
-            }
+        /* Convert split table into new combined table.
+         * We must incref keys; we can transfer values.
+         * Note that values of split table is always dense.
+         */
+        for (Py_ssize_t i = 0; i < numentries; i++) {
+            assert(oldvalues[i] != NULL);
+            PyDictKeyEntry *ep = &oldentries[i];
+            PyObject *key = ep->me_key;
+            Py_INCREF(key);
+            newentries[i].me_key = key;
+            newentries[i].me_hash = ep->me_hash;
+            newentries[i].me_value = oldvalues[i];
         }
-    }
-    /* Main loop */
-    for (i = 0; i < oldkeys->dk_nentries; i++) {
-        PyDictKeyEntry *ep = &ep0[i];
-        if (ep->me_value != NULL) {
-            insertdict_clean(mp, ep->me_key, ep->me_hash, ep->me_value);
-        }
-    }
-    mp->ma_keys->dk_usable -= mp->ma_used;
-    if (oldvalues != NULL) {
-        /* NULL out me_value slot in oldkeys, in case it was shared */
-        for (i = 0; i < oldkeys->dk_nentries; i++)
-            ep0[i].me_value = NULL;
+
         DK_DECREF(oldkeys);
+        mp->ma_values = NULL;
         if (oldvalues != empty_values) {
             free_values(oldvalues);
         }
     }
-    else {
+    else {  // combined table.
+        if (oldkeys->dk_nentries == numentries) {
+            memcpy(newentries, oldentries, numentries * sizeof(PyDictKeyEntry));
+        }
+        else {
+            PyDictKeyEntry *ep = oldentries;
+            for (Py_ssize_t i = 0; i < numentries; i++) {
+                while (ep->me_value == NULL)
+                    ep++;
+                newentries[i] = *ep++;
+            }
+        }
+
         assert(oldkeys->dk_lookup != lookdict_split);
         assert(oldkeys->dk_refcnt == 1);
-        DK_DEBUG_DECREF PyObject_FREE(oldkeys);
+        if (oldkeys->dk_size == PyDict_MINSIZE &&
+            numfreekeys < PyDict_MAXFREELIST) {
+            DK_DEBUG_DECREF keys_free_list[numfreekeys++] = oldkeys;
+        }
+        else {
+            DK_DEBUG_DECREF PyObject_FREE(oldkeys);
+        }
     }
+
+    build_indices(mp->ma_keys, newentries, numentries);
+    mp->ma_keys->dk_usable -= numentries;
+    mp->ma_keys->dk_nentries = numentries;
     return 0;
 }
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 11:14:59 2016
From: python-checkins at python.org (jason.coombs)
Date: Sun, 06 Nov 2016 16:14:59 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Additionally_s?=
 =?utf-8?q?how_that_a_backslash-escaped_opening_brace_is_treated_as_a?=
Message-ID: <20161106161459.7061.99676.74BD04F2@psf.io>

https://hg.python.org/cpython/rev/1d8b8a67b657
changeset:   104925:1d8b8a67b657
branch:      3.6
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Sun Nov 06 11:14:48 2016 -0500
summary:
  Additionally show that a backslash-escaped opening brace is treated as a literal and thus triggers the single closing brace error, clarifying #28590.

files:
  Lib/test/test_fstring.py |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py
--- a/Lib/test/test_fstring.py
+++ b/Lib/test/test_fstring.py
@@ -628,6 +628,7 @@
                              "f'}'",
                              "f'x}'",
                              "f'x}x'",
+                             r"f'\u007b}'",
 
                              # Can't have { or } in a format spec.
                              "f'{3:}>10}'",

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 11:14:59 2016
From: python-checkins at python.org (jason.coombs)
Date: Sun, 06 Nov 2016 16:14:59 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Add_an_additio?=
 =?utf-8?q?nal_test_with_a_newline=2C_one_that=27s_very_similar_to?=
Message-ID: <20161106161459.30814.52366.D976C4E5@psf.io>

https://hg.python.org/cpython/rev/b04347ffa283
changeset:   104924:b04347ffa283
branch:      3.6
parent:      104920:d06a6b0fd992
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Sun Nov 06 11:01:08 2016 -0500
summary:
  Add an additional test with a newline, one that's very similar to test_parens_in_expressions, but because the newline is not a literal newline, but a backslash en, this error is triggered.

files:
  Lib/test/test_fstring.py |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py
--- a/Lib/test/test_fstring.py
+++ b/Lib/test/test_fstring.py
@@ -378,6 +378,7 @@
                              r"rf'{\t3}'",
                              r"rf'{\}'",
                              r"""rf'{"\N{LEFT CURLY BRACKET}"}'""",
+                             r"f'{\n}'",
                              ])
 
     def test_no_escapes_for_braces(self):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 11:16:47 2016
From: python-checkins at python.org (jason.coombs)
Date: Sun, 06 Nov 2016 16:16:47 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_with_3=2E6?=
Message-ID: <20161106161647.29950.17038.93DDE2E5@psf.io>

https://hg.python.org/cpython/rev/5fda909c722c
changeset:   104926:5fda909c722c
parent:      104923:39f33c15243b
parent:      104925:1d8b8a67b657
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Sun Nov 06 11:16:42 2016 -0500
summary:
  Merge with 3.6

files:
  Lib/test/test_fstring.py |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py
--- a/Lib/test/test_fstring.py
+++ b/Lib/test/test_fstring.py
@@ -378,6 +378,7 @@
                              r"rf'{\t3}'",
                              r"rf'{\}'",
                              r"""rf'{"\N{LEFT CURLY BRACKET}"}'""",
+                             r"f'{\n}'",
                              ])
 
     def test_no_escapes_for_braces(self):
@@ -627,6 +628,7 @@
                              "f'}'",
                              "f'x}'",
                              "f'x}x'",
+                             r"f'\u007b}'",
 
                              # Can't have { or } in a format spec.
                              "f'{3:}>10}'",

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 11:27:39 2016
From: python-checkins at python.org (jason.coombs)
Date: Sun, 06 Nov 2016 16:27:39 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogVXBkYXRlIHRlc3Rf?=
 =?utf-8?q?no=5Fescapes=5Ffor=5Fbraces_to_clarify_behavior_with_a_docstrin?=
 =?utf-8?q?g_and?=
Message-ID: <20161106162739.111599.68819.6FF1DC0F@psf.io>

https://hg.python.org/cpython/rev/f18d6eb91b06
changeset:   104927:f18d6eb91b06
branch:      3.6
parent:      104925:1d8b8a67b657
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Sun Nov 06 11:25:54 2016 -0500
summary:
  Update test_no_escapes_for_braces to clarify behavior with a docstring and expressions that clearly are not evaluated.

files:
  Lib/test/test_fstring.py |  13 ++++++++-----
  1 files changed, 8 insertions(+), 5 deletions(-)


diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py
--- a/Lib/test/test_fstring.py
+++ b/Lib/test/test_fstring.py
@@ -382,11 +382,14 @@
                              ])
 
     def test_no_escapes_for_braces(self):
-        # \x7b is '{'.  Make sure it doesn't start an expression.
-        self.assertEqual(f'\x7b2}}', '{2}')
-        self.assertEqual(f'\x7b2', '{2')
-        self.assertEqual(f'\u007b2', '{2')
-        self.assertEqual(f'\N{LEFT CURLY BRACKET}2\N{RIGHT CURLY BRACKET}', '{2}')
+        """
+        Only literal curly braces begin an expression.
+        """
+        # \x7b is '{'.
+        self.assertEqual(f'\x7b1+1}}', '{1+1}')
+        self.assertEqual(f'\x7b1+1', '{1+1')
+        self.assertEqual(f'\u007b1+1', '{1+1')
+        self.assertEqual(f'\N{LEFT CURLY BRACKET}1+1\N{RIGHT CURLY BRACKET}', '{1+1}')
 
     def test_newlines_in_expressions(self):
         self.assertEqual(f'{0}', '0')

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 11:27:39 2016
From: python-checkins at python.org (jason.coombs)
Date: Sun, 06 Nov 2016 16:27:39 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Update_docs_to?=
 =?utf-8?q?_reflect_new_behavior_around_backslashes_in_expressions_=28not?=
Message-ID: <20161106162739.29926.65998.A6C7CA66@psf.io>

https://hg.python.org/cpython/rev/d9eb609be2ca
changeset:   104928:d9eb609be2ca
branch:      3.6
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Sun Nov 06 11:27:17 2016 -0500
summary:
  Update docs to reflect new behavior around backslashes in expressions (not allowed), matching recent changes to PEP 498.

files:
  Doc/reference/lexical_analysis.rst |  19 +++++++++++------
  1 files changed, 12 insertions(+), 7 deletions(-)


diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst
--- a/Doc/reference/lexical_analysis.rst
+++ b/Doc/reference/lexical_analysis.rst
@@ -679,17 +679,22 @@
 
 A consequence of sharing the same syntax as regular string literals is
 that characters in the replacement fields must not conflict with the
-quoting used in the outer formatted string literal.  Also, escape
-sequences normally apply to the outer formatted string literal,
-rather than inner string literals::
+quoting used in the outer formatted string literal::
 
    f"abc {a["x"]} def"    # error: outer string literal ended prematurely
-   f"abc {a[\"x\"]} def"  # workaround: escape the inner quotes
    f"abc {a['x']} def"    # workaround: use different quoting
 
-   f"newline: {ord('\n')}"   # error: literal line break in inner string
-   f"newline: {ord('\\n')}"  # workaround: double escaping
-   fr"newline: {ord('\n')}"  # workaround: raw outer string
+Backslashes are not allowed in format expressions and will raise
+an error::
+
+   f"newline: {ord('\n')}"  # raises SyntaxError
+
+To include a value in which a backslash escape is required, create
+a temporary variable.
+
+   >>> newline = ord('\n')
+   >>> f"newline: {newline}"
+   'newline: 10'
 
 See also :pep:`498` for the proposal that added formatted string literals,
 and :meth:`str.format`, which uses a related format string mechanism.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 11:27:58 2016
From: python-checkins at python.org (jason.coombs)
Date: Sun, 06 Nov 2016 16:27:58 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_with_3=2E6?=
Message-ID: <20161106162758.111566.58592.BE761124@psf.io>

https://hg.python.org/cpython/rev/f332032b1934
changeset:   104929:f332032b1934
parent:      104926:5fda909c722c
parent:      104928:d9eb609be2ca
user:        Jason R. Coombs <jaraco at jaraco.com>
date:        Sun Nov 06 11:27:50 2016 -0500
summary:
  Merge with 3.6

files:
  Doc/reference/lexical_analysis.rst |  19 +++++++++++------
  Lib/test/test_fstring.py           |  13 +++++++----
  2 files changed, 20 insertions(+), 12 deletions(-)


diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst
--- a/Doc/reference/lexical_analysis.rst
+++ b/Doc/reference/lexical_analysis.rst
@@ -679,17 +679,22 @@
 
 A consequence of sharing the same syntax as regular string literals is
 that characters in the replacement fields must not conflict with the
-quoting used in the outer formatted string literal.  Also, escape
-sequences normally apply to the outer formatted string literal,
-rather than inner string literals::
+quoting used in the outer formatted string literal::
 
    f"abc {a["x"]} def"    # error: outer string literal ended prematurely
-   f"abc {a[\"x\"]} def"  # workaround: escape the inner quotes
    f"abc {a['x']} def"    # workaround: use different quoting
 
-   f"newline: {ord('\n')}"   # error: literal line break in inner string
-   f"newline: {ord('\\n')}"  # workaround: double escaping
-   fr"newline: {ord('\n')}"  # workaround: raw outer string
+Backslashes are not allowed in format expressions and will raise
+an error::
+
+   f"newline: {ord('\n')}"  # raises SyntaxError
+
+To include a value in which a backslash escape is required, create
+a temporary variable.
+
+   >>> newline = ord('\n')
+   >>> f"newline: {newline}"
+   'newline: 10'
 
 See also :pep:`498` for the proposal that added formatted string literals,
 and :meth:`str.format`, which uses a related format string mechanism.
diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py
--- a/Lib/test/test_fstring.py
+++ b/Lib/test/test_fstring.py
@@ -382,11 +382,14 @@
                              ])
 
     def test_no_escapes_for_braces(self):
-        # \x7b is '{'.  Make sure it doesn't start an expression.
-        self.assertEqual(f'\x7b2}}', '{2}')
-        self.assertEqual(f'\x7b2', '{2')
-        self.assertEqual(f'\u007b2', '{2')
-        self.assertEqual(f'\N{LEFT CURLY BRACKET}2\N{RIGHT CURLY BRACKET}', '{2}')
+        """
+        Only literal curly braces begin an expression.
+        """
+        # \x7b is '{'.
+        self.assertEqual(f'\x7b1+1}}', '{1+1}')
+        self.assertEqual(f'\x7b1+1', '{1+1')
+        self.assertEqual(f'\u007b1+1', '{1+1')
+        self.assertEqual(f'\N{LEFT CURLY BRACKET}1+1\N{RIGHT CURLY BRACKET}', '{1+1}')
 
     def test_newlines_in_expressions(self):
         self.assertEqual(f'{0}', '0')

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 11:48:03 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 06 Nov 2016 16:48:03 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIzOTk2?=
 =?utf-8?q?=3A_Added_=5FPyGen=5FSetStopIterationValue_for_safe_raising?=
Message-ID: <20161106164802.31966.15005.9B4D8653@psf.io>

https://hg.python.org/cpython/rev/bce18f5c0bc4
changeset:   104930:bce18f5c0bc4
branch:      3.5
parent:      104915:8448152f64d7
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 06 18:44:42 2016 +0200
summary:
  Issue #23996: Added _PyGen_SetStopIterationValue for safe raising
StopIteration with value. More safely handle non-normalized exceptions
in -_PyGen_FetchStopIterationValue.

files:
  Include/genobject.h         |   1 +
  Lib/test/test_coroutines.py |  61 +++++++++++++++++++++++++
  Lib/test/test_generators.py |  21 ++++++++
  Objects/genobject.c         |  60 ++++++++++++++++++++----
  4 files changed, 133 insertions(+), 10 deletions(-)


diff --git a/Include/genobject.h b/Include/genobject.h
--- a/Include/genobject.h
+++ b/Include/genobject.h
@@ -41,6 +41,7 @@
 PyAPI_FUNC(PyObject *) PyGen_NewWithQualName(struct _frame *,
     PyObject *name, PyObject *qualname);
 PyAPI_FUNC(int) PyGen_NeedsFinalizing(PyGenObject *);
+PyAPI_FUNC(int) _PyGen_SetStopIterationValue(PyObject *);
 PyAPI_FUNC(int) _PyGen_FetchStopIterationValue(PyObject **);
 PyObject *_PyGen_Send(PyGenObject *, PyObject *);
 PyObject *_PyGen_yf(PyGenObject *);
diff --git a/Lib/test/test_coroutines.py b/Lib/test/test_coroutines.py
--- a/Lib/test/test_coroutines.py
+++ b/Lib/test/test_coroutines.py
@@ -710,6 +710,21 @@
             coro.close()
             self.assertEqual(CHK, 1)
 
+    def test_coro_wrapper_send_tuple(self):
+        async def foo():
+            return (10,)
+
+        result = run_async__await__(foo())
+        self.assertEqual(result, ([], (10,)))
+
+    def test_coro_wrapper_send_stop_iterator(self):
+        async def foo():
+            return StopIteration(10)
+
+        result = run_async__await__(foo())
+        self.assertIsInstance(result[1], StopIteration)
+        self.assertEqual(result[1].value, 10)
+
     def test_cr_await(self):
         @types.coroutine
         def a():
@@ -1537,6 +1552,52 @@
                 warnings.simplefilter("error")
                 run_async(foo())
 
+    def test_for_tuple(self):
+        class Done(Exception): pass
+
+        class AIter(tuple):
+            i = 0
+            def __aiter__(self):
+                return self
+            async def __anext__(self):
+                if self.i >= len(self):
+                    raise StopAsyncIteration
+                self.i += 1
+                return self[self.i - 1]
+
+        result = []
+        async def foo():
+            async for i in AIter([42]):
+                result.append(i)
+            raise Done
+
+        with self.assertRaises(Done):
+            foo().send(None)
+        self.assertEqual(result, [42])
+
+    def test_for_stop_iteration(self):
+        class Done(Exception): pass
+
+        class AIter(StopIteration):
+            i = 0
+            def __aiter__(self):
+                return self
+            async def __anext__(self):
+                if self.i:
+                    raise StopAsyncIteration
+                self.i += 1
+                return self.value
+
+        result = []
+        async def foo():
+            async for i in AIter(42):
+                result.append(i)
+            raise Done
+
+        with self.assertRaises(Done):
+            foo().send(None)
+        self.assertEqual(result, [42])
+
     def test_copy(self):
         async def func(): pass
         coro = func()
diff --git a/Lib/test/test_generators.py b/Lib/test/test_generators.py
--- a/Lib/test/test_generators.py
+++ b/Lib/test/test_generators.py
@@ -277,6 +277,27 @@
             # hence no warning.
             next(g)
 
+    def test_return_tuple(self):
+        def g():
+            return (yield 1)
+
+        gen = g()
+        self.assertEqual(next(gen), 1)
+        with self.assertRaises(StopIteration) as cm:
+            gen.send((2,))
+        self.assertEqual(cm.exception.value, (2,))
+
+    def test_return_stopiteration(self):
+        def g():
+            return (yield 1)
+
+        gen = g()
+        self.assertEqual(next(gen), 1)
+        with self.assertRaises(StopIteration) as cm:
+            gen.send(StopIteration(2))
+        self.assertIsInstance(cm.exception.value, StopIteration)
+        self.assertEqual(cm.exception.value.value, 2)
+
 
 class YieldFromTests(unittest.TestCase):
     def test_generator_gi_yieldfrom(self):
diff --git a/Objects/genobject.c b/Objects/genobject.c
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -154,12 +154,7 @@
             /* Delay exception instantiation if we can */
             PyErr_SetNone(PyExc_StopIteration);
         } else {
-            PyObject *e = PyObject_CallFunctionObjArgs(
-                               PyExc_StopIteration, result, NULL);
-            if (e != NULL) {
-                PyErr_SetObject(PyExc_StopIteration, e);
-                Py_DECREF(e);
-            }
+            _PyGen_SetStopIterationValue(result);
         }
         Py_CLEAR(result);
     }
@@ -460,6 +455,43 @@
 }
 
 /*
+ * Set StopIteration with specified value.  Value can be arbitrary object
+ * or NULL.
+ *
+ * Returns 0 if StopIteration is set and -1 if any other exception is set.
+ */
+int
+_PyGen_SetStopIterationValue(PyObject *value)
+{
+    PyObject *e;
+
+    if (value == NULL ||
+        (!PyTuple_Check(value) &&
+         !PyObject_TypeCheck(value, (PyTypeObject *) PyExc_StopIteration)))
+    {
+        /* Delay exception instantiation if we can */
+        PyErr_SetObject(PyExc_StopIteration, value);
+        return 0;
+    }
+    /* Construct an exception instance manually with
+     * PyObject_CallFunctionObjArgs and pass it to PyErr_SetObject.
+     *
+     * We do this to handle a situation when "value" is a tuple, in which
+     * case PyErr_SetObject would set the value of StopIteration to
+     * the first element of the tuple.
+     *
+     * (See PyErr_SetObject/_PyErr_CreateException code for details.)
+     */
+    e = PyObject_CallFunctionObjArgs(PyExc_StopIteration, value, NULL);
+    if (e == NULL) {
+        return -1;
+    }
+    PyErr_SetObject(PyExc_StopIteration, e);
+    Py_DECREF(e);
+    return 0;
+}
+
+/*
  *   If StopIteration exception is set, fetches its 'value'
  *   attribute if any, otherwise sets pvalue to None.
  *
@@ -469,7 +501,8 @@
  */
 
 int
-_PyGen_FetchStopIterationValue(PyObject **pvalue) {
+_PyGen_FetchStopIterationValue(PyObject **pvalue)
+{
     PyObject *et, *ev, *tb;
     PyObject *value = NULL;
 
@@ -481,8 +514,15 @@
                 value = ((PyStopIterationObject *)ev)->value;
                 Py_INCREF(value);
                 Py_DECREF(ev);
-            } else if (et == PyExc_StopIteration) {
-                /* avoid normalisation and take ev as value */
+            } else if (et == PyExc_StopIteration && !PyTuple_Check(ev)) {
+                /* Avoid normalisation and take ev as value.
+                 *
+                 * Normalization is required if the value is a tuple, in
+                 * that case the value of StopIteration would be set to
+                 * the first element of the tuple.
+                 *
+                 * (See _PyErr_CreateException code for details.)
+                 */
                 value = ev;
             } else {
                 /* normalisation required */
@@ -1012,7 +1052,7 @@
 static PyObject *
 aiter_wrapper_iternext(PyAIterWrapper *aw)
 {
-    PyErr_SetObject(PyExc_StopIteration, aw->aw_aiter);
+    _PyGen_SetStopIterationValue(aw->aw_aiter);
     return NULL;
 }
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 11:48:03 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 06 Nov 2016 16:48:03 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2323996=3A_Added_=5FPyGen=5FSetStopIterationValue_for_s?=
 =?utf-8?q?afe_raising?=
Message-ID: <20161106164803.38763.57697.97820EDD@psf.io>

https://hg.python.org/cpython/rev/a2c9f06ada28
changeset:   104931:a2c9f06ada28
branch:      3.6
parent:      104928:d9eb609be2ca
parent:      104930:bce18f5c0bc4
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 06 18:47:03 2016 +0200
summary:
  Issue #23996: Added _PyGen_SetStopIterationValue for safe raising
StopIteration with value. More safely handle non-normalized exceptions
in -_PyGen_FetchStopIterationValue.

files:
  Include/genobject.h         |   1 +
  Lib/test/test_asyncgen.py   |  79 +++++++++++++++++++++
  Lib/test/test_coroutines.py |  61 ++++++++++++++++
  Lib/test/test_generators.py |  21 +++++
  Lib/test/test_yield_from.py |  93 +++++++++++++++++-------
  Modules/_asynciomodule.c    |  22 +----
  Objects/genobject.c         |  72 +++++++++++++-----
  7 files changed, 281 insertions(+), 68 deletions(-)


diff --git a/Include/genobject.h b/Include/genobject.h
--- a/Include/genobject.h
+++ b/Include/genobject.h
@@ -41,6 +41,7 @@
 PyAPI_FUNC(PyObject *) PyGen_NewWithQualName(struct _frame *,
     PyObject *name, PyObject *qualname);
 PyAPI_FUNC(int) PyGen_NeedsFinalizing(PyGenObject *);
+PyAPI_FUNC(int) _PyGen_SetStopIterationValue(PyObject *);
 PyAPI_FUNC(int) _PyGen_FetchStopIterationValue(PyObject **);
 PyAPI_FUNC(PyObject *) _PyGen_Send(PyGenObject *, PyObject *);
 PyObject *_PyGen_yf(PyGenObject *);
diff --git a/Lib/test/test_asyncgen.py b/Lib/test/test_asyncgen.py
--- a/Lib/test/test_asyncgen.py
+++ b/Lib/test/test_asyncgen.py
@@ -450,6 +450,48 @@
 
         self.loop.run_until_complete(run())
 
+    def test_async_gen_asyncio_anext_tuple(self):
+        async def foo():
+            try:
+                yield (1,)
+            except ZeroDivisionError:
+                yield (2,)
+
+        async def run():
+            it = foo().__aiter__()
+
+            self.assertEqual(await it.__anext__(), (1,))
+            with self.assertRaises(StopIteration) as cm:
+                it.__anext__().throw(ZeroDivisionError)
+            self.assertEqual(cm.exception.args[0], (2,))
+            with self.assertRaises(StopAsyncIteration):
+                await it.__anext__()
+
+        self.loop.run_until_complete(run())
+
+    def test_async_gen_asyncio_anext_stopiteration(self):
+        async def foo():
+            try:
+                yield StopIteration(1)
+            except ZeroDivisionError:
+                yield StopIteration(3)
+
+        async def run():
+            it = foo().__aiter__()
+
+            v = await it.__anext__()
+            self.assertIsInstance(v, StopIteration)
+            self.assertEqual(v.value, 1)
+            with self.assertRaises(StopIteration) as cm:
+                it.__anext__().throw(ZeroDivisionError)
+            v = cm.exception.args[0]
+            self.assertIsInstance(v, StopIteration)
+            self.assertEqual(v.value, 3)
+            with self.assertRaises(StopAsyncIteration):
+                await it.__anext__()
+
+        self.loop.run_until_complete(run())
+
     def test_async_gen_asyncio_aclose_06(self):
         async def foo():
             try:
@@ -759,6 +801,43 @@
             self.loop.run_until_complete(run())
         self.assertEqual(DONE, 1)
 
+    def test_async_gen_asyncio_athrow_tuple(self):
+        async def gen():
+            try:
+                yield 1
+            except ZeroDivisionError:
+                yield (2,)
+
+        async def run():
+            g = gen()
+            v = await g.asend(None)
+            self.assertEqual(v, 1)
+            v = await g.athrow(ZeroDivisionError)
+            self.assertEqual(v, (2,))
+            with self.assertRaises(StopAsyncIteration):
+                await g.asend(None)
+
+        self.loop.run_until_complete(run())
+
+    def test_async_gen_asyncio_athrow_stopiteration(self):
+        async def gen():
+            try:
+                yield 1
+            except ZeroDivisionError:
+                yield StopIteration(2)
+
+        async def run():
+            g = gen()
+            v = await g.asend(None)
+            self.assertEqual(v, 1)
+            v = await g.athrow(ZeroDivisionError)
+            self.assertIsInstance(v, StopIteration)
+            self.assertEqual(v.value, 2)
+            with self.assertRaises(StopAsyncIteration):
+                await g.asend(None)
+
+        self.loop.run_until_complete(run())
+
     def test_async_gen_asyncio_shutdown_01(self):
         finalized = 0
 
diff --git a/Lib/test/test_coroutines.py b/Lib/test/test_coroutines.py
--- a/Lib/test/test_coroutines.py
+++ b/Lib/test/test_coroutines.py
@@ -838,6 +838,21 @@
             coro.close()
             self.assertEqual(CHK, 1)
 
+    def test_coro_wrapper_send_tuple(self):
+        async def foo():
+            return (10,)
+
+        result = run_async__await__(foo())
+        self.assertEqual(result, ([], (10,)))
+
+    def test_coro_wrapper_send_stop_iterator(self):
+        async def foo():
+            return StopIteration(10)
+
+        result = run_async__await__(foo())
+        self.assertIsInstance(result[1], StopIteration)
+        self.assertEqual(result[1].value, 10)
+
     def test_cr_await(self):
         @types.coroutine
         def a():
@@ -1665,6 +1680,52 @@
                 warnings.simplefilter("error")
                 run_async(foo())
 
+    def test_for_tuple(self):
+        class Done(Exception): pass
+
+        class AIter(tuple):
+            i = 0
+            def __aiter__(self):
+                return self
+            async def __anext__(self):
+                if self.i >= len(self):
+                    raise StopAsyncIteration
+                self.i += 1
+                return self[self.i - 1]
+
+        result = []
+        async def foo():
+            async for i in AIter([42]):
+                result.append(i)
+            raise Done
+
+        with self.assertRaises(Done):
+            foo().send(None)
+        self.assertEqual(result, [42])
+
+    def test_for_stop_iteration(self):
+        class Done(Exception): pass
+
+        class AIter(StopIteration):
+            i = 0
+            def __aiter__(self):
+                return self
+            async def __anext__(self):
+                if self.i:
+                    raise StopAsyncIteration
+                self.i += 1
+                return self.value
+
+        result = []
+        async def foo():
+            async for i in AIter(42):
+                result.append(i)
+            raise Done
+
+        with self.assertRaises(Done):
+            foo().send(None)
+        self.assertEqual(result, [42])
+
     def test_comp_1(self):
         async def f(i):
             return i
diff --git a/Lib/test/test_generators.py b/Lib/test/test_generators.py
--- a/Lib/test/test_generators.py
+++ b/Lib/test/test_generators.py
@@ -277,6 +277,27 @@
             # hence no warning.
             next(g)
 
+    def test_return_tuple(self):
+        def g():
+            return (yield 1)
+
+        gen = g()
+        self.assertEqual(next(gen), 1)
+        with self.assertRaises(StopIteration) as cm:
+            gen.send((2,))
+        self.assertEqual(cm.exception.value, (2,))
+
+    def test_return_stopiteration(self):
+        def g():
+            return (yield 1)
+
+        gen = g()
+        self.assertEqual(next(gen), 1)
+        with self.assertRaises(StopIteration) as cm:
+            gen.send(StopIteration(2))
+        self.assertIsInstance(cm.exception.value, StopIteration)
+        self.assertEqual(cm.exception.value.value, 2)
+
 
 class YieldFromTests(unittest.TestCase):
     def test_generator_gi_yieldfrom(self):
diff --git a/Lib/test/test_yield_from.py b/Lib/test/test_yield_from.py
--- a/Lib/test/test_yield_from.py
+++ b/Lib/test/test_yield_from.py
@@ -384,9 +384,10 @@
             trace.append("Starting g1")
             yield "g1 ham"
             ret = yield from g2()
-            trace.append("g2 returned %s" % (ret,))
-            ret = yield from g2(42)
-            trace.append("g2 returned %s" % (ret,))
+            trace.append("g2 returned %r" % (ret,))
+            for v in 1, (2,), StopIteration(3):
+                ret = yield from g2(v)
+                trace.append("g2 returned %r" % (ret,))
             yield "g1 eggs"
             trace.append("Finishing g1")
         def g2(v = None):
@@ -410,7 +411,17 @@
             "Yielded g2 spam",
             "Yielded g2 more spam",
             "Finishing g2",
-            "g2 returned 42",
+            "g2 returned 1",
+            "Starting g2",
+            "Yielded g2 spam",
+            "Yielded g2 more spam",
+            "Finishing g2",
+            "g2 returned (2,)",
+            "Starting g2",
+            "Yielded g2 spam",
+            "Yielded g2 more spam",
+            "Finishing g2",
+            "g2 returned StopIteration(3,)",
             "Yielded g1 eggs",
             "Finishing g1",
         ])
@@ -670,14 +681,16 @@
                 next(gi)
                 trace.append("f SHOULD NOT BE HERE")
             except StopIteration as e:
-                trace.append("f caught %s" % (repr(e),))
+                trace.append("f caught %r" % (e,))
         def g(r):
             trace.append("g starting")
             yield
-            trace.append("g returning %s" % (r,))
+            trace.append("g returning %r" % (r,))
             return r
         f(None)
-        f(42)
+        f(1)
+        f((2,))
+        f(StopIteration(3))
         self.assertEqual(trace,[
             "g starting",
             "f resuming g",
@@ -685,8 +698,16 @@
             "f caught StopIteration()",
             "g starting",
             "f resuming g",
-            "g returning 42",
-            "f caught StopIteration(42,)",
+            "g returning 1",
+            "f caught StopIteration(1,)",
+            "g starting",
+            "f resuming g",
+            "g returning (2,)",
+            "f caught StopIteration((2,),)",
+            "g starting",
+            "f resuming g",
+            "g returning StopIteration(3,)",
+            "f caught StopIteration(StopIteration(3,),)",
         ])
 
     def test_send_and_return_with_value(self):
@@ -706,22 +727,34 @@
         def g(r):
             trace.append("g starting")
             x = yield
-            trace.append("g received %s" % (x,))
-            trace.append("g returning %s" % (r,))
+            trace.append("g received %r" % (x,))
+            trace.append("g returning %r" % (r,))
             return r
         f(None)
-        f(42)
-        self.assertEqual(trace,[
+        f(1)
+        f((2,))
+        f(StopIteration(3))
+        self.assertEqual(trace, [
             "g starting",
             "f sending spam to g",
-            "g received spam",
+            "g received 'spam'",
             "g returning None",
             "f caught StopIteration()",
             "g starting",
             "f sending spam to g",
-            "g received spam",
-            "g returning 42",
-            "f caught StopIteration(42,)",
+            "g received 'spam'",
+            "g returning 1",
+            'f caught StopIteration(1,)',
+            'g starting',
+            'f sending spam to g',
+            "g received 'spam'",
+            'g returning (2,)',
+            'f caught StopIteration((2,),)',
+            'g starting',
+            'f sending spam to g',
+            "g received 'spam'",
+            'g returning StopIteration(3,)',
+            'f caught StopIteration(StopIteration(3,),)'
         ])
 
     def test_catching_exception_from_subgen_and_returning(self):
@@ -729,27 +762,29 @@
         Test catching an exception thrown into a
         subgenerator and returning a value
         """
-        trace = []
         def inner():
             try:
                 yield 1
             except ValueError:
                 trace.append("inner caught ValueError")
-            return 2
+            return value
 
         def outer():
             v = yield from inner()
-            trace.append("inner returned %r to outer" % v)
+            trace.append("inner returned %r to outer" % (v,))
             yield v
-        g = outer()
-        trace.append(next(g))
-        trace.append(g.throw(ValueError))
-        self.assertEqual(trace,[
-            1,
-            "inner caught ValueError",
-            "inner returned 2 to outer",
-            2,
-        ])
+
+        for value in 2, (2,), StopIteration(2):
+            trace = []
+            g = outer()
+            trace.append(next(g))
+            trace.append(repr(g.throw(ValueError)))
+            self.assertEqual(trace, [
+                1,
+                "inner caught ValueError",
+                "inner returned %r to outer" % (value,),
+                repr(value),
+            ])
 
     def test_throwing_GeneratorExit_into_subgen_that_returns(self):
         """
diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -997,26 +997,12 @@
 
     res = _asyncio_Future_result_impl(fut);
     if (res != NULL) {
-        /* The result of the Future is not an exception.
-
-           We construct an exception instance manually with
-           PyObject_CallFunctionObjArgs and pass it to PyErr_SetObject
-           (similarly to what genobject.c does).
-
-           We do this to handle a situation when "res" is a tuple, in which
-           case PyErr_SetObject would set the value of StopIteration to
-           the first element of the tuple.
-
-           (See PyErr_SetObject/_PyErr_CreateException code for details.)
-        */
-        PyObject *e = PyObject_CallFunctionObjArgs(
-            PyExc_StopIteration, res, NULL);
-        Py_DECREF(res);
-        if (e == NULL) {
+        /* The result of the Future is not an exception. */
+        if (_PyGen_SetStopIterationValue(res) < 0) {
+            Py_DECREF(res);
             return NULL;
         }
-        PyErr_SetObject(PyExc_StopIteration, e);
-        Py_DECREF(e);
+        Py_DECREF(res);
     }
 
     it->future = NULL;
diff --git a/Objects/genobject.c b/Objects/genobject.c
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -208,16 +208,9 @@
             }
         }
         else {
-            PyObject *e = PyObject_CallFunctionObjArgs(
-                               PyExc_StopIteration, result, NULL);
-
             /* Async generators cannot return anything but None */
             assert(!PyAsyncGen_CheckExact(gen));
-
-            if (e != NULL) {
-                PyErr_SetObject(PyExc_StopIteration, e);
-                Py_DECREF(e);
-            }
+            _PyGen_SetStopIterationValue(result);
         }
         Py_CLEAR(result);
     }
@@ -562,6 +555,43 @@
 }
 
 /*
+ * Set StopIteration with specified value.  Value can be arbitrary object
+ * or NULL.
+ *
+ * Returns 0 if StopIteration is set and -1 if any other exception is set.
+ */
+int
+_PyGen_SetStopIterationValue(PyObject *value)
+{
+    PyObject *e;
+
+    if (value == NULL ||
+        (!PyTuple_Check(value) &&
+         !PyObject_TypeCheck(value, (PyTypeObject *) PyExc_StopIteration)))
+    {
+        /* Delay exception instantiation if we can */
+        PyErr_SetObject(PyExc_StopIteration, value);
+        return 0;
+    }
+    /* Construct an exception instance manually with
+     * PyObject_CallFunctionObjArgs and pass it to PyErr_SetObject.
+     *
+     * We do this to handle a situation when "value" is a tuple, in which
+     * case PyErr_SetObject would set the value of StopIteration to
+     * the first element of the tuple.
+     *
+     * (See PyErr_SetObject/_PyErr_CreateException code for details.)
+     */
+    e = PyObject_CallFunctionObjArgs(PyExc_StopIteration, value, NULL);
+    if (e == NULL) {
+        return -1;
+    }
+    PyErr_SetObject(PyExc_StopIteration, e);
+    Py_DECREF(e);
+    return 0;
+}
+
+/*
  *   If StopIteration exception is set, fetches its 'value'
  *   attribute if any, otherwise sets pvalue to None.
  *
@@ -571,7 +601,8 @@
  */
 
 int
-_PyGen_FetchStopIterationValue(PyObject **pvalue) {
+_PyGen_FetchStopIterationValue(PyObject **pvalue)
+{
     PyObject *et, *ev, *tb;
     PyObject *value = NULL;
 
@@ -583,8 +614,15 @@
                 value = ((PyStopIterationObject *)ev)->value;
                 Py_INCREF(value);
                 Py_DECREF(ev);
-            } else if (et == PyExc_StopIteration) {
-                /* avoid normalisation and take ev as value */
+            } else if (et == PyExc_StopIteration && !PyTuple_Check(ev)) {
+                /* Avoid normalisation and take ev as value.
+                 *
+                 * Normalization is required if the value is a tuple, in
+                 * that case the value of StopIteration would be set to
+                 * the first element of the tuple.
+                 *
+                 * (See _PyErr_CreateException code for details.)
+                 */
                 value = ev;
             } else {
                 /* normalisation required */
@@ -1106,7 +1144,7 @@
 static PyObject *
 aiter_wrapper_iternext(PyAIterWrapper *aw)
 {
-    PyErr_SetObject(PyExc_StopIteration, aw->ags_aiter);
+    _PyGen_SetStopIterationValue(aw->ags_aiter);
     return NULL;
 }
 
@@ -1504,16 +1542,8 @@
 
     if (_PyAsyncGenWrappedValue_CheckExact(result)) {
         /* async yield */
-        PyObject *e = PyObject_CallFunctionObjArgs(
-            PyExc_StopIteration,
-            ((_PyAsyncGenWrappedValue*)result)->agw_val,
-            NULL);
+        _PyGen_SetStopIterationValue(((_PyAsyncGenWrappedValue*)result)->agw_val);
         Py_DECREF(result);
-        if (e == NULL) {
-            return NULL;
-        }
-        PyErr_SetObject(PyExc_StopIteration, e);
-        Py_DECREF(e);
         return NULL;
     }
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 11:48:03 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 06 Nov 2016 16:48:03 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2323996=3A_Added_=5FPyGen=5FSetStopIterationValue?=
 =?utf-8?q?_for_safe_raising?=
Message-ID: <20161106164803.37387.65907.6D2530D1@psf.io>

https://hg.python.org/cpython/rev/d33b9fd46cef
changeset:   104932:d33b9fd46cef
parent:      104929:f332032b1934
parent:      104931:a2c9f06ada28
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 06 18:47:35 2016 +0200
summary:
  Issue #23996: Added _PyGen_SetStopIterationValue for safe raising
StopIteration with value. More safely handle non-normalized exceptions
in -_PyGen_FetchStopIterationValue.

files:
  Include/genobject.h         |   1 +
  Lib/test/test_asyncgen.py   |  79 +++++++++++++++++++++
  Lib/test/test_coroutines.py |  61 ++++++++++++++++
  Lib/test/test_generators.py |  21 +++++
  Lib/test/test_yield_from.py |  93 +++++++++++++++++-------
  Modules/_asynciomodule.c    |  22 +----
  Objects/genobject.c         |  72 +++++++++++++-----
  7 files changed, 281 insertions(+), 68 deletions(-)


diff --git a/Include/genobject.h b/Include/genobject.h
--- a/Include/genobject.h
+++ b/Include/genobject.h
@@ -41,6 +41,7 @@
 PyAPI_FUNC(PyObject *) PyGen_NewWithQualName(struct _frame *,
     PyObject *name, PyObject *qualname);
 PyAPI_FUNC(int) PyGen_NeedsFinalizing(PyGenObject *);
+PyAPI_FUNC(int) _PyGen_SetStopIterationValue(PyObject *);
 PyAPI_FUNC(int) _PyGen_FetchStopIterationValue(PyObject **);
 PyAPI_FUNC(PyObject *) _PyGen_Send(PyGenObject *, PyObject *);
 PyObject *_PyGen_yf(PyGenObject *);
diff --git a/Lib/test/test_asyncgen.py b/Lib/test/test_asyncgen.py
--- a/Lib/test/test_asyncgen.py
+++ b/Lib/test/test_asyncgen.py
@@ -450,6 +450,48 @@
 
         self.loop.run_until_complete(run())
 
+    def test_async_gen_asyncio_anext_tuple(self):
+        async def foo():
+            try:
+                yield (1,)
+            except ZeroDivisionError:
+                yield (2,)
+
+        async def run():
+            it = foo().__aiter__()
+
+            self.assertEqual(await it.__anext__(), (1,))
+            with self.assertRaises(StopIteration) as cm:
+                it.__anext__().throw(ZeroDivisionError)
+            self.assertEqual(cm.exception.args[0], (2,))
+            with self.assertRaises(StopAsyncIteration):
+                await it.__anext__()
+
+        self.loop.run_until_complete(run())
+
+    def test_async_gen_asyncio_anext_stopiteration(self):
+        async def foo():
+            try:
+                yield StopIteration(1)
+            except ZeroDivisionError:
+                yield StopIteration(3)
+
+        async def run():
+            it = foo().__aiter__()
+
+            v = await it.__anext__()
+            self.assertIsInstance(v, StopIteration)
+            self.assertEqual(v.value, 1)
+            with self.assertRaises(StopIteration) as cm:
+                it.__anext__().throw(ZeroDivisionError)
+            v = cm.exception.args[0]
+            self.assertIsInstance(v, StopIteration)
+            self.assertEqual(v.value, 3)
+            with self.assertRaises(StopAsyncIteration):
+                await it.__anext__()
+
+        self.loop.run_until_complete(run())
+
     def test_async_gen_asyncio_aclose_06(self):
         async def foo():
             try:
@@ -759,6 +801,43 @@
             self.loop.run_until_complete(run())
         self.assertEqual(DONE, 1)
 
+    def test_async_gen_asyncio_athrow_tuple(self):
+        async def gen():
+            try:
+                yield 1
+            except ZeroDivisionError:
+                yield (2,)
+
+        async def run():
+            g = gen()
+            v = await g.asend(None)
+            self.assertEqual(v, 1)
+            v = await g.athrow(ZeroDivisionError)
+            self.assertEqual(v, (2,))
+            with self.assertRaises(StopAsyncIteration):
+                await g.asend(None)
+
+        self.loop.run_until_complete(run())
+
+    def test_async_gen_asyncio_athrow_stopiteration(self):
+        async def gen():
+            try:
+                yield 1
+            except ZeroDivisionError:
+                yield StopIteration(2)
+
+        async def run():
+            g = gen()
+            v = await g.asend(None)
+            self.assertEqual(v, 1)
+            v = await g.athrow(ZeroDivisionError)
+            self.assertIsInstance(v, StopIteration)
+            self.assertEqual(v.value, 2)
+            with self.assertRaises(StopAsyncIteration):
+                await g.asend(None)
+
+        self.loop.run_until_complete(run())
+
     def test_async_gen_asyncio_shutdown_01(self):
         finalized = 0
 
diff --git a/Lib/test/test_coroutines.py b/Lib/test/test_coroutines.py
--- a/Lib/test/test_coroutines.py
+++ b/Lib/test/test_coroutines.py
@@ -838,6 +838,21 @@
             coro.close()
             self.assertEqual(CHK, 1)
 
+    def test_coro_wrapper_send_tuple(self):
+        async def foo():
+            return (10,)
+
+        result = run_async__await__(foo())
+        self.assertEqual(result, ([], (10,)))
+
+    def test_coro_wrapper_send_stop_iterator(self):
+        async def foo():
+            return StopIteration(10)
+
+        result = run_async__await__(foo())
+        self.assertIsInstance(result[1], StopIteration)
+        self.assertEqual(result[1].value, 10)
+
     def test_cr_await(self):
         @types.coroutine
         def a():
@@ -1665,6 +1680,52 @@
                 warnings.simplefilter("error")
                 run_async(foo())
 
+    def test_for_tuple(self):
+        class Done(Exception): pass
+
+        class AIter(tuple):
+            i = 0
+            def __aiter__(self):
+                return self
+            async def __anext__(self):
+                if self.i >= len(self):
+                    raise StopAsyncIteration
+                self.i += 1
+                return self[self.i - 1]
+
+        result = []
+        async def foo():
+            async for i in AIter([42]):
+                result.append(i)
+            raise Done
+
+        with self.assertRaises(Done):
+            foo().send(None)
+        self.assertEqual(result, [42])
+
+    def test_for_stop_iteration(self):
+        class Done(Exception): pass
+
+        class AIter(StopIteration):
+            i = 0
+            def __aiter__(self):
+                return self
+            async def __anext__(self):
+                if self.i:
+                    raise StopAsyncIteration
+                self.i += 1
+                return self.value
+
+        result = []
+        async def foo():
+            async for i in AIter(42):
+                result.append(i)
+            raise Done
+
+        with self.assertRaises(Done):
+            foo().send(None)
+        self.assertEqual(result, [42])
+
     def test_comp_1(self):
         async def f(i):
             return i
diff --git a/Lib/test/test_generators.py b/Lib/test/test_generators.py
--- a/Lib/test/test_generators.py
+++ b/Lib/test/test_generators.py
@@ -277,6 +277,27 @@
             # hence no warning.
             next(g)
 
+    def test_return_tuple(self):
+        def g():
+            return (yield 1)
+
+        gen = g()
+        self.assertEqual(next(gen), 1)
+        with self.assertRaises(StopIteration) as cm:
+            gen.send((2,))
+        self.assertEqual(cm.exception.value, (2,))
+
+    def test_return_stopiteration(self):
+        def g():
+            return (yield 1)
+
+        gen = g()
+        self.assertEqual(next(gen), 1)
+        with self.assertRaises(StopIteration) as cm:
+            gen.send(StopIteration(2))
+        self.assertIsInstance(cm.exception.value, StopIteration)
+        self.assertEqual(cm.exception.value.value, 2)
+
 
 class YieldFromTests(unittest.TestCase):
     def test_generator_gi_yieldfrom(self):
diff --git a/Lib/test/test_yield_from.py b/Lib/test/test_yield_from.py
--- a/Lib/test/test_yield_from.py
+++ b/Lib/test/test_yield_from.py
@@ -384,9 +384,10 @@
             trace.append("Starting g1")
             yield "g1 ham"
             ret = yield from g2()
-            trace.append("g2 returned %s" % (ret,))
-            ret = yield from g2(42)
-            trace.append("g2 returned %s" % (ret,))
+            trace.append("g2 returned %r" % (ret,))
+            for v in 1, (2,), StopIteration(3):
+                ret = yield from g2(v)
+                trace.append("g2 returned %r" % (ret,))
             yield "g1 eggs"
             trace.append("Finishing g1")
         def g2(v = None):
@@ -410,7 +411,17 @@
             "Yielded g2 spam",
             "Yielded g2 more spam",
             "Finishing g2",
-            "g2 returned 42",
+            "g2 returned 1",
+            "Starting g2",
+            "Yielded g2 spam",
+            "Yielded g2 more spam",
+            "Finishing g2",
+            "g2 returned (2,)",
+            "Starting g2",
+            "Yielded g2 spam",
+            "Yielded g2 more spam",
+            "Finishing g2",
+            "g2 returned StopIteration(3,)",
             "Yielded g1 eggs",
             "Finishing g1",
         ])
@@ -670,14 +681,16 @@
                 next(gi)
                 trace.append("f SHOULD NOT BE HERE")
             except StopIteration as e:
-                trace.append("f caught %s" % (repr(e),))
+                trace.append("f caught %r" % (e,))
         def g(r):
             trace.append("g starting")
             yield
-            trace.append("g returning %s" % (r,))
+            trace.append("g returning %r" % (r,))
             return r
         f(None)
-        f(42)
+        f(1)
+        f((2,))
+        f(StopIteration(3))
         self.assertEqual(trace,[
             "g starting",
             "f resuming g",
@@ -685,8 +698,16 @@
             "f caught StopIteration()",
             "g starting",
             "f resuming g",
-            "g returning 42",
-            "f caught StopIteration(42,)",
+            "g returning 1",
+            "f caught StopIteration(1,)",
+            "g starting",
+            "f resuming g",
+            "g returning (2,)",
+            "f caught StopIteration((2,),)",
+            "g starting",
+            "f resuming g",
+            "g returning StopIteration(3,)",
+            "f caught StopIteration(StopIteration(3,),)",
         ])
 
     def test_send_and_return_with_value(self):
@@ -706,22 +727,34 @@
         def g(r):
             trace.append("g starting")
             x = yield
-            trace.append("g received %s" % (x,))
-            trace.append("g returning %s" % (r,))
+            trace.append("g received %r" % (x,))
+            trace.append("g returning %r" % (r,))
             return r
         f(None)
-        f(42)
-        self.assertEqual(trace,[
+        f(1)
+        f((2,))
+        f(StopIteration(3))
+        self.assertEqual(trace, [
             "g starting",
             "f sending spam to g",
-            "g received spam",
+            "g received 'spam'",
             "g returning None",
             "f caught StopIteration()",
             "g starting",
             "f sending spam to g",
-            "g received spam",
-            "g returning 42",
-            "f caught StopIteration(42,)",
+            "g received 'spam'",
+            "g returning 1",
+            'f caught StopIteration(1,)',
+            'g starting',
+            'f sending spam to g',
+            "g received 'spam'",
+            'g returning (2,)',
+            'f caught StopIteration((2,),)',
+            'g starting',
+            'f sending spam to g',
+            "g received 'spam'",
+            'g returning StopIteration(3,)',
+            'f caught StopIteration(StopIteration(3,),)'
         ])
 
     def test_catching_exception_from_subgen_and_returning(self):
@@ -729,27 +762,29 @@
         Test catching an exception thrown into a
         subgenerator and returning a value
         """
-        trace = []
         def inner():
             try:
                 yield 1
             except ValueError:
                 trace.append("inner caught ValueError")
-            return 2
+            return value
 
         def outer():
             v = yield from inner()
-            trace.append("inner returned %r to outer" % v)
+            trace.append("inner returned %r to outer" % (v,))
             yield v
-        g = outer()
-        trace.append(next(g))
-        trace.append(g.throw(ValueError))
-        self.assertEqual(trace,[
-            1,
-            "inner caught ValueError",
-            "inner returned 2 to outer",
-            2,
-        ])
+
+        for value in 2, (2,), StopIteration(2):
+            trace = []
+            g = outer()
+            trace.append(next(g))
+            trace.append(repr(g.throw(ValueError)))
+            self.assertEqual(trace, [
+                1,
+                "inner caught ValueError",
+                "inner returned %r to outer" % (value,),
+                repr(value),
+            ])
 
     def test_throwing_GeneratorExit_into_subgen_that_returns(self):
         """
diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -997,26 +997,12 @@
 
     res = _asyncio_Future_result_impl(fut);
     if (res != NULL) {
-        /* The result of the Future is not an exception.
-
-           We construct an exception instance manually with
-           PyObject_CallFunctionObjArgs and pass it to PyErr_SetObject
-           (similarly to what genobject.c does).
-
-           We do this to handle a situation when "res" is a tuple, in which
-           case PyErr_SetObject would set the value of StopIteration to
-           the first element of the tuple.
-
-           (See PyErr_SetObject/_PyErr_CreateException code for details.)
-        */
-        PyObject *e = PyObject_CallFunctionObjArgs(
-            PyExc_StopIteration, res, NULL);
-        Py_DECREF(res);
-        if (e == NULL) {
+        /* The result of the Future is not an exception. */
+        if (_PyGen_SetStopIterationValue(res) < 0) {
+            Py_DECREF(res);
             return NULL;
         }
-        PyErr_SetObject(PyExc_StopIteration, e);
-        Py_DECREF(e);
+        Py_DECREF(res);
     }
 
     it->future = NULL;
diff --git a/Objects/genobject.c b/Objects/genobject.c
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -208,16 +208,9 @@
             }
         }
         else {
-            PyObject *e = PyObject_CallFunctionObjArgs(
-                               PyExc_StopIteration, result, NULL);
-
             /* Async generators cannot return anything but None */
             assert(!PyAsyncGen_CheckExact(gen));
-
-            if (e != NULL) {
-                PyErr_SetObject(PyExc_StopIteration, e);
-                Py_DECREF(e);
-            }
+            _PyGen_SetStopIterationValue(result);
         }
         Py_CLEAR(result);
     }
@@ -562,6 +555,43 @@
 }
 
 /*
+ * Set StopIteration with specified value.  Value can be arbitrary object
+ * or NULL.
+ *
+ * Returns 0 if StopIteration is set and -1 if any other exception is set.
+ */
+int
+_PyGen_SetStopIterationValue(PyObject *value)
+{
+    PyObject *e;
+
+    if (value == NULL ||
+        (!PyTuple_Check(value) &&
+         !PyObject_TypeCheck(value, (PyTypeObject *) PyExc_StopIteration)))
+    {
+        /* Delay exception instantiation if we can */
+        PyErr_SetObject(PyExc_StopIteration, value);
+        return 0;
+    }
+    /* Construct an exception instance manually with
+     * PyObject_CallFunctionObjArgs and pass it to PyErr_SetObject.
+     *
+     * We do this to handle a situation when "value" is a tuple, in which
+     * case PyErr_SetObject would set the value of StopIteration to
+     * the first element of the tuple.
+     *
+     * (See PyErr_SetObject/_PyErr_CreateException code for details.)
+     */
+    e = PyObject_CallFunctionObjArgs(PyExc_StopIteration, value, NULL);
+    if (e == NULL) {
+        return -1;
+    }
+    PyErr_SetObject(PyExc_StopIteration, e);
+    Py_DECREF(e);
+    return 0;
+}
+
+/*
  *   If StopIteration exception is set, fetches its 'value'
  *   attribute if any, otherwise sets pvalue to None.
  *
@@ -571,7 +601,8 @@
  */
 
 int
-_PyGen_FetchStopIterationValue(PyObject **pvalue) {
+_PyGen_FetchStopIterationValue(PyObject **pvalue)
+{
     PyObject *et, *ev, *tb;
     PyObject *value = NULL;
 
@@ -583,8 +614,15 @@
                 value = ((PyStopIterationObject *)ev)->value;
                 Py_INCREF(value);
                 Py_DECREF(ev);
-            } else if (et == PyExc_StopIteration) {
-                /* avoid normalisation and take ev as value */
+            } else if (et == PyExc_StopIteration && !PyTuple_Check(ev)) {
+                /* Avoid normalisation and take ev as value.
+                 *
+                 * Normalization is required if the value is a tuple, in
+                 * that case the value of StopIteration would be set to
+                 * the first element of the tuple.
+                 *
+                 * (See _PyErr_CreateException code for details.)
+                 */
                 value = ev;
             } else {
                 /* normalisation required */
@@ -1106,7 +1144,7 @@
 static PyObject *
 aiter_wrapper_iternext(PyAIterWrapper *aw)
 {
-    PyErr_SetObject(PyExc_StopIteration, aw->ags_aiter);
+    _PyGen_SetStopIterationValue(aw->ags_aiter);
     return NULL;
 }
 
@@ -1504,16 +1542,8 @@
 
     if (_PyAsyncGenWrappedValue_CheckExact(result)) {
         /* async yield */
-        PyObject *e = PyObject_CallFunctionObjArgs(
-            PyExc_StopIteration,
-            ((_PyAsyncGenWrappedValue*)result)->agw_val,
-            NULL);
+        _PyGen_SetStopIterationValue(((_PyAsyncGenWrappedValue*)result)->agw_val);
         Py_DECREF(result);
-        if (e == NULL) {
-            return NULL;
-        }
-        PyErr_SetObject(PyExc_StopIteration, e);
-        Py_DECREF(e);
         return NULL;
     }
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 13:01:00 2016
From: python-checkins at python.org (berker.peksag)
Date: Sun, 06 Nov 2016 18:01:00 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIxODY0?=
 =?utf-8?q?=3A_Remove_outdated_section_about_exceptions_from_the_tutorial?=
Message-ID: <20161106180100.22336.10889.ADCC12BD@psf.io>

https://hg.python.org/cpython/rev/db6d556365d7
changeset:   104933:db6d556365d7
branch:      3.5
parent:      104930:bce18f5c0bc4
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Sun Nov 06 21:15:01 2016 +0300
summary:
  Issue #21864: Remove outdated section about exceptions from the tutorial

Move the still relevant parts of it to the previous
chapter, "Errors and Exceptions".

files:
  Doc/tutorial/classes.rst |  49 ----------------------------
  Doc/tutorial/errors.rst  |  32 +++++++++++++++++-
  2 files changed, 31 insertions(+), 50 deletions(-)


diff --git a/Doc/tutorial/classes.rst b/Doc/tutorial/classes.rst
--- a/Doc/tutorial/classes.rst
+++ b/Doc/tutorial/classes.rst
@@ -744,55 +744,6 @@
 corresponding to the method.
 
 
-.. _tut-exceptionclasses:
-
-Exceptions Are Classes Too
-==========================
-
-User-defined exceptions are identified by classes as well.  Using this mechanism
-it is possible to create extensible hierarchies of exceptions.
-
-There are two new valid (semantic) forms for the :keyword:`raise` statement::
-
-   raise Class
-
-   raise Instance
-
-In the first form, ``Class`` must be an instance of :class:`type` or of a
-class derived from it.  The first form is a shorthand for::
-
-   raise Class()
-
-A class in an :keyword:`except` clause is compatible with an exception if it is
-the same class or a base class thereof (but not the other way around --- an
-except clause listing a derived class is not compatible with a base class).  For
-example, the following code will print B, C, D in that order::
-
-   class B(Exception):
-       pass
-   class C(B):
-       pass
-   class D(C):
-       pass
-
-   for cls in [B, C, D]:
-       try:
-           raise cls()
-       except D:
-           print("D")
-       except C:
-           print("C")
-       except B:
-           print("B")
-
-Note that if the except clauses were reversed (with ``except B`` first), it
-would have printed B, B, B --- the first matching except clause is triggered.
-
-When an error message is printed for an unhandled exception, the exception's
-class name is printed, then a colon and a space, and finally the instance
-converted to a string using the built-in function :func:`str`.
-
-
 .. _tut-iterators:
 
 Iterators
diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst
--- a/Doc/tutorial/errors.rst
+++ b/Doc/tutorial/errors.rst
@@ -120,6 +120,33 @@
    ... except (RuntimeError, TypeError, NameError):
    ...     pass
 
+A class in an :keyword:`except` clause is compatible with an exception if it is
+the same class or a base class thereof (but not the other way around --- an
+except clause listing a derived class is not compatible with a base class).  For
+example, the following code will print B, C, D in that order::
+
+   class B(Exception):
+       pass
+
+   class C(B):
+       pass
+
+   class D(C):
+       pass
+
+   for cls in [B, C, D]:
+       try:
+           raise cls()
+       except D:
+           print("D")
+       except C:
+           print("C")
+       except B:
+           print("B")
+
+Note that if the except clauses were reversed (with ``except B`` first), it
+would have printed B, B, B --- the first matching except clause is triggered.
+
 The last except clause may omit the exception name(s), to serve as a wildcard.
 Use this with extreme caution, since it is easy to mask a real programming error
 in this way!  It can also be used to print an error message and then re-raise
@@ -219,7 +246,10 @@
 
 The sole argument to :keyword:`raise` indicates the exception to be raised.
 This must be either an exception instance or an exception class (a class that
-derives from :class:`Exception`).
+derives from :class:`Exception`).  If an exception class is passed, it will
+be implicitly instantiated by calling its constructor with no arguments::
+
+   raise ValueError  # shorthand for 'raise ValueError()'
 
 If you need to determine whether an exception was raised but don't intend to
 handle it, a simpler form of the :keyword:`raise` statement allows you to

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 13:01:00 2016
From: python-checkins at python.org (berker.peksag)
Date: Sun, 06 Nov 2016 18:01:00 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2321864=3A_Merge_from_3=2E5?=
Message-ID: <20161106180100.3264.82650.C122A70D@psf.io>

https://hg.python.org/cpython/rev/f82e348946e3
changeset:   104934:f82e348946e3
branch:      3.6
parent:      104931:a2c9f06ada28
parent:      104933:db6d556365d7
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Sun Nov 06 21:15:48 2016 +0300
summary:
  Issue #21864: Merge from 3.5

files:
  Doc/tutorial/classes.rst |  49 ----------------------------
  Doc/tutorial/errors.rst  |  32 +++++++++++++++++-
  2 files changed, 31 insertions(+), 50 deletions(-)


diff --git a/Doc/tutorial/classes.rst b/Doc/tutorial/classes.rst
--- a/Doc/tutorial/classes.rst
+++ b/Doc/tutorial/classes.rst
@@ -744,55 +744,6 @@
 corresponding to the method.
 
 
-.. _tut-exceptionclasses:
-
-Exceptions Are Classes Too
-==========================
-
-User-defined exceptions are identified by classes as well.  Using this mechanism
-it is possible to create extensible hierarchies of exceptions.
-
-There are two new valid (semantic) forms for the :keyword:`raise` statement::
-
-   raise Class
-
-   raise Instance
-
-In the first form, ``Class`` must be an instance of :class:`type` or of a
-class derived from it.  The first form is a shorthand for::
-
-   raise Class()
-
-A class in an :keyword:`except` clause is compatible with an exception if it is
-the same class or a base class thereof (but not the other way around --- an
-except clause listing a derived class is not compatible with a base class).  For
-example, the following code will print B, C, D in that order::
-
-   class B(Exception):
-       pass
-   class C(B):
-       pass
-   class D(C):
-       pass
-
-   for cls in [B, C, D]:
-       try:
-           raise cls()
-       except D:
-           print("D")
-       except C:
-           print("C")
-       except B:
-           print("B")
-
-Note that if the except clauses were reversed (with ``except B`` first), it
-would have printed B, B, B --- the first matching except clause is triggered.
-
-When an error message is printed for an unhandled exception, the exception's
-class name is printed, then a colon and a space, and finally the instance
-converted to a string using the built-in function :func:`str`.
-
-
 .. _tut-iterators:
 
 Iterators
diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst
--- a/Doc/tutorial/errors.rst
+++ b/Doc/tutorial/errors.rst
@@ -120,6 +120,33 @@
    ... except (RuntimeError, TypeError, NameError):
    ...     pass
 
+A class in an :keyword:`except` clause is compatible with an exception if it is
+the same class or a base class thereof (but not the other way around --- an
+except clause listing a derived class is not compatible with a base class).  For
+example, the following code will print B, C, D in that order::
+
+   class B(Exception):
+       pass
+
+   class C(B):
+       pass
+
+   class D(C):
+       pass
+
+   for cls in [B, C, D]:
+       try:
+           raise cls()
+       except D:
+           print("D")
+       except C:
+           print("C")
+       except B:
+           print("B")
+
+Note that if the except clauses were reversed (with ``except B`` first), it
+would have printed B, B, B --- the first matching except clause is triggered.
+
 The last except clause may omit the exception name(s), to serve as a wildcard.
 Use this with extreme caution, since it is easy to mask a real programming error
 in this way!  It can also be used to print an error message and then re-raise
@@ -219,7 +246,10 @@
 
 The sole argument to :keyword:`raise` indicates the exception to be raised.
 This must be either an exception instance or an exception class (a class that
-derives from :class:`Exception`).
+derives from :class:`Exception`).  If an exception class is passed, it will
+be implicitly instantiated by calling its constructor with no arguments::
+
+   raise ValueError  # shorthand for 'raise ValueError()'
 
 If you need to determine whether an exception was raised but don't intend to
 handle it, a simpler form of the :keyword:`raise` statement allows you to

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 13:01:00 2016
From: python-checkins at python.org (berker.peksag)
Date: Sun, 06 Nov 2016 18:01:00 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2321864=3A_Merge_from_3=2E6?=
Message-ID: <20161106180100.30840.42835.BD0E2AAE@psf.io>

https://hg.python.org/cpython/rev/6ec669efeea5
changeset:   104935:6ec669efeea5
parent:      104932:d33b9fd46cef
parent:      104934:f82e348946e3
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Sun Nov 06 21:16:29 2016 +0300
summary:
  Issue #21864: Merge from 3.6

files:
  Doc/tutorial/classes.rst |  49 ----------------------------
  Doc/tutorial/errors.rst  |  32 +++++++++++++++++-
  2 files changed, 31 insertions(+), 50 deletions(-)


diff --git a/Doc/tutorial/classes.rst b/Doc/tutorial/classes.rst
--- a/Doc/tutorial/classes.rst
+++ b/Doc/tutorial/classes.rst
@@ -744,55 +744,6 @@
 corresponding to the method.
 
 
-.. _tut-exceptionclasses:
-
-Exceptions Are Classes Too
-==========================
-
-User-defined exceptions are identified by classes as well.  Using this mechanism
-it is possible to create extensible hierarchies of exceptions.
-
-There are two new valid (semantic) forms for the :keyword:`raise` statement::
-
-   raise Class
-
-   raise Instance
-
-In the first form, ``Class`` must be an instance of :class:`type` or of a
-class derived from it.  The first form is a shorthand for::
-
-   raise Class()
-
-A class in an :keyword:`except` clause is compatible with an exception if it is
-the same class or a base class thereof (but not the other way around --- an
-except clause listing a derived class is not compatible with a base class).  For
-example, the following code will print B, C, D in that order::
-
-   class B(Exception):
-       pass
-   class C(B):
-       pass
-   class D(C):
-       pass
-
-   for cls in [B, C, D]:
-       try:
-           raise cls()
-       except D:
-           print("D")
-       except C:
-           print("C")
-       except B:
-           print("B")
-
-Note that if the except clauses were reversed (with ``except B`` first), it
-would have printed B, B, B --- the first matching except clause is triggered.
-
-When an error message is printed for an unhandled exception, the exception's
-class name is printed, then a colon and a space, and finally the instance
-converted to a string using the built-in function :func:`str`.
-
-
 .. _tut-iterators:
 
 Iterators
diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst
--- a/Doc/tutorial/errors.rst
+++ b/Doc/tutorial/errors.rst
@@ -120,6 +120,33 @@
    ... except (RuntimeError, TypeError, NameError):
    ...     pass
 
+A class in an :keyword:`except` clause is compatible with an exception if it is
+the same class or a base class thereof (but not the other way around --- an
+except clause listing a derived class is not compatible with a base class).  For
+example, the following code will print B, C, D in that order::
+
+   class B(Exception):
+       pass
+
+   class C(B):
+       pass
+
+   class D(C):
+       pass
+
+   for cls in [B, C, D]:
+       try:
+           raise cls()
+       except D:
+           print("D")
+       except C:
+           print("C")
+       except B:
+           print("B")
+
+Note that if the except clauses were reversed (with ``except B`` first), it
+would have printed B, B, B --- the first matching except clause is triggered.
+
 The last except clause may omit the exception name(s), to serve as a wildcard.
 Use this with extreme caution, since it is easy to mask a real programming error
 in this way!  It can also be used to print an error message and then re-raise
@@ -219,7 +246,10 @@
 
 The sole argument to :keyword:`raise` indicates the exception to be raised.
 This must be either an exception instance or an exception class (a class that
-derives from :class:`Exception`).
+derives from :class:`Exception`).  If an exception class is passed, it will
+be implicitly instantiated by calling its constructor with no arguments::
+
+   raise ValueError  # shorthand for 'raise ValueError()'
 
 If you need to determine whether an exception was raised but don't intend to
 handle it, a simpler form of the :keyword:`raise` statement allows you to

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 13:30:24 2016
From: python-checkins at python.org (berker.peksag)
Date: Sun, 06 Nov 2016 18:30:24 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2321590=3A_Merge_from_3=2E6?=
Message-ID: <20161106183024.31027.21817.CB372552@psf.io>

https://hg.python.org/cpython/rev/442453fa3370
changeset:   104937:442453fa3370
parent:      104935:6ec669efeea5
parent:      104936:d05d312161f2
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Sun Nov 06 21:45:45 2016 +0300
summary:
  Issue #21590: Merge from 3.6

files:
  Doc/howto/instrumentation.rst |  49 ++++++++++------------
  1 files changed, 23 insertions(+), 26 deletions(-)


diff --git a/Doc/howto/instrumentation.rst b/Doc/howto/instrumentation.rst
--- a/Doc/howto/instrumentation.rst
+++ b/Doc/howto/instrumentation.rst
@@ -1,3 +1,5 @@
+.. highlight:: shell-session
+
 .. _instrumentation:
 
 ===============================================
@@ -20,9 +22,6 @@
 making it easier to monitor what the CPython processes on a system are
 doing.
 
-.. I'm using ".. code-block:: c" for SystemTap scripts, as "c" is syntactically
-   the closest match that Sphinx supports
-
 .. impl-detail::
 
    DTrace markers are implementation details of the CPython interpreter.
@@ -40,14 +39,16 @@
 
 On a Linux machine, this can be done via::
 
-   yum install systemtap-sdt-devel
+   $ yum install systemtap-sdt-devel
 
 or::
 
-   sudo apt-get install systemtap-sdt-dev
+   $ sudo apt-get install systemtap-sdt-dev
 
 
-CPython must then be configured `--with-dtrace`::
+CPython must then be configured ``--with-dtrace``:
+
+.. code-block:: none
 
    checking for --with-dtrace... yes
 
@@ -71,22 +72,18 @@
 On Linux, you can verify if the SystemTap static markers are present in
 the built binary by seeing if it contains a ".note.stapsdt" section.
 
-.. code-block:: bash
+::
 
    $ readelf -S ./python | grep .note.stapsdt
    [30] .note.stapsdt        NOTE         0000000000000000 00308d78
 
 If you've built Python as a shared library (with --enable-shared), you
-need to look instead within the shared library.  For example:
-
-.. code-block:: bash
+need to look instead within the shared library.  For example::
 
    $ readelf -S libpython3.3dm.so.1.0 | grep .note.stapsdt
    [29] .note.stapsdt        NOTE         0000000000000000 00365b68
 
-Sufficiently modern readelf can print the metadata:
-
-.. code-block:: bash
+Sufficiently modern readelf can print the metadata::
 
     $ readelf -n ./python
 
@@ -136,7 +133,7 @@
 a function called "start". In other words, import-time function
 invocations are not going to be listed:
 
-.. code-block:: c
+.. code-block:: none
 
     self int indent;
 
@@ -170,13 +167,13 @@
             self->trace = 0;
     }
 
-It can be invoked like this:
-
-.. code-block:: bash
+It can be invoked like this::
 
   $ sudo dtrace -q -s call_stack.d -c "python3.6 script.py"
 
-The output looks like this::
+The output looks like this:
+
+.. code-block:: none
 
     156641360502280  function-entry:call_stack.py:start:23
     156641360518804  function-entry: call_stack.py:function_1:1
@@ -208,7 +205,7 @@
 For example, this SystemTap script can be used to show the call/return
 hierarchy of a Python script:
 
-.. code-block:: c
+.. code-block:: none
 
    probe process("python").mark("function__entry") {
         filename = user_string($arg1);
@@ -228,15 +225,15 @@
               thread_indent(-1), funcname, filename, lineno);
    }
 
-It can be invoked like this:
-
-.. code-block:: bash
+It can be invoked like this::
 
    $ stap \
      show-call-hierarchy.stp \
      -c "./python test.py"
 
-The output looks like this::
+The output looks like this:
+
+.. code-block:: none
 
    11408 python(8274):        => __contains__ in Lib/_abcoll.py:362
    11414 python(8274):         => __getitem__ in Lib/os.py:425
@@ -325,7 +322,7 @@
 
 Here is a tapset file, based on a non-shared build of CPython:
 
-.. code-block:: c
+.. code-block:: none
 
     /*
        Provide a higher-level wrapping around the function__entry and
@@ -369,7 +366,7 @@
 example given above of tracing the Python function-call hierarchy, without
 needing to directly name the static markers:
 
-.. code-block:: c
+.. code-block:: none
 
     probe python.function.entry
     {
@@ -388,7 +385,7 @@
 running CPython code, showing the top 20 most frequently-entered bytecode
 frames, each second, across the whole system:
 
-.. code-block:: c
+.. code-block:: none
 
     global fn_calls;
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 13:30:24 2016
From: python-checkins at python.org (berker.peksag)
Date: Sun, 06 Nov 2016 18:30:24 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzIxNTkw?=
 =?utf-8?q?=3A_Silence_Sphinx_warnings_in_instrumentation=2Erst?=
Message-ID: <20161106183024.83732.48158.7430FB48@psf.io>

https://hg.python.org/cpython/rev/d05d312161f2
changeset:   104936:d05d312161f2
branch:      3.6
parent:      104934:f82e348946e3
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Sun Nov 06 21:45:16 2016 +0300
summary:
  Issue #21590: Silence Sphinx warnings in instrumentation.rst

WARNING: Could not lex literal_block as "c". Highlighting skipped.

Patch by SilentGhost.

files:
  Doc/howto/instrumentation.rst |  49 ++++++++++------------
  1 files changed, 23 insertions(+), 26 deletions(-)


diff --git a/Doc/howto/instrumentation.rst b/Doc/howto/instrumentation.rst
--- a/Doc/howto/instrumentation.rst
+++ b/Doc/howto/instrumentation.rst
@@ -1,3 +1,5 @@
+.. highlight:: shell-session
+
 .. _instrumentation:
 
 ===============================================
@@ -20,9 +22,6 @@
 making it easier to monitor what the CPython processes on a system are
 doing.
 
-.. I'm using ".. code-block:: c" for SystemTap scripts, as "c" is syntactically
-   the closest match that Sphinx supports
-
 .. impl-detail::
 
    DTrace markers are implementation details of the CPython interpreter.
@@ -40,14 +39,16 @@
 
 On a Linux machine, this can be done via::
 
-   yum install systemtap-sdt-devel
+   $ yum install systemtap-sdt-devel
 
 or::
 
-   sudo apt-get install systemtap-sdt-dev
+   $ sudo apt-get install systemtap-sdt-dev
 
 
-CPython must then be configured `--with-dtrace`::
+CPython must then be configured ``--with-dtrace``:
+
+.. code-block:: none
 
    checking for --with-dtrace... yes
 
@@ -71,22 +72,18 @@
 On Linux, you can verify if the SystemTap static markers are present in
 the built binary by seeing if it contains a ".note.stapsdt" section.
 
-.. code-block:: bash
+::
 
    $ readelf -S ./python | grep .note.stapsdt
    [30] .note.stapsdt        NOTE         0000000000000000 00308d78
 
 If you've built Python as a shared library (with --enable-shared), you
-need to look instead within the shared library.  For example:
-
-.. code-block:: bash
+need to look instead within the shared library.  For example::
 
    $ readelf -S libpython3.3dm.so.1.0 | grep .note.stapsdt
    [29] .note.stapsdt        NOTE         0000000000000000 00365b68
 
-Sufficiently modern readelf can print the metadata:
-
-.. code-block:: bash
+Sufficiently modern readelf can print the metadata::
 
     $ readelf -n ./python
 
@@ -136,7 +133,7 @@
 a function called "start". In other words, import-time function
 invocations are not going to be listed:
 
-.. code-block:: c
+.. code-block:: none
 
     self int indent;
 
@@ -170,13 +167,13 @@
             self->trace = 0;
     }
 
-It can be invoked like this:
-
-.. code-block:: bash
+It can be invoked like this::
 
   $ sudo dtrace -q -s call_stack.d -c "python3.6 script.py"
 
-The output looks like this::
+The output looks like this:
+
+.. code-block:: none
 
     156641360502280  function-entry:call_stack.py:start:23
     156641360518804  function-entry: call_stack.py:function_1:1
@@ -208,7 +205,7 @@
 For example, this SystemTap script can be used to show the call/return
 hierarchy of a Python script:
 
-.. code-block:: c
+.. code-block:: none
 
    probe process("python").mark("function__entry") {
         filename = user_string($arg1);
@@ -228,15 +225,15 @@
               thread_indent(-1), funcname, filename, lineno);
    }
 
-It can be invoked like this:
-
-.. code-block:: bash
+It can be invoked like this::
 
    $ stap \
      show-call-hierarchy.stp \
      -c "./python test.py"
 
-The output looks like this::
+The output looks like this:
+
+.. code-block:: none
 
    11408 python(8274):        => __contains__ in Lib/_abcoll.py:362
    11414 python(8274):         => __getitem__ in Lib/os.py:425
@@ -325,7 +322,7 @@
 
 Here is a tapset file, based on a non-shared build of CPython:
 
-.. code-block:: c
+.. code-block:: none
 
     /*
        Provide a higher-level wrapping around the function__entry and
@@ -369,7 +366,7 @@
 example given above of tracing the Python function-call hierarchy, without
 needing to directly name the static markers:
 
-.. code-block:: c
+.. code-block:: none
 
     probe python.function.entry
     {
@@ -388,7 +385,7 @@
 running CPython code, showing the top 20 most frequently-entered bytecode
 frames, each second, across the whole system:
 
-.. code-block:: c
+.. code-block:: none
 
     global fn_calls;
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 16:01:30 2016
From: python-checkins at python.org (benjamin.peterson)
Date: Sun, 06 Nov 2016 21:01:30 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogbWVyZ2UgMy42?=
Message-ID: <20161106210130.22316.8231.C9751C6B@psf.io>

https://hg.python.org/cpython/rev/cc71f2c5aa68
changeset:   104940:cc71f2c5aa68
parent:      104937:442453fa3370
parent:      104939:1f84be591d56
user:        Benjamin Peterson <benjamin at python.org>
date:        Sun Nov 06 13:01:23 2016 -0800
summary:
  merge 3.6

files:
  Objects/object.c |  9 +++++++++
  1 files changed, 9 insertions(+), 0 deletions(-)


diff --git a/Objects/object.c b/Objects/object.c
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -1598,6 +1598,15 @@
     if (PyType_Ready(&PyDict_Type) < 0)
         Py_FatalError("Can't initialize dict type");
 
+    if (PyType_Ready(&PyDictKeys_Type) < 0)
+        Py_FatalError("Can't initialize dict keys type");
+
+    if (PyType_Ready(&PyDictValues_Type) < 0)
+        Py_FatalError("Can't initialize dict values type");
+
+    if (PyType_Ready(&PyDictItems_Type) < 0)
+        Py_FatalError("Can't initialize dict items type");
+
     if (PyType_Ready(&PyODict_Type) < 0)
         Py_FatalError("Can't initialize OrderedDict type");
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 16:01:30 2016
From: python-checkins at python.org (benjamin.peterson)
Date: Sun, 06 Nov 2016 21:01:30 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_make_sure_dict?=
 =?utf-8?q?_view_types_are_initialized?=
Message-ID: <20161106210130.37368.6357.EB7D7491@psf.io>

https://hg.python.org/cpython/rev/ec96d06462a2
changeset:   104938:ec96d06462a2
branch:      3.5
parent:      104933:db6d556365d7
user:        Benjamin Peterson <benjamin at python.org>
date:        Sun Nov 06 13:01:07 2016 -0800
summary:
  make sure dict view types are initialized

files:
  Objects/object.c |  9 +++++++++
  1 files changed, 9 insertions(+), 0 deletions(-)


diff --git a/Objects/object.c b/Objects/object.c
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -1607,6 +1607,15 @@
     if (PyType_Ready(&PyDict_Type) < 0)
         Py_FatalError("Can't initialize dict type");
 
+    if (PyType_Ready(&PyDictKeys_Type) < 0)
+        Py_FatalError("Can't initialize dict keys type");
+
+    if (PyType_Ready(&PyDictValues_Type) < 0)
+        Py_FatalError("Can't initialize dict values type");
+
+    if (PyType_Ready(&PyDictItems_Type) < 0)
+        Py_FatalError("Can't initialize dict items type");
+
     if (PyType_Ready(&PyODict_Type) < 0)
         Py_FatalError("Can't initialize OrderedDict type");
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 16:01:30 2016
From: python-checkins at python.org (benjamin.peterson)
Date: Sun, 06 Nov 2016 21:01:30 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_merge_3=2E5?=
Message-ID: <20161106210130.22961.75855.36C2931C@psf.io>

https://hg.python.org/cpython/rev/1f84be591d56
changeset:   104939:1f84be591d56
branch:      3.6
parent:      104936:d05d312161f2
parent:      104938:ec96d06462a2
user:        Benjamin Peterson <benjamin at python.org>
date:        Sun Nov 06 13:01:15 2016 -0800
summary:
  merge 3.5

files:
  Objects/object.c |  9 +++++++++
  1 files changed, 9 insertions(+), 0 deletions(-)


diff --git a/Objects/object.c b/Objects/object.c
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -1598,6 +1598,15 @@
     if (PyType_Ready(&PyDict_Type) < 0)
         Py_FatalError("Can't initialize dict type");
 
+    if (PyType_Ready(&PyDictKeys_Type) < 0)
+        Py_FatalError("Can't initialize dict keys type");
+
+    if (PyType_Ready(&PyDictValues_Type) < 0)
+        Py_FatalError("Can't initialize dict values type");
+
+    if (PyType_Ready(&PyDictItems_Type) < 0)
+        Py_FatalError("Can't initialize dict items type");
+
     if (PyType_Ready(&PyODict_Type) < 0)
         Py_FatalError("Can't initialize OrderedDict type");
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 21:26:25 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Mon, 07 Nov 2016 02:26:25 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogaXNzdWUgIzI4NjIy?=
 =?utf-8?q?=3A_Remove_redundant_variable_annotation_test_from_test=5Fgramm?=
 =?utf-8?q?ar=2E_Ivan?=
Message-ID: <20161107022625.102444.81775.53616F73@psf.io>

https://hg.python.org/cpython/rev/048f82c86928
changeset:   104941:048f82c86928
branch:      3.6
parent:      104939:1f84be591d56
user:        Guido van Rossum <guido at python.org>
date:        Sun Nov 06 18:25:39 2016 -0800
summary:
  issue #28622: Remove redundant variable annotation test from test_grammar. Ivan L.

files:
  Lib/test/test_grammar.py |  3 ---
  1 files changed, 0 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py
--- a/Lib/test/test_grammar.py
+++ b/Lib/test/test_grammar.py
@@ -359,9 +359,6 @@
         self.assertEqual(ann_module.M.__annotations__,
                               {'123': 123, 'o': type})
         self.assertEqual(ann_module2.__annotations__, {})
-        self.assertEqual(typing.get_type_hints(ann_module2.CV,
-                                               ann_module2.__dict__),
-                         ChainMap({'var': typing.ClassVar[ann_module2.CV]}, {}))
 
     def test_var_annot_in_module(self):
         # check that functions fail the same way when executed

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 21:26:25 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Mon, 07 Nov 2016 02:26:25 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_issue_=2328622=3A_Remove_redundant_variable_annotation_t?=
 =?utf-8?q?est_from_test=5Fgrammar=2E_Ivan?=
Message-ID: <20161107022625.111819.34845.06F0292C@psf.io>

https://hg.python.org/cpython/rev/e60c1aef639a
changeset:   104942:e60c1aef639a
parent:      104940:cc71f2c5aa68
parent:      104941:048f82c86928
user:        Guido van Rossum <guido at python.org>
date:        Sun Nov 06 18:26:08 2016 -0800
summary:
  issue #28622: Remove redundant variable annotation test from test_grammar. Ivan L. (3.6->3.7)

files:
  Lib/test/test_grammar.py |  3 ---
  1 files changed, 0 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py
--- a/Lib/test/test_grammar.py
+++ b/Lib/test/test_grammar.py
@@ -359,9 +359,6 @@
         self.assertEqual(ann_module.M.__annotations__,
                               {'123': 123, 'o': type})
         self.assertEqual(ann_module2.__annotations__, {})
-        self.assertEqual(typing.get_type_hints(ann_module2.CV,
-                                               ann_module2.__dict__),
-                         ChainMap({'var': typing.ClassVar[ann_module2.CV]}, {}))
 
     def test_var_annot_in_module(self):
         # check that functions fail the same way when executed

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 22:36:02 2016
From: python-checkins at python.org (steve.dower)
Date: Mon, 07 Nov 2016 03:36:02 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogQ2xvc2VzICMyNzc4?=
 =?utf-8?q?1=3A_Removes_special_cases_for_the_experimental_aspect_of_PEP_5?=
 =?utf-8?q?29?=
Message-ID: <20161107033602.38935.66890.D18635A4@psf.io>

https://hg.python.org/cpython/rev/b26c8104e54f
changeset:   104943:b26c8104e54f
branch:      3.6
parent:      104941:048f82c86928
user:        Steve Dower <steve.dower at microsoft.com>
date:        Sun Nov 06 19:35:08 2016 -0800
summary:
  Closes #27781: Removes special cases for the experimental aspect of PEP 529

files:
  Doc/whatsnew/3.6.rst    |   5 -----
  Lib/test/test_os.py     |   9 ++-------
  Misc/NEWS               |   4 ++--
  Objects/unicodeobject.c |  11 +----------
  4 files changed, 5 insertions(+), 24 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -355,11 +355,6 @@
 See :pep:`529` for more information and discussion of code modifications that
 may be required.
 
-.. note::
-
-   This change is considered experimental for 3.6.0 beta releases. The default
-   encoding may change before the final release.
-
 .. _whatsnew-pep487:
 
 PEP 487: Simpler customization of class creation
diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
--- a/Lib/test/test_os.py
+++ b/Lib/test/test_os.py
@@ -2860,13 +2860,8 @@
                             func(name, *func_args)
                 except OSError as err:
                     self.assertIs(err.filename, name, str(func))
-                except RuntimeError as err:
-                    if sys.platform != 'win32':
-                        raise
-
-                    # issue27781: undecodable bytes currently raise RuntimeError
-                    # by 3.6.0b4 this will become UnicodeDecodeError or nothing
-                    self.assertIsInstance(err.__context__, UnicodeDecodeError)
+                except UnicodeDecodeError:
+                    pass
                 else:
                     self.fail("No exception thrown by {}".format(func))
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -3270,7 +3270,7 @@
 - Issue #24774: Fix docstring in http.server.test. Patch from Chiu-Hsiang Hsu.
 
 - Issue #21159: Improve message in configparser.InterpolationMissingOptionError.
-  Patch from ??ukasz Langa.
+  Patch from ?ukasz Langa.
 
 - Issue #20362: Honour TestCase.longMessage correctly in assertRegex.
   Patch from Ilia Kurenkov.
@@ -5198,7 +5198,7 @@
   Based on patch by Martin Panter.
 
 - Issue #17293: uuid.getnode() now determines MAC address on AIX using netstat.
-  Based on patch by Aivars Kalv??ns.
+  Based on patch by Aivars Kalv?ns.
 
 - Issue #22769: Fixed ttk.Treeview.tag_has() when called without arguments.
 
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -3843,18 +3843,9 @@
        cannot only rely on it: check also interp->fscodec_initialized for
        subinterpreters. */
     if (Py_FileSystemDefaultEncoding && interp->fscodec_initialized) {
-        PyObject *res = PyUnicode_Decode(s, size,
+        return PyUnicode_Decode(s, size,
                                 Py_FileSystemDefaultEncoding,
                                 Py_FileSystemDefaultEncodeErrors);
-#ifdef MS_WINDOWS
-        if (!res && PyErr_ExceptionMatches(PyExc_UnicodeDecodeError)) {
-            _PyErr_FormatFromCause(PyExc_RuntimeError,
-                "filesystem path bytes were not correctly encoded with '%s'. "
-                "Please report this at http://bugs.python.org/issue27781",
-                Py_FileSystemDefaultEncoding);
-        }
-#endif
-        return res;
     }
     else {
         return PyUnicode_DecodeLocaleAndSize(s, size, Py_FileSystemDefaultEncodeErrors);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov  6 22:36:02 2016
From: python-checkins at python.org (steve.dower)
Date: Mon, 07 Nov 2016 03:36:02 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Closes_=2327781=3A_Removes_special_cases_for_the_experim?=
 =?utf-8?q?ental_aspect_of_PEP_529?=
Message-ID: <20161107033602.3685.65155.6E39A54F@psf.io>

https://hg.python.org/cpython/rev/b8233c779ff7
changeset:   104944:b8233c779ff7
parent:      104942:e60c1aef639a
parent:      104943:b26c8104e54f
user:        Steve Dower <steve.dower at microsoft.com>
date:        Sun Nov 06 19:35:24 2016 -0800
summary:
  Closes #27781: Removes special cases for the experimental aspect of PEP 529

files:
  Doc/whatsnew/3.6.rst    |   5 -----
  Lib/test/test_os.py     |   9 ++-------
  Misc/NEWS               |   4 ++--
  Objects/unicodeobject.c |  11 +----------
  4 files changed, 5 insertions(+), 24 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -356,11 +356,6 @@
 See :pep:`529` for more information and discussion of code modifications that
 may be required.
 
-.. note::
-
-   This change is considered experimental for 3.6.0 beta releases. The default
-   encoding may change before the final release.
-
 .. _whatsnew-pep487:
 
 PEP 487: Simpler customization of class creation
diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
--- a/Lib/test/test_os.py
+++ b/Lib/test/test_os.py
@@ -2860,13 +2860,8 @@
                             func(name, *func_args)
                 except OSError as err:
                     self.assertIs(err.filename, name, str(func))
-                except RuntimeError as err:
-                    if sys.platform != 'win32':
-                        raise
-
-                    # issue27781: undecodable bytes currently raise RuntimeError
-                    # by 3.6.0b4 this will become UnicodeDecodeError or nothing
-                    self.assertIsInstance(err.__context__, UnicodeDecodeError)
+                except UnicodeDecodeError:
+                    pass
                 else:
                     self.fail("No exception thrown by {}".format(func))
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -3278,7 +3278,7 @@
 - Issue #24774: Fix docstring in http.server.test. Patch from Chiu-Hsiang Hsu.
 
 - Issue #21159: Improve message in configparser.InterpolationMissingOptionError.
-  Patch from ??ukasz Langa.
+  Patch from ?ukasz Langa.
 
 - Issue #20362: Honour TestCase.longMessage correctly in assertRegex.
   Patch from Ilia Kurenkov.
@@ -5206,7 +5206,7 @@
   Based on patch by Martin Panter.
 
 - Issue #17293: uuid.getnode() now determines MAC address on AIX using netstat.
-  Based on patch by Aivars Kalv??ns.
+  Based on patch by Aivars Kalv?ns.
 
 - Issue #22769: Fixed ttk.Treeview.tag_has() when called without arguments.
 
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -3832,18 +3832,9 @@
        cannot only rely on it: check also interp->fscodec_initialized for
        subinterpreters. */
     if (Py_FileSystemDefaultEncoding && interp->fscodec_initialized) {
-        PyObject *res = PyUnicode_Decode(s, size,
+        return PyUnicode_Decode(s, size,
                                 Py_FileSystemDefaultEncoding,
                                 Py_FileSystemDefaultEncodeErrors);
-#ifdef MS_WINDOWS
-        if (!res && PyErr_ExceptionMatches(PyExc_UnicodeDecodeError)) {
-            _PyErr_FormatFromCause(PyExc_RuntimeError,
-                "filesystem path bytes were not correctly encoded with '%s'. "
-                "Please report this at http://bugs.python.org/issue27781",
-                Py_FileSystemDefaultEncoding);
-        }
-#endif
-        return res;
     }
     else {
         return PyUnicode_DecodeLocaleAndSize(s, size, Py_FileSystemDefaultEncodeErrors);

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Mon Nov  7 11:25:12 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Mon, 7 Nov 2016 16:25:12 +0000
Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7
 2016-11-07
Message-ID: <4e9679a2-6725-4888-b4e6-1eb20d195a9d@irsmsx101.ger.corp.intel.com>

Results for project Python 2.7, build date 2016-11-07 03:47:59 +0000
commit:		2b58cde6a7f2
previous commit:	91f024fc9b3a
revision date:	2016-11-05 03:15:20 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.15%          0.35%          5.08%             5.99%
:-)             pybench      0.16%          0.11%          5.80%             3.43%
:-(            regex_v8      0.68%          0.30%         -2.07%             9.94%
:-)               nbody      0.07%         -0.00%          7.96%             2.71%
:-)        json_dump_v2      0.27%          0.15%          2.72%             9.88%
:-|      normal_startup      1.13%          0.00%         -0.51%             1.77%
:-)             ssbench      0.20%         -0.20%          2.34%             1.13%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-11-07/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Mon Nov  7 11:25:31 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Mon, 7 Nov 2016 16:25:31 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python Default
 2016-11-07
Message-ID: <c0befbe0-9fee-45df-873e-472d87b1591e@irsmsx101.ger.corp.intel.com>

Results for project Python default, build date 2016-11-07 03:01:35 +0000
commit:		e60c1aef639a
previous commit:	4000de2dcd24
revision date:	2016-11-07 02:26:08 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.17%          1.18%          5.71%            14.29%
:-)             pybench      0.13%         -0.19%          5.24%             4.91%
:-(            regex_v8      3.80%         -0.38%         -2.86%             0.84%
:-)               nbody      0.24%         -0.74%          2.14%             5.80%
:-(        json_dump_v2      0.26%          1.73%         -9.03%            15.03%
:-|      normal_startup      0.37%         -0.40%         -0.68%             6.29%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-11-07/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Mon Nov  7 14:29:23 2016
From: python-checkins at python.org (zach.ware)
Date: Mon, 07 Nov 2016 19:29:23 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogRml4IFBDL1ZTOS4w?=
 =?utf-8?q?/build=5Fssl=2Epy_for_recent_OpenSSL?=
Message-ID: <20161107192923.29950.50307.61C0ABE2@psf.io>

https://hg.python.org/cpython/rev/1f61a5d2f448
changeset:   104945:1f61a5d2f448
branch:      2.7
parent:      104919:2b58cde6a7f2
user:        Zachary Ware <zachary.ware at gmail.com>
date:        Mon Nov 07 13:29:07 2016 -0600
summary:
  Fix PC/VS9.0/build_ssl.py for recent OpenSSL

files:
  PC/VS9.0/build_ssl.py |  13 ++++++++-----
  1 files changed, 8 insertions(+), 5 deletions(-)


diff --git a/PC/VS9.0/build_ssl.py b/PC/VS9.0/build_ssl.py
--- a/PC/VS9.0/build_ssl.py
+++ b/PC/VS9.0/build_ssl.py
@@ -26,6 +26,7 @@
 
 from __future__ import with_statement
 import os, sys, re, shutil
+import subprocess
 
 # Find all "foo.exe" files on the PATH.
 def find_all_on_path(filename, extras = None):
@@ -48,12 +49,12 @@
 # is available.
 def find_working_perl(perls):
     for perl in perls:
-        fh = os.popen('"%s" -e "use Win32;"' % perl)
-        fh.read()
-        rc = fh.close()
-        if rc:
+        try:
+            subprocess.check_output([perl, "-e", "use win32;"])
+        except Subprocess.CalledProcessError:
             continue
-        return perl
+        else:
+            return perl
     print("Can not find a suitable PERL:")
     if perls:
         print(" the following perl interpreters were found:")
@@ -163,6 +164,8 @@
     perl = find_working_perl(perls)
     if perl:
         print("Found a working perl at '%s'" % (perl,))
+        # Set PERL for the makefile to find it
+        os.environ["PERL"] = perl
     else:
         print("No Perl installation was found. Existing Makefiles are used.")
     sys.stdout.flush()

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 15:21:39 2016
From: python-checkins at python.org (berker.peksag)
Date: Mon, 07 Nov 2016 20:21:39 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328088=3A_Merge_from_3=2E5?=
Message-ID: <20161107202139.7102.71334.0FFFFAD5@psf.io>

https://hg.python.org/cpython/rev/f2858945c058
changeset:   104947:f2858945c058
branch:      3.6
parent:      104943:b26c8104e54f
parent:      104946:f4e86b1b051e
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Mon Nov 07 23:36:50 2016 +0300
summary:
  Issue #28088: Merge from 3.5

files:
  Doc/library/asyncio-protocol.rst |  8 ++++----
  1 files changed, 4 insertions(+), 4 deletions(-)


diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst
--- a/Doc/library/asyncio-protocol.rst
+++ b/Doc/library/asyncio-protocol.rst
@@ -87,18 +87,18 @@
 
         - ``'subprocess'``: :class:`subprocess.Popen` instance
 
-   .. method:: set_protocol(self, protocol)
+   .. method:: set_protocol(protocol)
 
       Set a new protocol.  Switching protocol should only be done when both
       protocols are documented to support the switch.
 
-      .. versionadded:: 3.6.0
+      .. versionadded:: 3.5.3
 
-   .. method:: get_protocol(self)
+   .. method:: get_protocol
 
       Return the current protocol.
 
-      .. versionadded:: 3.6.0
+      .. versionadded:: 3.5.3
 
    .. versionchanged:: 3.5.1
       ``'ssl_object'`` info was added to SSL sockets.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 15:21:39 2016
From: python-checkins at python.org (berker.peksag)
Date: Mon, 07 Nov 2016 20:21:39 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328088=3A_Merge_from_3=2E6?=
Message-ID: <20161107202139.3264.28997.8C06FA93@psf.io>

https://hg.python.org/cpython/rev/62b7614970bd
changeset:   104948:62b7614970bd
parent:      104944:b8233c779ff7
parent:      104947:f2858945c058
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Mon Nov 07 23:37:12 2016 +0300
summary:
  Issue #28088: Merge from 3.6

files:
  Doc/library/asyncio-protocol.rst |  8 ++++----
  1 files changed, 4 insertions(+), 4 deletions(-)


diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst
--- a/Doc/library/asyncio-protocol.rst
+++ b/Doc/library/asyncio-protocol.rst
@@ -87,18 +87,18 @@
 
         - ``'subprocess'``: :class:`subprocess.Popen` instance
 
-   .. method:: set_protocol(self, protocol)
+   .. method:: set_protocol(protocol)
 
       Set a new protocol.  Switching protocol should only be done when both
       protocols are documented to support the switch.
 
-      .. versionadded:: 3.6.0
+      .. versionadded:: 3.5.3
 
-   .. method:: get_protocol(self)
+   .. method:: get_protocol
 
       Return the current protocol.
 
-      .. versionadded:: 3.6.0
+      .. versionadded:: 3.5.3
 
    .. versionchanged:: 3.5.1
       ``'ssl_object'`` info was added to SSL sockets.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 15:21:39 2016
From: python-checkins at python.org (berker.peksag)
Date: Mon, 07 Nov 2016 20:21:39 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4MDg4?=
 =?utf-8?q?=3A_Don=27t_include_self_in_method_signature?=
Message-ID: <20161107202139.30840.16783.49B9CF00@psf.io>

https://hg.python.org/cpython/rev/f4e86b1b051e
changeset:   104946:f4e86b1b051e
branch:      3.5
parent:      104938:ec96d06462a2
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Mon Nov 07 23:36:14 2016 +0300
summary:
  Issue #28088: Don't include self in method signature

Also update versionadded directive to 3.5.3.

files:
  Doc/library/asyncio-protocol.rst |  8 ++++----
  1 files changed, 4 insertions(+), 4 deletions(-)


diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst
--- a/Doc/library/asyncio-protocol.rst
+++ b/Doc/library/asyncio-protocol.rst
@@ -87,18 +87,18 @@
 
         - ``'subprocess'``: :class:`subprocess.Popen` instance
 
-   .. method:: set_protocol(self, protocol)
+   .. method:: set_protocol(protocol)
 
       Set a new protocol.  Switching protocol should only be done when both
       protocols are documented to support the switch.
 
-      .. versionadded:: 3.6.0
+      .. versionadded:: 3.5.3
 
-   .. method:: get_protocol(self)
+   .. method:: get_protocol
 
       Return the current protocol.
 
-      .. versionadded:: 3.6.0
+      .. versionadded:: 3.5.3
 
    .. versionchanged:: 3.5.1
       ``'ssl_object'`` info was added to SSL sockets.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 15:36:09 2016
From: python-checkins at python.org (yury.selivanov)
Date: Mon, 07 Nov 2016 20:36:09 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3Mzky?=
 =?utf-8?q?=3A_Document_loop=2Econnect=5Faccepted=5Fsocket=28=29?=
Message-ID: <20161107203608.37454.89066.1EFF728E@psf.io>

https://hg.python.org/cpython/rev/3e6570231c80
changeset:   104949:3e6570231c80
branch:      3.5
parent:      104946:f4e86b1b051e
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 07 15:35:25 2016 -0500
summary:
  Issue #27392: Document loop.connect_accepted_socket()

Patch by Jim Fulton.

files:
  Doc/library/asyncio-eventloop.rst |  17 +++++++++++++++++
  1 files changed, 17 insertions(+), 0 deletions(-)


diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst
--- a/Doc/library/asyncio-eventloop.rst
+++ b/Doc/library/asyncio-eventloop.rst
@@ -461,6 +461,23 @@
 
    Availability: UNIX.
 
+.. coroutinemethod:: BaseEventLoop.connect_accepted_socket(protocol_factory, sock, \*, ssl=None)
+
+   Handle an accepted connection.
+
+   This is used by servers that accept connections outside of
+   asyncio but that use asyncio to handle them.
+
+   Parameters:
+
+   * *sock* is a preexisting socket object returned from an ``accept``
+     call.
+
+   * *ssl* can be set to an :class:`~ssl.SSLContext` to enable SSL over the
+     accepted connections.
+
+   This method is a :ref:`coroutine <coroutine>`.  When completed, the
+   coroutine returns a ``(transport, protocol)`` pair.
 
 Watch file descriptors
 ----------------------

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 15:36:09 2016
From: python-checkins at python.org (yury.selivanov)
Date: Mon, 07 Nov 2016 20:36:09 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5_=28issue_=2327392=29?=
Message-ID: <20161107203608.111770.7840.2D5E1F01@psf.io>

https://hg.python.org/cpython/rev/6811df9e9797
changeset:   104950:6811df9e9797
branch:      3.6
parent:      104947:f2858945c058
parent:      104949:3e6570231c80
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 07 15:35:45 2016 -0500
summary:
  Merge 3.5 (issue #27392)

files:
  Doc/library/asyncio-eventloop.rst |  17 +++++++++++++++++
  1 files changed, 17 insertions(+), 0 deletions(-)


diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst
--- a/Doc/library/asyncio-eventloop.rst
+++ b/Doc/library/asyncio-eventloop.rst
@@ -462,6 +462,23 @@
 
    Availability: UNIX.
 
+.. coroutinemethod:: BaseEventLoop.connect_accepted_socket(protocol_factory, sock, \*, ssl=None)
+
+   Handle an accepted connection.
+
+   This is used by servers that accept connections outside of
+   asyncio but that use asyncio to handle them.
+
+   Parameters:
+
+   * *sock* is a preexisting socket object returned from an ``accept``
+     call.
+
+   * *ssl* can be set to an :class:`~ssl.SSLContext` to enable SSL over the
+     accepted connections.
+
+   This method is a :ref:`coroutine <coroutine>`.  When completed, the
+   coroutine returns a ``(transport, protocol)`` pair.
 
 Watch file descriptors
 ----------------------

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 15:36:09 2016
From: python-checkins at python.org (yury.selivanov)
Date: Mon, 07 Nov 2016 20:36:09 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjczOTIp?=
Message-ID: <20161107203609.29926.39740.4C2299F2@psf.io>

https://hg.python.org/cpython/rev/573bc1f9900e
changeset:   104951:573bc1f9900e
parent:      104948:62b7614970bd
parent:      104950:6811df9e9797
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 07 15:36:03 2016 -0500
summary:
  Merge 3.6 (issue #27392)

files:
  Doc/library/asyncio-eventloop.rst |  17 +++++++++++++++++
  1 files changed, 17 insertions(+), 0 deletions(-)


diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst
--- a/Doc/library/asyncio-eventloop.rst
+++ b/Doc/library/asyncio-eventloop.rst
@@ -462,6 +462,23 @@
 
    Availability: UNIX.
 
+.. coroutinemethod:: BaseEventLoop.connect_accepted_socket(protocol_factory, sock, \*, ssl=None)
+
+   Handle an accepted connection.
+
+   This is used by servers that accept connections outside of
+   asyncio but that use asyncio to handle them.
+
+   Parameters:
+
+   * *sock* is a preexisting socket object returned from an ``accept``
+     call.
+
+   * *ssl* can be set to an :class:`~ssl.SSLContext` to enable SSL over the
+     accepted connections.
+
+   This method is a :ref:`coroutine <coroutine>`.  When completed, the
+   coroutine returns a ``(transport, protocol)`` pair.
 
 Watch file descriptors
 ----------------------

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 16:08:03 2016
From: python-checkins at python.org (yury.selivanov)
Date: Mon, 07 Nov 2016 21:08:03 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjM0?=
 =?utf-8?q?=3A_Fix_asyncio=2Eisfuture=28=29_to_support_mocks?=
Message-ID: <20161107210802.37680.71315.99E0E7F7@psf.io>

https://hg.python.org/cpython/rev/0669dcf1eb36
changeset:   104952:0669dcf1eb36
branch:      3.5
parent:      104949:3e6570231c80
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 07 16:00:50 2016 -0500
summary:
  Issue #28634: Fix asyncio.isfuture() to support mocks

files:
  Lib/asyncio/futures.py                |   5 +-
  Lib/test/test_asyncio/test_futures.py |  23 +++++++++++++++
  2 files changed, 26 insertions(+), 2 deletions(-)


diff --git a/Lib/asyncio/futures.py b/Lib/asyncio/futures.py
--- a/Lib/asyncio/futures.py
+++ b/Lib/asyncio/futures.py
@@ -2,7 +2,7 @@
 
 __all__ = ['CancelledError', 'TimeoutError',
            'InvalidStateError',
-           'Future', 'wrap_future',
+           'Future', 'wrap_future', 'isfuture',
            ]
 
 import concurrent.futures._base
@@ -117,7 +117,8 @@
     itself as duck-type compatible by setting _asyncio_future_blocking.
     See comment in Future for more details.
     """
-    return getattr(obj, '_asyncio_future_blocking', None) is not None
+    return (hasattr(obj.__class__, '_asyncio_future_blocking') and
+            obj._asyncio_future_blocking is not None)
 
 
 class Future:
diff --git a/Lib/test/test_asyncio/test_futures.py b/Lib/test/test_asyncio/test_futures.py
--- a/Lib/test/test_asyncio/test_futures.py
+++ b/Lib/test/test_asyncio/test_futures.py
@@ -101,6 +101,29 @@
         self.loop = self.new_test_loop()
         self.addCleanup(self.loop.close)
 
+    def test_isfuture(self):
+        class MyFuture:
+            _asyncio_future_blocking = None
+
+            def __init__(self):
+                self._asyncio_future_blocking = False
+
+        self.assertFalse(asyncio.isfuture(MyFuture))
+        self.assertTrue(asyncio.isfuture(MyFuture()))
+
+        self.assertFalse(asyncio.isfuture(1))
+        self.assertFalse(asyncio.isfuture(asyncio.Future))
+
+        # As `isinstance(Mock(), Future)` returns `False`
+        self.assertFalse(asyncio.isfuture(mock.Mock()))
+
+        # As `isinstance(Mock(Future), Future)` returns `True`
+        self.assertTrue(asyncio.isfuture(mock.Mock(asyncio.Future)))
+
+        f = asyncio.Future(loop=self.loop)
+        self.assertTrue(asyncio.isfuture(f))
+        f.cancel()
+
     def test_initial_state(self):
         f = asyncio.Future(loop=self.loop)
         self.assertFalse(f.cancelled())

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 16:08:03 2016
From: python-checkins at python.org (yury.selivanov)
Date: Mon, 07 Nov 2016 21:08:03 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg2MzQp?=
Message-ID: <20161107210803.102175.58374.49E87950@psf.io>

https://hg.python.org/cpython/rev/44c1ec7689e5
changeset:   104954:44c1ec7689e5
parent:      104951:573bc1f9900e
parent:      104953:a02915e4c165
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 07 16:07:58 2016 -0500
summary:
  Merge 3.6 (issue #28634)

files:
  Lib/asyncio/base_futures.py           |   3 +-
  Lib/test/test_asyncio/test_futures.py |  23 +++++++++++++++
  2 files changed, 25 insertions(+), 1 deletions(-)


diff --git a/Lib/asyncio/base_futures.py b/Lib/asyncio/base_futures.py
--- a/Lib/asyncio/base_futures.py
+++ b/Lib/asyncio/base_futures.py
@@ -27,7 +27,8 @@
     itself as duck-type compatible by setting _asyncio_future_blocking.
     See comment in Future for more details.
     """
-    return getattr(obj, '_asyncio_future_blocking', None) is not None
+    return (hasattr(obj.__class__, '_asyncio_future_blocking') and
+            obj._asyncio_future_blocking is not None)
 
 
 def _format_callbacks(cb):
diff --git a/Lib/test/test_asyncio/test_futures.py b/Lib/test/test_asyncio/test_futures.py
--- a/Lib/test/test_asyncio/test_futures.py
+++ b/Lib/test/test_asyncio/test_futures.py
@@ -105,6 +105,29 @@
         self.loop = self.new_test_loop()
         self.addCleanup(self.loop.close)
 
+    def test_isfuture(self):
+        class MyFuture:
+            _asyncio_future_blocking = None
+
+            def __init__(self):
+                self._asyncio_future_blocking = False
+
+        self.assertFalse(asyncio.isfuture(MyFuture))
+        self.assertTrue(asyncio.isfuture(MyFuture()))
+        self.assertFalse(asyncio.isfuture(1))
+
+        # As `isinstance(Mock(), Future)` returns `False`
+        self.assertFalse(asyncio.isfuture(mock.Mock()))
+
+        f = self._new_future(loop=self.loop)
+        self.assertTrue(asyncio.isfuture(f))
+        self.assertFalse(asyncio.isfuture(type(f)))
+
+        # As `isinstance(Mock(Future), Future)` returns `True`
+        self.assertTrue(asyncio.isfuture(mock.Mock(type(f))))
+
+        f.cancel()
+
     def test_initial_state(self):
         f = self._new_future(loop=self.loop)
         self.assertFalse(f.cancelled())

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 16:08:04 2016
From: python-checkins at python.org (yury.selivanov)
Date: Mon, 07 Nov 2016 21:08:04 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5_=28issue_=2328634=29?=
Message-ID: <20161107210802.3212.63601.8F55E91D@psf.io>

https://hg.python.org/cpython/rev/a02915e4c165
changeset:   104953:a02915e4c165
branch:      3.6
parent:      104950:6811df9e9797
parent:      104952:0669dcf1eb36
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 07 16:07:30 2016 -0500
summary:
  Merge 3.5 (issue #28634)

files:
  Lib/asyncio/base_futures.py           |   3 +-
  Lib/test/test_asyncio/test_futures.py |  23 +++++++++++++++
  Misc/NEWS                             |   2 +
  3 files changed, 27 insertions(+), 1 deletions(-)


diff --git a/Lib/asyncio/base_futures.py b/Lib/asyncio/base_futures.py
--- a/Lib/asyncio/base_futures.py
+++ b/Lib/asyncio/base_futures.py
@@ -27,7 +27,8 @@
     itself as duck-type compatible by setting _asyncio_future_blocking.
     See comment in Future for more details.
     """
-    return getattr(obj, '_asyncio_future_blocking', None) is not None
+    return (hasattr(obj.__class__, '_asyncio_future_blocking') and
+            obj._asyncio_future_blocking is not None)
 
 
 def _format_callbacks(cb):
diff --git a/Lib/test/test_asyncio/test_futures.py b/Lib/test/test_asyncio/test_futures.py
--- a/Lib/test/test_asyncio/test_futures.py
+++ b/Lib/test/test_asyncio/test_futures.py
@@ -105,6 +105,29 @@
         self.loop = self.new_test_loop()
         self.addCleanup(self.loop.close)
 
+    def test_isfuture(self):
+        class MyFuture:
+            _asyncio_future_blocking = None
+
+            def __init__(self):
+                self._asyncio_future_blocking = False
+
+        self.assertFalse(asyncio.isfuture(MyFuture))
+        self.assertTrue(asyncio.isfuture(MyFuture()))
+        self.assertFalse(asyncio.isfuture(1))
+
+        # As `isinstance(Mock(), Future)` returns `False`
+        self.assertFalse(asyncio.isfuture(mock.Mock()))
+
+        f = self._new_future(loop=self.loop)
+        self.assertTrue(asyncio.isfuture(f))
+        self.assertFalse(asyncio.isfuture(type(f)))
+
+        # As `isinstance(Mock(Future), Future)` returns `True`
+        self.assertTrue(asyncio.isfuture(mock.Mock(type(f))))
+
+        f.cancel()
+
     def test_initial_state(self):
         f = self._new_future(loop=self.loop)
         self.assertFalse(f.cancelled())
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -25,6 +25,8 @@
 - Issue #28613: Fix get_event_loop() return the current loop if
   called from coroutines/callbacks.
 
+- Issue #28634: Fix asyncio.isfuture() to support unittest.Mock.
+
 Documentation
 -------------
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 16:44:31 2016
From: python-checkins at python.org (yury.selivanov)
Date: Mon, 07 Nov 2016 21:44:31 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg2MzUp?=
Message-ID: <20161107214431.7144.31481.28E015BD@psf.io>

https://hg.python.org/cpython/rev/38c806f0943b
changeset:   104956:38c806f0943b
parent:      104954:44c1ec7689e5
parent:      104955:5cf13fd36af1
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 07 16:44:27 2016 -0500
summary:
  Merge 3.6 (issue #28635)

files:
  Doc/whatsnew/3.6.rst |  588 ++++++++++++++++++------------
  1 files changed, 360 insertions(+), 228 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -4,6 +4,7 @@
 
 :Release: |release|
 :Date: |today|
+:Editors: Elvis Pranskevichus <elvis at magic.io>, Yury Selivanov <yury at magic.io>
 
 .. Rules for maintenance:
 
@@ -45,16 +46,17 @@
    This saves the maintainer the effort of going through the Mercurial log
    when researching a change.
 
+.. note::
+
+  Prerelease users should be aware that this document is currently in draft
+  form. It will be updated substantially as Python 3.6 moves towards release,
+  so it's worth checking back even after reading earlier versions.
+
 This article explains the new features in Python 3.6, compared to 3.5.
 
-For full details, see the
-`changelog <https://docs.python.org/3.6/whatsnew/changelog.html>`_.
+.. seealso::
 
-.. note::
-
-   Prerelease users should be aware that this document is currently in draft
-   form. It will be updated substantially as Python 3.6 moves towards release,
-   so it's worth checking back even after reading earlier versions.
+    :pep:`494` - Python 3.6 Release Schedule
 
 
 Summary -- Release highlights
@@ -65,40 +67,73 @@
 
 New syntax features:
 
-* A ``global`` or ``nonlocal`` statement must now textually appear
-  before the first use of the affected name in the same scope.
-  Previously this was a SyntaxWarning.
+* :ref:`PEP 498 <whatsnew36-pep498>`, formatted string literals.
 
-* PEP 498: :ref:`Formatted string literals <whatsnew-fstrings>`
+* :ref:`PEP 515 <whatsnew36-pep515>`, underscores in numeric literals.
 
-* PEP 515: Underscores in Numeric Literals
+* :ref:`PEP 526 <whatsnew36-pep526>`, syntax for variable annotations.
 
-* PEP 526: :ref:`Syntax for Variable Annotations <variable-annotations>`
+* :ref:`PEP 525 <whatsnew36-pep525>`, asynchronous generators.
 
-* PEP 525: Asynchronous Generators
+* :ref:`PEP 530 <whatsnew36-pep530>`: asynchronous comprehensions.
 
-* PEP 530: Asynchronous Comprehensions
 
-Standard library improvements:
+New library modules:
+
+* :mod:`secrets`: :ref:`PEP 506 -- Adding A Secrets Module To The Standard Library <whatsnew36-pep506>`.
+
+
+CPython implementation improvements:
+
+* The :ref:`dict <typesmapping>` type has been reimplemented to use
+  a :ref:`faster, more compact representation <whatsnew36-compactdict>`
+  similar to the `PyPy dict implementation`_. This resulted in dictionaries
+  using 20% to 25% less memory when compared to Python 3.5.
+
+* Customization of class creation has been simplified with the
+  :ref:`new protocol <whatsnew36-pep487>`.
+
+* The class attibute definition order is
+  :ref:`now preserved  <whatsnew36-pep520>`.
+
+* The order of elements in ``**kwargs`` now corresponds to the order in
+  the function signature:
+  :ref:`Preserving Keyword Argument Order <whatsnew36-pep468>`.
+
+* DTrace and SystemTap :ref:`probing support <whatsnew36-tracing>`.
+
+
+Significant improvements in the standard library:
+
+* A new :ref:`file system path protocol <whatsnew36-pep519>` has been
+  implemented to support :term:`path-like objects <path-like object>`.
+  All standard library functions operating on paths have been updated to
+  work with the new protocol.
+
+* The overhead of :mod:`asyncio` implementation has been reduced by
+  up to 50% thanks to the new C implementation of the :class:`asyncio.Future`
+  and :class:`asyncio.Task` classes and other optimizations.
+
 
 Security improvements:
 
-* On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool
-  is initialized to increase the security. See the :pep:`524` for the
+* On Linux, :func:`os.urandom` now blocks until the system urandom entropy
+  pool is initialized to increase the security. See the :pep:`524` for the
   rationale.
 
-* :mod:`hashlib` and :mod:`ssl` now support OpenSSL 1.1.0.
+* The :mod:`hashlib` and :mod:`ssl` modules now support OpenSSL 1.1.0.
 
-* The default settings and feature set of the :mod:`ssl` have been improved.
+* The default settings and feature set of the :mod:`ssl` module have been
+  improved.
 
-* The :mod:`hashlib` module has got support for BLAKE2, SHA-3 and SHAKE hash
-  algorithms and :func:`~hashlib.scrypt` key derivation function.
+* The :mod:`hashlib` module received support for the BLAKE2, SHA-3 and SHAKE
+  hash algorithms and the :func:`~hashlib.scrypt` key derivation function.
+
 
 Windows improvements:
 
-* PEP 529: :ref:`Change Windows filesystem encoding to UTF-8 <pep-529>`
-
-* PEP 528: :ref:`Change Windows console encoding to UTF-8 <pep-528>`
+* :ref:`PEP 528 <whatsnew36-pep529>` and :ref:`PEP 529 <whatsnew36-pep529>`,
+  Windows filesystem and console encoding changed to UTF-8.
 
 * The ``py.exe`` launcher, when used interactively, no longer prefers
   Python 2 over Python 3 when the user doesn't specify a version (via
@@ -117,66 +152,111 @@
   :envvar:`PYTHONHOME`. See :ref:`the documentation <finding_modules>` for
   more information.
 
-.. PEP-sized items next.
-
-.. _pep-4XX:
-
-.. PEP 4XX: Virtual Environments
-.. =============================
-
-
-.. (Implemented by Foo Bar.)
-
-.. .. seealso::
-
-    :pep:`4XX` - Python Virtual Environments
-       PEP written by Carl Meyer
-
-New built-in features:
-
-* PEP 520: :ref:`Preserving Class Attribute Definition Order<whatsnew-deforder>`
-
-* PEP 468: :ref:`Preserving Keyword Argument Order<whatsnew-kwargs>`
 
 A complete list of PEP's implemented in Python 3.6:
 
-* :pep:`468`, :ref:`Preserving Keyword Argument Order<whatsnew-kwargs>`
-* :pep:`487`, :ref:`Simpler customization of class creation<whatsnew-pep487>`
+* :pep:`468`, :ref:`Preserving Keyword Argument Order <whatsnew36-pep468>`
+* :pep:`487`, :ref:`Simpler customization of class creation <whatsnew36-pep487>`
 * :pep:`495`, Local Time Disambiguation
-* :pep:`498`, :ref:`Formatted string literals <whatsnew-fstrings>`
+* :pep:`498`, :ref:`Formatted string literals <whatsnew36-pep498>`
 * :pep:`506`, Adding A Secrets Module To The Standard Library
-* :pep:`509`, :ref:`Add a private version to dict<whatsnew-pep509>`
-* :pep:`515`, :ref:`Underscores in Numeric Literals<pep-515>`
-* :pep:`519`, :ref:`Adding a file system path protocol<pep-519>`
-* :pep:`520`, :ref:`Preserving Class Attribute Definition Order<whatsnew-deforder>`
-* :pep:`523`, :ref:`Adding a frame evaluation API to CPython<pep-523>`
+* :pep:`509`, :ref:`Add a private version to dict <whatsnew36-pep509>`
+* :pep:`515`, :ref:`Underscores in Numeric Literals <whatsnew36-pep515>`
+* :pep:`519`, :ref:`Adding a file system path protocol <whatsnew36-pep519>`
+* :pep:`520`, :ref:`Preserving Class Attribute Definition Order <whatsnew36-pep520>`
+* :pep:`523`, :ref:`Adding a frame evaluation API to CPython <whatsnew36-pep523>`
 * :pep:`524`, Make os.urandom() blocking on Linux (during system startup)
 * :pep:`525`, Asynchronous Generators (provisional)
-* :pep:`526`, :ref:`Syntax for Variable Annotations (provisional)<variable-annotations>`
-* :pep:`528`, :ref:`Change Windows console encoding to UTF-8 (provisional)<pep-528>`
-* :pep:`529`, :ref:`Change Windows filesystem encoding to UTF-8 (provisional)<pep-529>`
+* :pep:`526`, :ref:`Syntax for Variable Annotations (provisional) <whatsnew36-pep526>`
+* :pep:`528`, :ref:`Change Windows console encoding to UTF-8 (provisional) <whatsnew36-pep528>`
+* :pep:`529`, :ref:`Change Windows filesystem encoding to UTF-8 (provisional) <whatsnew36-pep529>`
 * :pep:`530`, Asynchronous Comprehensions
 
 
+.. _PyPy dict implementation: https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html
+
+
 New Features
 ============
 
-.. _pep-515:
+.. _whatsnew36-pep498:
+
+PEP 498: Formatted string literals
+----------------------------------
+
+:pep:`498` introduces a new kind of string literals: *f-strings*, or
+*formatted string literals*.
+
+Formatted string literals are prefixed with ``'f'`` and are similar to
+the format strings accepted by :meth:`str.format`.  They contain replacement
+fields surrounded by curly braces.  The replacement fields are expressions,
+which are evaluated at run time, and then formatted using the
+:func:`format` protocol::
+
+    >>> name = "Fred"
+    >>> f"He said his name is {name}."
+    'He said his name is Fred.'
+
+.. seealso::
+
+    :pep:`498` -- Literal String Interpolation.
+       PEP written and implemented by Eric V. Smith.
+
+    :ref:`Feature documentation <f-strings>`.
+
+
+.. _whatsnew36-pep526:
+
+PEP 526: Syntax for variable annotations
+----------------------------------------
+
+:pep:`484` introduced the standard for type annotations of function
+parameters, a.k.a. type hints. This PEP adds syntax to Python for annotating
+the types of variables including class variables and instance variables::
+
+    primes: List[int] = []
+
+    captain: str  # Note: no initial value!
+
+    class Starship:
+        stats: Dict[str, int] = {}
+
+Just as for function annotations, the Python interpreter does not attach any
+particular meaning to variable annotations and only stores them in the
+``__annotations__`` attribute of a class or module.
+
+In contrast to variable declarations in statically typed languages,
+the goal of annotation syntax is to provide an easy way to specify structured
+type metadata for third party tools and libraries via the abstract syntax tree
+and the ``__annotations__`` attribute.
+
+.. seealso::
+
+   :pep:`526` -- Syntax for variable annotations.
+      PEP written by Ryan Gonzalez, Philip House, Ivan Levkivskyi, Lisa Roach,
+      and Guido van Rossum. Implemented by Ivan Levkivskyi.
+
+   Tools that use or will use the new syntax:
+   `mypy <http://github.com/python/mypy>`_,
+   `pytype <http://github.com/google/pytype>`_, PyCharm, etc.
+
+
+.. _whatsnew36-pep515:
 
 PEP 515: Underscores in Numeric Literals
 ----------------------------------------
 
-Prior to PEP 515, there was no support for writing long numeric
-literals with some form of separator to improve readability. For
-instance, how big is ``1000000000000000``? With :pep:`515`, though,
-you can use underscores to separate digits as desired to make numeric
-literals easier to read: ``1_000_000_000_000_000``. Underscores can be
-used with other numeric literals beyond integers, e.g.
-``0x_FF_FF_FF_FF``.
+:pep:`515` adds the ability to use underscores in numeric literals for
+improved readability.  For example::
+
+    >>> 1_000_000_000_000_000
+    1000000000000000
+    >>> 0x_FF_FF_FF_FF
+    4294967295
 
 Single underscores are allowed between digits and after any base
-specifier. More than a single underscore in a row, leading, or
-trailing underscores are not allowed.
+specifier.  Leading, trailing, or multiple underscores in a row are not
+allowed.
 
 .. seealso::
 
@@ -184,36 +264,115 @@
       PEP written by Georg Brandl and Serhiy Storchaka.
 
 
-.. _pep-523:
+.. _whatsnew36-pep525:
 
-PEP 523: Adding a frame evaluation API to CPython
--------------------------------------------------
+PEP 525: Asynchronous Generators
+--------------------------------
 
-While Python provides extensive support to customize how code
-executes, one place it has not done so is in the evaluation of frame
-objects. If you wanted some way to intercept frame evaluation in
-Python there really wasn't any way without directly manipulating
-function pointers for defined functions.
+:pep:`492` introduced support for native coroutines and ``async`` / ``await``
+syntax to Python 3.5.  A notable limitation of the Python 3.5 implementation
+is that it was not possible to use ``await`` and ``yield`` in the same
+function body.  In Python 3.6 this restriction has been lifted, making it
+possible to define *asynchronous generators*::
 
-:pep:`523` changes this by providing an API to make frame
-evaluation pluggable at the C level. This will allow for tools such
-as debuggers and JITs to intercept frame evaluation before the
-execution of Python code begins. This enables the use of alternative
-evaluation implementations for Python code, tracking frame
-evaluation, etc.
+    async def ticker(delay, to):
+        """Yield numbers from 0 to `to` every `delay` seconds."""
+        for i in range(to):
+            yield i
+            await asyncio.sleep(delay)
 
-This API is not part of the limited C API and is marked as private to
-signal that usage of this API is expected to be limited and only
-applicable to very select, low-level use-cases. Semantics of the
-API will change with Python as necessary.
+The new syntax allows for faster and more concise code.
 
 .. seealso::
 
-    :pep:`523` -- Adding a frame evaluation API to CPython
-       PEP written by Brett Cannon and Dino Viehland.
+   :pep:`525` -- Asynchronous Generators
+      PEP written and implemented by Yury Selivanov.
 
 
-.. _pep-519:
+.. _whatsnew36-pep530:
+
+PEP 530: Asynchronous Comprehensions
+------------------------------------
+
+:pep:`530` adds support for using ``async for`` in list, set, dict
+comprehensions and generator expressions::
+
+    result = [i async for i in aiter() if i % 2]
+
+Additionally, ``await`` expressions are supported in all kinds
+of comprehensions::
+
+    result = [await fun() for fun in funcs]
+
+.. seealso::
+
+ :pep:`530` -- Asynchronous Comprehensions
+    PEP written and implemented by Yury Selivanov.
+
+
+.. _whatsnew36-pep487:
+
+PEP 487: Simpler customization of class creation
+------------------------------------------------
+
+It is now possible to customize subclass creation without using a metaclass.
+The new ``__init_subclass__`` classmethod will be called on the base class
+whenever a new subclass is created::
+
+    >>> class QuestBase:
+    ...    # this is implicitly a @classmethod
+    ...    def __init_subclass__(cls, swallow, **kwargs):
+    ...        cls.swallow = swallow
+    ...        super().__init_subclass__(**kwargs)
+
+    >>> class Quest(QuestBase, swallow="african"):
+    ...    pass
+
+    >>> Quest.swallow
+    'african'
+
+.. seealso::
+
+ :pep:`487` -- Simpler customization of class creation
+    PEP written and implemented by Martin Teichmann.
+
+ :ref:`Feature documentation <class-customization>`
+
+
+.. _whatsnew36-pep487-descriptors:
+
+PEP 487: Descriptor Protocol Enhancements
+-----------------------------------------
+
+:pep:`487` extends the descriptor protocol has to include the new optional
+``__set_name__`` method.  Whenever a new class is defined, the new method
+will be called on all descriptors included in the definition, providing
+them with a reference to the class being defined and the name given to the
+descriptor within the class namespace.
+
+.. seealso::
+
+    :pep:`487` -- Simpler customization of class creation
+        PEP written and implemented by Martin Teichmann.
+
+    :ref:`Feature documentation <descriptors>`
+
+
+.. _whatsnew36-pep506:
+
+PEP 506: Adding A Secrets Module To The Standard Library
+--------------------------------------------------------
+
+
+
+.. seealso::
+
+    :pep:`506` -- Adding A Secrets Module To The Standard Library
+      PEP written and implemented by Steven D'Aprano.
+
+
+
+.. _whatsnew36-pep519:
 
 PEP 519: Adding a file system path protocol
 -------------------------------------------
@@ -280,60 +439,7 @@
        PEP written by Brett Cannon and Koos Zevenhoven.
 
 
-.. _whatsnew-fstrings:
-
-PEP 498: Formatted string literals
-----------------------------------
-
-Formatted string literals are a new kind of string literal, prefixed
-with ``'f'``.  They are similar to the format strings accepted by
-:meth:`str.format`.  They contain replacement fields surrounded by
-curly braces.  The replacement fields are expressions, which are
-evaluated at run time, and then formatted using the :func:`format` protocol::
-
-    >>> name = "Fred"
-    >>> f"He said his name is {name}."
-    'He said his name is Fred.'
-
-See :pep:`498` and the main documentation at :ref:`f-strings`.
-
-
-.. _variable-annotations:
-
-PEP 526: Syntax for variable annotations
-----------------------------------------
-
-:pep:`484` introduced standard for type annotations of function parameters,
-a.k.a. type hints. This PEP adds syntax to Python for annotating the
-types of variables including class variables and instance variables::
-
-    primes: List[int] = []
-
-    captain: str  # Note: no initial value!
-
-    class Starship:
-        stats: Dict[str, int] = {}
-
-Just as for function annotations, the Python interpreter does not attach any
-particular meaning to variable annotations and only stores them in a special
-attribute ``__annotations__`` of a class or module.
-In contrast to variable declarations in statically typed languages,
-the goal of annotation syntax is to provide an easy way to specify structured
-type metadata for third party tools and libraries via the abstract syntax tree
-and the ``__annotations__`` attribute.
-
-.. seealso::
-
-   :pep:`526` -- Syntax for variable annotations.
-      PEP written by Ryan Gonzalez, Philip House, Ivan Levkivskyi, Lisa Roach,
-      and Guido van Rossum. Implemented by Ivan Levkivskyi.
-
-   Tools that use or will use the new syntax:
-   `mypy <http://github.com/python/mypy>`_,
-   `pytype <http://github.com/google/pytype>`_, PyCharm, etc.
-
-
-.. _pep-529:
+.. _whatsnew36-pep529:
 
 PEP 529: Change Windows filesystem encoding to UTF-8
 ----------------------------------------------------
@@ -356,28 +462,8 @@
 See :pep:`529` for more information and discussion of code modifications that
 may be required.
 
-.. _whatsnew-pep487:
 
-PEP 487: Simpler customization of class creation
-------------------------------------------------
-
-Upon subclassing a class, the ``__init_subclass__`` classmethod (if defined) is
-called on the base class. This makes it straightforward to write classes that
-customize initialization of future subclasses without introducing the
-complexity of a full custom metaclass.
-
-The descriptor protocol has also been expanded to include a new optional method,
-``__set_name__``. Whenever a new class is defined, the new method will be called
-on all descriptors included in the definition, providing them with a reference
-to the class being defined and the name given to the descriptor within the
-class namespace.
-
-Also see :pep:`487` and the updated class customization documentation at
-:ref:`class-customization` and :ref:`descriptors`.
-
-(Contributed by Martin Teichmann in :issue:`27366`)
-
-.. _pep-528:
+.. _whatsnew36-pep528:
 
 PEP 528: Change Windows console encoding to UTF-8
 -------------------------------------------------
@@ -395,6 +481,104 @@
    :pep:`528` -- Change Windows console encoding to UTF-8
       PEP written and implemented by Steve Dower.
 
+
+.. _whatsnew36-pep520:
+
+PEP 520: Preserving Class Attribute Definition Order
+----------------------------------------------------
+
+Attributes in a class definition body have a natural ordering: the same
+order in which the names appear in the source.  This order is now
+preserved in the new class's ``__dict__`` attribute.
+
+Also, the effective default class *execution* namespace (returned from
+``type.__prepare__()``) is now an insertion-order-preserving mapping.
+
+.. seealso::
+
+   :pep:`520` -- Preserving Class Attribute Definition Order
+      PEP written and implemented by Eric Snow.
+
+
+.. _whatsnew36-pep468:
+
+PEP 468: Preserving Keyword Argument Order
+------------------------------------------
+
+``**kwargs`` in a function signature is now guaranteed to be an
+insertion-order-preserving mapping.
+
+.. seealso::
+
+   :pep:`468` -- Preserving Keyword Argument Order
+      PEP written and implemented by Eric Snow.
+
+
+.. _whatsnew36-compactdict:
+
+New :ref:`dict <typesmapping>` implementation
+---------------------------------------------
+
+The :ref:`dict <typesmapping>` type now uses a "compact" representation
+`pioneered by PyPy <https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html>`_.
+The memory usage of the new :func:`dict` is between 20% and 25% smaller
+compared to Python 3.5.
+:pep:`468` (Preserving the order of ``**kwargs`` in a function.) is
+implemented by this. The order-preserving aspect of this new
+implementation is considered an implementation detail and should
+not be relied upon (this may change in the future, but it is desired
+to have this new dict implementation in the language for a few
+releases before changing the language spec to mandate
+order-preserving semantics for all current and future Python
+implementations; this also helps preserve backwards-compatibility
+with older versions of the language where random iteration order is
+still in effect, e.g. Python 3.5).
+(Contributed by INADA Naoki in :issue:`27350`. Idea
+`originally suggested by Raymond Hettinger
+<https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_.)
+
+
+.. _whatsnew36-pep523:
+
+PEP 523: Adding a frame evaluation API to CPython
+-------------------------------------------------
+
+While Python provides extensive support to customize how code
+executes, one place it has not done so is in the evaluation of frame
+objects. If you wanted some way to intercept frame evaluation in
+Python there really wasn't any way without directly manipulating
+function pointers for defined functions.
+
+:pep:`523` changes this by providing an API to make frame
+evaluation pluggable at the C level. This will allow for tools such
+as debuggers and JITs to intercept frame evaluation before the
+execution of Python code begins. This enables the use of alternative
+evaluation implementations for Python code, tracking frame
+evaluation, etc.
+
+This API is not part of the limited C API and is marked as private to
+signal that usage of this API is expected to be limited and only
+applicable to very select, low-level use-cases. Semantics of the
+API will change with Python as necessary.
+
+.. seealso::
+
+  :pep:`523` -- Adding a frame evaluation API to CPython
+     PEP written by Brett Cannon and Dino Viehland.
+
+
+.. _whatsnew36-pep509:
+
+PEP 509: Add a private version to dict
+--------------------------------------
+
+Add a new private version to the builtin ``dict`` type, incremented at
+each dictionary creation and at each dictionary change, to implement
+fast guards on namespaces.
+
+(Contributed by Victor Stinner in :issue:`26058`.)
+
+
 PYTHONMALLOC environment variable
 ---------------------------------
 
@@ -469,6 +653,8 @@
 (Contributed by Victor Stinner in :issue:`26516` and :issue:`26564`.)
 
 
+.. _whatsnew36-tracing:
+
 DTrace and SystemTap probing support
 ------------------------------------
 
@@ -494,71 +680,14 @@
 Jes?s Cea Avi?n, David Malcolm, and Nikhil Benesch.)
 
 
-.. _whatsnew-deforder:
-
-PEP 520: Preserving Class Attribute Definition Order
-----------------------------------------------------
-
-Attributes in a class definition body have a natural ordering: the same
-order in which the names appear in the source.  This order is now
-preserved in the new class's ``__dict__`` attribute.
-
-Also, the effective default class *execution* namespace (returned from
-``type.__prepare__()``) is now an insertion-order-preserving mapping.
-
-.. seealso::
-
-   :pep:`520` -- Preserving Class Attribute Definition Order
-      PEP written and implemented by Eric Snow.
-
-
-.. _whatsnew-kwargs:
-
-PEP 468: Preserving Keyword Argument Order
-------------------------------------------
-
-``**kwargs`` in a function signature is now guaranteed to be an
-insertion-order-preserving mapping.
-
-.. seealso::
-
-   :pep:`468` -- Preserving Keyword Argument Order
-      PEP written and implemented by Eric Snow.
-
-.. _whatsnew-pep509:
-
-PEP 509: Add a private version to dict
---------------------------------------
-
-Add a new private version to the builtin ``dict`` type, incremented at
-each dictionary creation and at each dictionary change, to implement
-fast guards on namespaces.
-
-(Contributed by Victor Stinner in :issue:`26058`.)
-
-
 Other Language Changes
 ======================
 
 Some smaller changes made to the core Python language are:
 
-* :func:`dict` now uses a "compact" representation `pioneered by PyPy
-  <https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html>`_.
-  The memory usage of the new :func:`dict` is between 20% and 25% smaller
-  compared to Python 3.5.
-  :pep:`468` (Preserving the order of ``**kwargs`` in a function.) is
-  implemented by this. The order-preserving aspect of this new
-  implementation is considered an implementation detail and should
-  not be relied upon (this may change in the future, but it is desired
-  to have this new dict implementation in the language for a few
-  releases before changing the language spec to mandate
-  order-preserving semantics for all current and future Python
-  implementations; this also helps preserve backwards-compatibility
-  with older versions of the language where random iteration order is
-  still in effect, e.g. Python 3.5).
-  (Contributed by INADA Naoki in :issue:`27350`. Idea
-  `originally suggested by Raymond Hettinger
-  <https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_.)
+* A ``global`` or ``nonlocal`` statement must now textually appear
+  before the first use of the affected name in the same scope.
+  Previously this was a ``SyntaxWarning``.
 
 * Long sequences of repeated traceback lines are now abbreviated as
   ``"[Previous line repeated {count} more times]"`` (see
@@ -573,16 +702,15 @@
 New Modules
 ===========
 
-* None yet.
+secrets
+-------
+
+The new :mod:`secrets` module.
 
 
 Improved Modules
 ================
 
-On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool
-is initialized to increase the security. See the :pep:`524` for the rationale.
-
-
 asyncio
 -------
 
@@ -784,11 +912,14 @@
 will be emitted in its destructor.
 (Contributed by Serhiy Storchaka in :issue:`25994`.)
 
+On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool
+is initialized to increase the security. See the :pep:`524` for the rationale.
+
 The Linux ``getrandom()`` syscall (get random bytes) is now exposed as the new
 :func:`os.getrandom` function.
 (Contributed by Victor Stinner, part of the :pep:`524`)
 
-See the summary for :ref:`PEP 519 <pep-519>` for details on how the
+See the summary for :ref:`PEP 519 <whatsnew36-pep519>` for details on how the
 :mod:`os` and :mod:`os.path` modules now support
 :term:`path-like objects <path-like object>`.
 
@@ -1160,6 +1291,7 @@
   it is now about 1.5--4 times faster.
   (Contributed by Serhiy Storchaka in :issue:`26032`).
 
+
 Build and C API Changes
 =======================
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 16:44:31 2016
From: python-checkins at python.org (yury.selivanov)
Date: Mon, 07 Nov 2016 21:44:31 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogd2hhdHNuZXc6IElu?=
 =?utf-8?q?ital_pass_on_=22What=27s_New_in_Python_3=2E6=22?=
Message-ID: <20161107214431.7832.12838.178CC23C@psf.io>

https://hg.python.org/cpython/rev/5cf13fd36af1
changeset:   104955:5cf13fd36af1
branch:      3.6
parent:      104953:a02915e4c165
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 07 16:40:20 2016 -0500
summary:
  whatsnew: Inital pass on "What's New in Python 3.6"

Patch by Elvis Pranskevichus.

files:
  Doc/whatsnew/3.6.rst |  587 +++++++++++++++++++-----------
  1 files changed, 360 insertions(+), 227 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -4,6 +4,7 @@
 
 :Release: |release|
 :Date: |today|
+:Editors: Elvis Pranskevichus <elvis at magic.io>, Yury Selivanov <yury at magic.io>
 
 .. Rules for maintenance:
 
@@ -45,15 +46,17 @@
    This saves the maintainer the effort of going through the Mercurial log
    when researching a change.
 
+.. note::
+
+  Prerelease users should be aware that this document is currently in draft
+  form. It will be updated substantially as Python 3.6 moves towards release,
+  so it's worth checking back even after reading earlier versions.
+
 This article explains the new features in Python 3.6, compared to 3.5.
 
-For full details, see the :ref:`changelog <changelog>`.
+.. seealso::
 
-.. note::
-
-   Prerelease users should be aware that this document is currently in draft
-   form. It will be updated substantially as Python 3.6 moves towards release,
-   so it's worth checking back even after reading earlier versions.
+    :pep:`494` - Python 3.6 Release Schedule
 
 
 Summary -- Release highlights
@@ -64,40 +67,73 @@
 
 New syntax features:
 
-* A ``global`` or ``nonlocal`` statement must now textually appear
-  before the first use of the affected name in the same scope.
-  Previously this was a SyntaxWarning.
+* :ref:`PEP 498 <whatsnew36-pep498>`, formatted string literals.
 
-* PEP 498: :ref:`Formatted string literals <whatsnew-fstrings>`
+* :ref:`PEP 515 <whatsnew36-pep515>`, underscores in numeric literals.
 
-* PEP 515: Underscores in Numeric Literals
+* :ref:`PEP 526 <whatsnew36-pep526>`, syntax for variable annotations.
 
-* PEP 526: :ref:`Syntax for Variable Annotations <variable-annotations>`
+* :ref:`PEP 525 <whatsnew36-pep525>`, asynchronous generators.
 
-* PEP 525: Asynchronous Generators
+* :ref:`PEP 530 <whatsnew36-pep530>`: asynchronous comprehensions.
 
-* PEP 530: Asynchronous Comprehensions
 
-Standard library improvements:
+New library modules:
+
+* :mod:`secrets`: :ref:`PEP 506 -- Adding A Secrets Module To The Standard Library <whatsnew36-pep506>`.
+
+
+CPython implementation improvements:
+
+* The :ref:`dict <typesmapping>` type has been reimplemented to use
+  a :ref:`faster, more compact representation <whatsnew36-compactdict>`
+  similar to the `PyPy dict implementation`_. This resulted in dictionaries
+  using 20% to 25% less memory when compared to Python 3.5.
+
+* Customization of class creation has been simplified with the
+  :ref:`new protocol <whatsnew36-pep487>`.
+
+* The class attibute definition order is
+  :ref:`now preserved  <whatsnew36-pep520>`.
+
+* The order of elements in ``**kwargs`` now corresponds to the order in
+  the function signature:
+  :ref:`Preserving Keyword Argument Order <whatsnew36-pep468>`.
+
+* DTrace and SystemTap :ref:`probing support <whatsnew36-tracing>`.
+
+
+Significant improvements in the standard library:
+
+* A new :ref:`file system path protocol <whatsnew36-pep519>` has been
+  implemented to support :term:`path-like objects <path-like object>`.
+  All standard library functions operating on paths have been updated to
+  work with the new protocol.
+
+* The overhead of :mod:`asyncio` implementation has been reduced by
+  up to 50% thanks to the new C implementation of the :class:`asyncio.Future`
+  and :class:`asyncio.Task` classes and other optimizations.
+
 
 Security improvements:
 
-* On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool
-  is initialized to increase the security. See the :pep:`524` for the
+* On Linux, :func:`os.urandom` now blocks until the system urandom entropy
+  pool is initialized to increase the security. See the :pep:`524` for the
   rationale.
 
-* :mod:`hashlib` and :mod:`ssl` now support OpenSSL 1.1.0.
+* The :mod:`hashlib` and :mod:`ssl` modules now support OpenSSL 1.1.0.
 
-* The default settings and feature set of the :mod:`ssl` have been improved.
+* The default settings and feature set of the :mod:`ssl` module have been
+  improved.
 
-* The :mod:`hashlib` module has got support for BLAKE2, SHA-3 and SHAKE hash
-  algorithms and :func:`~hashlib.scrypt` key derivation function.
+* The :mod:`hashlib` module received support for the BLAKE2, SHA-3 and SHAKE
+  hash algorithms and the :func:`~hashlib.scrypt` key derivation function.
+
 
 Windows improvements:
 
-* PEP 529: :ref:`Change Windows filesystem encoding to UTF-8 <pep-529>`
-
-* PEP 528: :ref:`Change Windows console encoding to UTF-8 <pep-528>`
+* :ref:`PEP 528 <whatsnew36-pep529>` and :ref:`PEP 529 <whatsnew36-pep529>`,
+  Windows filesystem and console encoding changed to UTF-8.
 
 * The ``py.exe`` launcher, when used interactively, no longer prefers
   Python 2 over Python 3 when the user doesn't specify a version (via
@@ -116,66 +152,111 @@
   :envvar:`PYTHONHOME`. See :ref:`the documentation <finding_modules>` for
   more information.
 
-.. PEP-sized items next.
-
-.. _pep-4XX:
-
-.. PEP 4XX: Virtual Environments
-.. =============================
-
-
-.. (Implemented by Foo Bar.)
-
-.. .. seealso::
-
-    :pep:`4XX` - Python Virtual Environments
-       PEP written by Carl Meyer
-
-New built-in features:
-
-* PEP 520: :ref:`Preserving Class Attribute Definition Order<whatsnew-deforder>`
-
-* PEP 468: :ref:`Preserving Keyword Argument Order<whatsnew-kwargs>`
 
 A complete list of PEP's implemented in Python 3.6:
 
-* :pep:`468`, :ref:`Preserving Keyword Argument Order<whatsnew-kwargs>`
-* :pep:`487`, :ref:`Simpler customization of class creation<whatsnew-pep487>`
+* :pep:`468`, :ref:`Preserving Keyword Argument Order <whatsnew36-pep468>`
+* :pep:`487`, :ref:`Simpler customization of class creation <whatsnew36-pep487>`
 * :pep:`495`, Local Time Disambiguation
-* :pep:`498`, :ref:`Formatted string literals <whatsnew-fstrings>`
+* :pep:`498`, :ref:`Formatted string literals <whatsnew36-pep498>`
 * :pep:`506`, Adding A Secrets Module To The Standard Library
-* :pep:`509`, :ref:`Add a private version to dict<whatsnew-pep509>`
-* :pep:`515`, :ref:`Underscores in Numeric Literals<pep-515>`
-* :pep:`519`, :ref:`Adding a file system path protocol<pep-519>`
-* :pep:`520`, :ref:`Preserving Class Attribute Definition Order<whatsnew-deforder>`
-* :pep:`523`, :ref:`Adding a frame evaluation API to CPython<pep-523>`
+* :pep:`509`, :ref:`Add a private version to dict <whatsnew36-pep509>`
+* :pep:`515`, :ref:`Underscores in Numeric Literals <whatsnew36-pep515>`
+* :pep:`519`, :ref:`Adding a file system path protocol <whatsnew36-pep519>`
+* :pep:`520`, :ref:`Preserving Class Attribute Definition Order <whatsnew36-pep520>`
+* :pep:`523`, :ref:`Adding a frame evaluation API to CPython <whatsnew36-pep523>`
 * :pep:`524`, Make os.urandom() blocking on Linux (during system startup)
 * :pep:`525`, Asynchronous Generators (provisional)
-* :pep:`526`, :ref:`Syntax for Variable Annotations (provisional)<variable-annotations>`
-* :pep:`528`, :ref:`Change Windows console encoding to UTF-8 (provisional)<pep-528>`
-* :pep:`529`, :ref:`Change Windows filesystem encoding to UTF-8 (provisional)<pep-529>`
+* :pep:`526`, :ref:`Syntax for Variable Annotations (provisional) <whatsnew36-pep526>`
+* :pep:`528`, :ref:`Change Windows console encoding to UTF-8 (provisional) <whatsnew36-pep528>`
+* :pep:`529`, :ref:`Change Windows filesystem encoding to UTF-8 (provisional) <whatsnew36-pep529>`
 * :pep:`530`, Asynchronous Comprehensions
 
 
+.. _PyPy dict implementation: https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html
+
+
 New Features
 ============
 
-.. _pep-515:
+.. _whatsnew36-pep498:
+
+PEP 498: Formatted string literals
+----------------------------------
+
+:pep:`498` introduces a new kind of string literals: *f-strings*, or
+*formatted string literals*.
+
+Formatted string literals are prefixed with ``'f'`` and are similar to
+the format strings accepted by :meth:`str.format`.  They contain replacement
+fields surrounded by curly braces.  The replacement fields are expressions,
+which are evaluated at run time, and then formatted using the
+:func:`format` protocol::
+
+    >>> name = "Fred"
+    >>> f"He said his name is {name}."
+    'He said his name is Fred.'
+
+.. seealso::
+
+    :pep:`498` -- Literal String Interpolation.
+       PEP written and implemented by Eric V. Smith.
+
+    :ref:`Feature documentation <f-strings>`.
+
+
+.. _whatsnew36-pep526:
+
+PEP 526: Syntax for variable annotations
+----------------------------------------
+
+:pep:`484` introduced the standard for type annotations of function
+parameters, a.k.a. type hints. This PEP adds syntax to Python for annotating
+the types of variables including class variables and instance variables::
+
+    primes: List[int] = []
+
+    captain: str  # Note: no initial value!
+
+    class Starship:
+        stats: Dict[str, int] = {}
+
+Just as for function annotations, the Python interpreter does not attach any
+particular meaning to variable annotations and only stores them in the
+``__annotations__`` attribute of a class or module.
+
+In contrast to variable declarations in statically typed languages,
+the goal of annotation syntax is to provide an easy way to specify structured
+type metadata for third party tools and libraries via the abstract syntax tree
+and the ``__annotations__`` attribute.
+
+.. seealso::
+
+   :pep:`526` -- Syntax for variable annotations.
+      PEP written by Ryan Gonzalez, Philip House, Ivan Levkivskyi, Lisa Roach,
+      and Guido van Rossum. Implemented by Ivan Levkivskyi.
+
+   Tools that use or will use the new syntax:
+   `mypy <http://github.com/python/mypy>`_,
+   `pytype <http://github.com/google/pytype>`_, PyCharm, etc.
+
+
+.. _whatsnew36-pep515:
 
 PEP 515: Underscores in Numeric Literals
 ----------------------------------------
 
-Prior to PEP 515, there was no support for writing long numeric
-literals with some form of separator to improve readability. For
-instance, how big is ``1000000000000000``? With :pep:`515`, though,
-you can use underscores to separate digits as desired to make numeric
-literals easier to read: ``1_000_000_000_000_000``. Underscores can be
-used with other numeric literals beyond integers, e.g.
-``0x_FF_FF_FF_FF``.
+:pep:`515` adds the ability to use underscores in numeric literals for
+improved readability.  For example::
+
+    >>> 1_000_000_000_000_000
+    1000000000000000
+    >>> 0x_FF_FF_FF_FF
+    4294967295
 
 Single underscores are allowed between digits and after any base
-specifier. More than a single underscore in a row, leading, or
-trailing underscores are not allowed.
+specifier.  Leading, trailing, or multiple underscores in a row are not
+allowed.
 
 .. seealso::
 
@@ -183,36 +264,115 @@
       PEP written by Georg Brandl and Serhiy Storchaka.
 
 
-.. _pep-523:
+.. _whatsnew36-pep525:
 
-PEP 523: Adding a frame evaluation API to CPython
--------------------------------------------------
+PEP 525: Asynchronous Generators
+--------------------------------
 
-While Python provides extensive support to customize how code
-executes, one place it has not done so is in the evaluation of frame
-objects. If you wanted some way to intercept frame evaluation in
-Python there really wasn't any way without directly manipulating
-function pointers for defined functions.
+:pep:`492` introduced support for native coroutines and ``async`` / ``await``
+syntax to Python 3.5.  A notable limitation of the Python 3.5 implementation
+is that it was not possible to use ``await`` and ``yield`` in the same
+function body.  In Python 3.6 this restriction has been lifted, making it
+possible to define *asynchronous generators*::
 
-:pep:`523` changes this by providing an API to make frame
-evaluation pluggable at the C level. This will allow for tools such
-as debuggers and JITs to intercept frame evaluation before the
-execution of Python code begins. This enables the use of alternative
-evaluation implementations for Python code, tracking frame
-evaluation, etc.
+    async def ticker(delay, to):
+        """Yield numbers from 0 to `to` every `delay` seconds."""
+        for i in range(to):
+            yield i
+            await asyncio.sleep(delay)
 
-This API is not part of the limited C API and is marked as private to
-signal that usage of this API is expected to be limited and only
-applicable to very select, low-level use-cases. Semantics of the
-API will change with Python as necessary.
+The new syntax allows for faster and more concise code.
 
 .. seealso::
 
-    :pep:`523` -- Adding a frame evaluation API to CPython
-       PEP written by Brett Cannon and Dino Viehland.
+   :pep:`525` -- Asynchronous Generators
+      PEP written and implemented by Yury Selivanov.
 
 
-.. _pep-519:
+.. _whatsnew36-pep530:
+
+PEP 530: Asynchronous Comprehensions
+------------------------------------
+
+:pep:`530` adds support for using ``async for`` in list, set, dict
+comprehensions and generator expressions::
+
+    result = [i async for i in aiter() if i % 2]
+
+Additionally, ``await`` expressions are supported in all kinds
+of comprehensions::
+
+    result = [await fun() for fun in funcs]
+
+.. seealso::
+
+ :pep:`530` -- Asynchronous Comprehensions
+    PEP written and implemented by Yury Selivanov.
+
+
+.. _whatsnew36-pep487:
+
+PEP 487: Simpler customization of class creation
+------------------------------------------------
+
+It is now possible to customize subclass creation without using a metaclass.
+The new ``__init_subclass__`` classmethod will be called on the base class
+whenever a new subclass is created::
+
+    >>> class QuestBase:
+    ...    # this is implicitly a @classmethod
+    ...    def __init_subclass__(cls, swallow, **kwargs):
+    ...        cls.swallow = swallow
+    ...        super().__init_subclass__(**kwargs)
+
+    >>> class Quest(QuestBase, swallow="african"):
+    ...    pass
+
+    >>> Quest.swallow
+    'african'
+
+.. seealso::
+
+ :pep:`487` -- Simpler customization of class creation
+    PEP written and implemented by Martin Teichmann.
+
+ :ref:`Feature documentation <class-customization>`
+
+
+.. _whatsnew36-pep487-descriptors:
+
+PEP 487: Descriptor Protocol Enhancements
+-----------------------------------------
+
+:pep:`487` extends the descriptor protocol has to include the new optional
+``__set_name__`` method.  Whenever a new class is defined, the new method
+will be called on all descriptors included in the definition, providing
+them with a reference to the class being defined and the name given to the
+descriptor within the class namespace.
+
+.. seealso::
+
+    :pep:`487` -- Simpler customization of class creation
+        PEP written and implemented by Martin Teichmann.
+
+    :ref:`Feature documentation <descriptors>`
+
+
+.. _whatsnew36-pep506:
+
+PEP 506: Adding A Secrets Module To The Standard Library
+--------------------------------------------------------
+
+
+
+.. seealso::
+
+    :pep:`506` -- Adding A Secrets Module To The Standard Library
+      PEP written and implemented by Steven D'Aprano.
+
+
+
+.. _whatsnew36-pep519:
 
 PEP 519: Adding a file system path protocol
 -------------------------------------------
@@ -279,60 +439,7 @@
        PEP written by Brett Cannon and Koos Zevenhoven.
 
 
-.. _whatsnew-fstrings:
-
-PEP 498: Formatted string literals
-----------------------------------
-
-Formatted string literals are a new kind of string literal, prefixed
-with ``'f'``.  They are similar to the format strings accepted by
-:meth:`str.format`.  They contain replacement fields surrounded by
-curly braces.  The replacement fields are expressions, which are
-evaluated at run time, and then formatted using the :func:`format` protocol::
-
-    >>> name = "Fred"
-    >>> f"He said his name is {name}."
-    'He said his name is Fred.'
-
-See :pep:`498` and the main documentation at :ref:`f-strings`.
-
-
-.. _variable-annotations:
-
-PEP 526: Syntax for variable annotations
-----------------------------------------
-
-:pep:`484` introduced standard for type annotations of function parameters,
-a.k.a. type hints. This PEP adds syntax to Python for annotating the
-types of variables including class variables and instance variables::
-
-    primes: List[int] = []
-
-    captain: str  # Note: no initial value!
-
-    class Starship:
-        stats: Dict[str, int] = {}
-
-Just as for function annotations, the Python interpreter does not attach any
-particular meaning to variable annotations and only stores them in a special
-attribute ``__annotations__`` of a class or module.
-In contrast to variable declarations in statically typed languages,
-the goal of annotation syntax is to provide an easy way to specify structured
-type metadata for third party tools and libraries via the abstract syntax tree
-and the ``__annotations__`` attribute.
-
-.. seealso::
-
-   :pep:`526` -- Syntax for variable annotations.
-      PEP written by Ryan Gonzalez, Philip House, Ivan Levkivskyi, Lisa Roach,
-      and Guido van Rossum. Implemented by Ivan Levkivskyi.
-
-   Tools that use or will use the new syntax:
-   `mypy <http://github.com/python/mypy>`_,
-   `pytype <http://github.com/google/pytype>`_, PyCharm, etc.
-
-
-.. _pep-529:
+.. _whatsnew36-pep529:
 
 PEP 529: Change Windows filesystem encoding to UTF-8
 ----------------------------------------------------
@@ -355,28 +462,8 @@
 See :pep:`529` for more information and discussion of code modifications that
 may be required.
 
-.. _whatsnew-pep487:
 
-PEP 487: Simpler customization of class creation
-------------------------------------------------
-
-Upon subclassing a class, the ``__init_subclass__`` classmethod (if defined) is
-called on the base class. This makes it straightforward to write classes that
-customize initialization of future subclasses without introducing the
-complexity of a full custom metaclass.
-
-The descriptor protocol has also been expanded to include a new optional method,
-``__set_name__``. Whenever a new class is defined, the new method will be called
-on all descriptors included in the definition, providing them with a reference
-to the class being defined and the name given to the descriptor within the
-class namespace.
-
-Also see :pep:`487` and the updated class customization documentation at
-:ref:`class-customization` and :ref:`descriptors`.
-
-(Contributed by Martin Teichmann in :issue:`27366`)
-
-.. _pep-528:
+.. _whatsnew36-pep528:
 
 PEP 528: Change Windows console encoding to UTF-8
 -------------------------------------------------
@@ -394,6 +481,104 @@
    :pep:`528` -- Change Windows console encoding to UTF-8
       PEP written and implemented by Steve Dower.
 
+
+.. _whatsnew36-pep520:
+
+PEP 520: Preserving Class Attribute Definition Order
+----------------------------------------------------
+
+Attributes in a class definition body have a natural ordering: the same
+order in which the names appear in the source.  This order is now
+preserved in the new class's ``__dict__`` attribute.
+
+Also, the effective default class *execution* namespace (returned from
+``type.__prepare__()``) is now an insertion-order-preserving mapping.
+
+.. seealso::
+
+   :pep:`520` -- Preserving Class Attribute Definition Order
+      PEP written and implemented by Eric Snow.
+
+
+.. _whatsnew36-pep468:
+
+PEP 468: Preserving Keyword Argument Order
+------------------------------------------
+
+``**kwargs`` in a function signature is now guaranteed to be an
+insertion-order-preserving mapping.
+
+.. seealso::
+
+   :pep:`468` -- Preserving Keyword Argument Order
+      PEP written and implemented by Eric Snow.
+
+
+.. _whatsnew36-compactdict:
+
+New :ref:`dict <typesmapping>` implementation
+---------------------------------------------
+
+The :ref:`dict <typesmapping>` type now uses a "compact" representation
+`pioneered by PyPy <https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html>`_.
+The memory usage of the new :func:`dict` is between 20% and 25% smaller
+compared to Python 3.5.
+:pep:`468` (Preserving the order of ``**kwargs`` in a function.) is
+implemented by this. The order-preserving aspect of this new
+implementation is considered an implementation detail and should
+not be relied upon (this may change in the future, but it is desired
+to have this new dict implementation in the language for a few
+releases before changing the language spec to mandate
+order-preserving semantics for all current and future Python
+implementations; this also helps preserve backwards-compatibility
+with older versions of the language where random iteration order is
+still in effect, e.g. Python 3.5).
+(Contributed by INADA Naoki in :issue:`27350`. Idea
+`originally suggested by Raymond Hettinger
+<https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_.)
+
+
+.. _whatsnew36-pep523:
+
+PEP 523: Adding a frame evaluation API to CPython
+-------------------------------------------------
+
+While Python provides extensive support to customize how code
+executes, one place it has not done so is in the evaluation of frame
+objects. If you wanted some way to intercept frame evaluation in
+Python there really wasn't any way without directly manipulating
+function pointers for defined functions.
+
+:pep:`523` changes this by providing an API to make frame
+evaluation pluggable at the C level. This will allow for tools such
+as debuggers and JITs to intercept frame evaluation before the
+execution of Python code begins. This enables the use of alternative
+evaluation implementations for Python code, tracking frame
+evaluation, etc.
+
+This API is not part of the limited C API and is marked as private to
+signal that usage of this API is expected to be limited and only
+applicable to very select, low-level use-cases. Semantics of the
+API will change with Python as necessary.
+
+.. seealso::
+
+  :pep:`523` -- Adding a frame evaluation API to CPython
+     PEP written by Brett Cannon and Dino Viehland.
+
+
+.. _whatsnew36-pep509:
+
+PEP 509: Add a private version to dict
+--------------------------------------
+
+Add a new private version to the builtin ``dict`` type, incremented at
+each dictionary creation and at each dictionary change, to implement
+fast guards on namespaces.
+
+(Contributed by Victor Stinner in :issue:`26058`.)
+
+
 PYTHONMALLOC environment variable
 ---------------------------------
 
@@ -468,6 +653,8 @@
 (Contributed by Victor Stinner in :issue:`26516` and :issue:`26564`.)
 
 
+.. _whatsnew36-tracing:
+
 DTrace and SystemTap probing support
 ------------------------------------
 
@@ -493,71 +680,14 @@
 Jes?s Cea Avi?n, David Malcolm, and Nikhil Benesch.)
 
 
-.. _whatsnew-deforder:
-
-PEP 520: Preserving Class Attribute Definition Order
-----------------------------------------------------
-
-Attributes in a class definition body have a natural ordering: the same
-order in which the names appear in the source.  This order is now
-preserved in the new class's ``__dict__`` attribute.
-
-Also, the effective default class *execution* namespace (returned from
-``type.__prepare__()``) is now an insertion-order-preserving mapping.
-
-.. seealso::
-
-   :pep:`520` -- Preserving Class Attribute Definition Order
-      PEP written and implemented by Eric Snow.
-
-
-.. _whatsnew-kwargs:
-
-PEP 468: Preserving Keyword Argument Order
-------------------------------------------
-
-``**kwargs`` in a function signature is now guaranteed to be an
-insertion-order-preserving mapping.
-
-.. seealso::
-
-   :pep:`468` -- Preserving Keyword Argument Order
-      PEP written and implemented by Eric Snow.
-
-.. _whatsnew-pep509:
-
-PEP 509: Add a private version to dict
---------------------------------------
-
-Add a new private version to the builtin ``dict`` type, incremented at
-each dictionary creation and at each dictionary change, to implement
-fast guards on namespaces.
-
-(Contributed by Victor Stinner in :issue:`26058`.)
-
-
 Other Language Changes
 ======================
 
 Some smaller changes made to the core Python language are:
 
-* :func:`dict` now uses a "compact" representation `pioneered by PyPy
-  <https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html>`_.
-  The memory usage of the new :func:`dict` is between 20% and 25% smaller
-  compared to Python 3.5.
-  :pep:`468` (Preserving the order of ``**kwargs`` in a function.) is
-  implemented by this. The order-preserving aspect of this new
-  implementation is considered an implementation detail and should
-  not be relied upon (this may change in the future, but it is desired
-  to have this new dict implementation in the language for a few
-  releases before changing the language spec to mandate
-  order-preserving semantics for all current and future Python
-  implementations; this also helps preserve backwards-compatibility
-  with older versions of the language where random iteration order is
-  still in effect, e.g. Python 3.5).
-  (Contributed by INADA Naoki in :issue:`27350`. Idea
-  `originally suggested by Raymond Hettinger
-  <https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_.)
+* A ``global`` or ``nonlocal`` statement must now textually appear
+  before the first use of the affected name in the same scope.
+  Previously this was a ``SyntaxWarning``.
 
 * Long sequences of repeated traceback lines are now abbreviated as
   ``"[Previous line repeated {count} more times]"`` (see
@@ -572,16 +702,15 @@
 New Modules
 ===========
 
-* None yet.
+secrets
+-------
+
+The new :mod:`secrets` module.
 
 
 Improved Modules
 ================
 
-On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool
-is initialized to increase the security. See the :pep:`524` for the rationale.
-
-
 asyncio
 -------
 
@@ -783,11 +912,14 @@
 will be emitted in its destructor.
 (Contributed by Serhiy Storchaka in :issue:`25994`.)
 
+On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool
+is initialized to increase the security. See the :pep:`524` for the rationale.
+
 The Linux ``getrandom()`` syscall (get random bytes) is now exposed as the new
 :func:`os.getrandom` function.
 (Contributed by Victor Stinner, part of the :pep:`524`)
 
-See the summary for :ref:`PEP 519 <pep-519>` for details on how the
+See the summary for :ref:`PEP 519 <whatsnew36-pep519>` for details on how the
 :mod:`os` and :mod:`os.path` modules now support
 :term:`path-like objects <path-like object>`.
 
@@ -1159,6 +1291,7 @@
   it is now about 1.5--4 times faster.
   (Contributed by Serhiy Storchaka in :issue:`26032`).
 
+
 Build and C API Changes
 =======================
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 17:15:31 2016
From: python-checkins at python.org (terry.reedy)
Date: Mon, 07 Nov 2016 22:15:31 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NTcy?=
 =?utf-8?q?=3A_Add_10=25_to_coverage_of_IDLE=27s_test=5Fconfigdialog=2E?=
Message-ID: <20161107221531.111677.51353.7E1DA114@psf.io>

https://hg.python.org/cpython/rev/d6440718eb30
changeset:   104957:d6440718eb30
branch:      3.6
parent:      104955:5cf13fd36af1
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Mon Nov 07 17:15:01 2016 -0500
summary:
  Issue #28572: Add 10% to coverage of IDLE's test_configdialog.
Update and augment description of the configuration system.

files:
  Lib/idlelib/config-main.def                |   68 ++--
  Lib/idlelib/config.py                      |   23 +-
  Lib/idlelib/configdialog.py                |   32 +-
  Lib/idlelib/idle_test/test_configdialog.py |  129 ++++++++-
  4 files changed, 179 insertions(+), 73 deletions(-)


diff --git a/Lib/idlelib/config-main.def b/Lib/idlelib/config-main.def
--- a/Lib/idlelib/config-main.def
+++ b/Lib/idlelib/config-main.def
@@ -4,44 +4,50 @@
 # When IDLE starts, it will look in
 # the following two sets of files, in order:
 #
-#     default configuration
-#     ---------------------
-#     config-main.def         the default general config file
-#     config-extensions.def   the default extension config file
-#     config-highlight.def    the default highlighting config file
-#     config-keys.def         the default keybinding config file
+#     default configuration files in idlelib
+#     --------------------------------------
+#     config-main.def         default general config file
+#     config-extensions.def   default extension config file
+#     config-highlight.def    default highlighting config file
+#     config-keys.def         default keybinding config file
 #
-#     user configuration
-#     -------------------
-#     ~/.idlerc/config-main.cfg            the user general config file
-#     ~/.idlerc/config-extensions.cfg      the user extension config file
-#     ~/.idlerc/config-highlight.cfg       the user highlighting config file
-#     ~/.idlerc/config-keys.cfg            the user keybinding config file
+#     user configuration files in ~/.idlerc
+#     -------------------------------------
+#     config-main.cfg         user general config file
+#     config-extensions.cfg   user extension config file
+#     config-highlight.cfg    user highlighting config file
+#     config-keys.cfg         user keybinding config file
 #
-# On Windows2000 and Windows XP the .idlerc directory is at
-#     Documents and Settings\<username>\.idlerc
-#
-# On Windows98 it is at c:\.idlerc
+# On Windows, the default location of the home directory ('~' above)
+# depends on the version.  For Windows 10, it is C:\Users\<username>.
 #
 # Any options the user saves through the config dialog will be saved to
-# the relevant user config file. Reverting any general setting to the
-# default causes that entry to be wiped from the user file and re-read
-# from the default file. User highlighting themes or keybinding sets are
-# retained unless specifically deleted within the config dialog. Choosing
-# one of the default themes or keysets just applies the relevant settings
-# from the default file.
+# the relevant user config file. Reverting any general or extension
+# setting to the default causes that entry to be wiped from the user
+# file and re-read from the default file.  This rule applies to each
+# item, except that the three editor font items are saved as a group.
 #
-# Additional help sources are listed in the [HelpFiles] section and must be
-# viewable by a web browser (or the Windows Help viewer in the case of .chm
-# files). These sources will be listed on the Help menu.  The pattern is
+# User highlighting themes and keybinding sets must have (section) names
+# distinct from the default names.  All items are added and saved as a
+# group. They are retained unless specifically deleted within the config
+# dialog. Choosing one of the default themes or keysets just applies the
+# relevant settings from the default file.
+#
+# Additional help sources are listed in the [HelpFiles] section below
+# and should be viewable by a web browser (or the Windows Help viewer in
+# the case of .chm files). These sources will be listed on the Help
+# menu.  The pattern, and two examples, are
+#
 # <sequence_number = menu item;/path/to/help/source>
-# You can't use a semi-colon in a menu item or path.  The path will be platform
-# specific because of path separators, drive specs etc.
+# 1 = IDLE;C:/Programs/Python36/Lib/idlelib/help.html
+# 2 = Pillow;https://pillow.readthedocs.io/en/latest/
 #
-# It is best to use the Configuration GUI to set up additional help sources!
-# Example:
-#1 = My Extra Help Source;/usr/share/doc/foo/index.html
-#2 = Another Help Source;/path/to/another.pdf
+# You can't use a semi-colon in a menu item or path.  The path will be
+# platform specific because of path separators, drive specs etc.
+#
+# The default files should not be edited except to add new sections to
+# config-extensions.def for added extensions .  The user files should be
+# modified through the Settings dialog.
 
 [General]
 editor-on-startup= 0
diff --git a/Lib/idlelib/config.py b/Lib/idlelib/config.py
--- a/Lib/idlelib/config.py
+++ b/Lib/idlelib/config.py
@@ -1,13 +1,20 @@
-"""Provides access to stored IDLE configuration information.
+"""idlelib.config -- Manage IDLE configuration information.
 
-Refer to the comments at the beginning of config-main.def for a description of
-the available configuration files and the design implemented to update user
-configuration information.  In particular, user configuration choices which
-duplicate the defaults will be removed from the user's configuration files,
-and if a file becomes empty, it will be deleted.
+The comments at the beginning of config-main.def describe the
+configuration files and the design implemented to update user
+configuration information.  In particular, user configuration choices
+which duplicate the defaults will be removed from the user's
+configuration files, and if a user file becomes empty, it will be
+deleted.
 
-The contents of the user files may be altered using the Options/Configure IDLE
-menu to access the configuration GUI (configdialog.py), or manually.
+The configuration database maps options to values.  Comceptually, the
+database keys are tuples (config-type, section, item).  As implemented,
+there are  separate dicts for default and user values.  Each has
+config-type keys 'main', 'extensions', 'highlight', and 'keys'.  The
+value for each key is a ConfigParser instance that maps section and item
+to values.  For 'main' and 'extenstons', user values override
+default values.  For 'highlight' and 'keys', user sections augment the
+default sections (and must, therefore, have distinct names).
 
 Throughout this module there is an emphasis on returning useable defaults
 when a problem occurs in returning a requested configuration value back to
diff --git a/Lib/idlelib/configdialog.py b/Lib/idlelib/configdialog.py
--- a/Lib/idlelib/configdialog.py
+++ b/Lib/idlelib/configdialog.py
@@ -392,28 +392,28 @@
                                text=' Additional Help Sources ')
         #frameRun
         labelRunChoiceTitle = Label(frameRun, text='At Startup')
-        radioStartupEdit = Radiobutton(
+        self.radioStartupEdit = Radiobutton(
                 frameRun, variable=self.startupEdit, value=1,
-                command=self.SetKeysType, text="Open Edit Window")
-        radioStartupShell = Radiobutton(
+                text="Open Edit Window")
+        self.radioStartupShell = Radiobutton(
                 frameRun, variable=self.startupEdit, value=0,
-                command=self.SetKeysType, text='Open Shell Window')
+                text='Open Shell Window')
         #frameSave
         labelRunSaveTitle = Label(frameSave, text='At Start of Run (F5)  ')
-        radioSaveAsk = Radiobutton(
+        self.radioSaveAsk = Radiobutton(
                 frameSave, variable=self.autoSave, value=0,
-                command=self.SetKeysType, text="Prompt to Save")
-        radioSaveAuto = Radiobutton(
+                text="Prompt to Save")
+        self.radioSaveAuto = Radiobutton(
                 frameSave, variable=self.autoSave, value=1,
-                command=self.SetKeysType, text='No Prompt')
+                text='No Prompt')
         #frameWinSize
         labelWinSizeTitle = Label(
                 frameWinSize, text='Initial Window Size  (in characters)')
         labelWinWidthTitle = Label(frameWinSize, text='Width')
-        entryWinWidth = Entry(
+        self.entryWinWidth = Entry(
                 frameWinSize, textvariable=self.winWidth, width=3)
         labelWinHeightTitle = Label(frameWinSize, text='Height')
-        entryWinHeight = Entry(
+        self.entryWinHeight = Entry(
                 frameWinSize, textvariable=self.winHeight, width=3)
         #frameHelp
         frameHelpList = Frame(frameHelp)
@@ -443,17 +443,17 @@
         frameHelp.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH)
         #frameRun
         labelRunChoiceTitle.pack(side=LEFT, anchor=W, padx=5, pady=5)
-        radioStartupShell.pack(side=RIGHT, anchor=W, padx=5, pady=5)
-        radioStartupEdit.pack(side=RIGHT, anchor=W, padx=5, pady=5)
+        self.radioStartupShell.pack(side=RIGHT, anchor=W, padx=5, pady=5)
+        self.radioStartupEdit.pack(side=RIGHT, anchor=W, padx=5, pady=5)
         #frameSave
         labelRunSaveTitle.pack(side=LEFT, anchor=W, padx=5, pady=5)
-        radioSaveAuto.pack(side=RIGHT, anchor=W, padx=5, pady=5)
-        radioSaveAsk.pack(side=RIGHT, anchor=W, padx=5, pady=5)
+        self.radioSaveAuto.pack(side=RIGHT, anchor=W, padx=5, pady=5)
+        self.radioSaveAsk.pack(side=RIGHT, anchor=W, padx=5, pady=5)
         #frameWinSize
         labelWinSizeTitle.pack(side=LEFT, anchor=W, padx=5, pady=5)
-        entryWinHeight.pack(side=RIGHT, anchor=E, padx=10, pady=5)
+        self.entryWinHeight.pack(side=RIGHT, anchor=E, padx=10, pady=5)
         labelWinHeightTitle.pack(side=RIGHT, anchor=E, pady=5)
-        entryWinWidth.pack(side=RIGHT, anchor=E, padx=10, pady=5)
+        self.entryWinWidth.pack(side=RIGHT, anchor=E, padx=10, pady=5)
         labelWinWidthTitle.pack(side=RIGHT, anchor=E, pady=5)
         #frameHelp
         frameHelpListButtons.pack(side=RIGHT, padx=5, pady=5, fill=Y)
diff --git a/Lib/idlelib/idle_test/test_configdialog.py b/Lib/idlelib/idle_test/test_configdialog.py
--- a/Lib/idlelib/idle_test/test_configdialog.py
+++ b/Lib/idlelib/idle_test/test_configdialog.py
@@ -1,30 +1,123 @@
-'''Test idlelib.configdialog.
+"""Test idlelib.configdialog.
 
-Coverage: 46% just by creating dialog.
-The other half is code for working with user customizations.
-'''
-from idlelib.configdialog import ConfigDialog  # always test import
+Half the class creates dialog, half works with user customizations.
+Coverage: 46% just by creating dialog, 56% with current tests.
+"""
+from idlelib.configdialog import ConfigDialog, idleConf  # test import
 from test.support import requires
 requires('gui')
 from tkinter import Tk
 import unittest
+import idlelib.config as config
 
-class ConfigDialogTest(unittest.TestCase):
+# Tests should not depend on fortuitous user configurations.
+# They must not affect actual user .cfg files.
+# Use solution from test_config: empty parsers with no filename.
+usercfg = idleConf.userCfg
+testcfg = {
+    'main': config.IdleUserConfParser(''),
+    'highlight': config.IdleUserConfParser(''),
+    'keys': config.IdleUserConfParser(''),
+    'extensions': config.IdleUserConfParser(''),
+}
 
-    @classmethod
-    def setUpClass(cls):
-        cls.root = Tk()
-        cls.root.withdraw()
+# ConfigDialog.changedItems is a 3-level hierarchical dictionary of
+# pending changes that mirrors the multilevel user config dict.
+# For testing, record args in a list for comparison with expected.
+changes = []
+class TestDialog(ConfigDialog):
+    def AddChangedItem(self, *args):
+        changes.append(args)
 
-    @classmethod
-    def tearDownClass(cls):
-        cls.root.update_idletasks()
-        cls.root.destroy()
-        del cls.root
+def setUpModule():
+    global root, configure
+    idleConf.userCfg = testcfg
+    root = Tk()
+    root.withdraw()
+    configure = TestDialog(root, 'Test', _utest=True)
 
-    def test_configdialog(self):
-        d = ConfigDialog(self.root, 'Test', _utest=True)
-        d.remove_var_callbacks()
+
+def tearDownModule():
+    global root, configure
+    idleConf.userCfg = testcfg
+    configure.remove_var_callbacks()
+    del configure
+    root.update_idletasks()
+    root.destroy()
+    del root
+
+
+class FontTabTest(unittest.TestCase):
+
+
+    def setUp(self):
+        changes.clear()
+
+    def test_font(self):
+        configure.fontName.set('Test Font')
+        expected = [
+            ('main', 'EditorWindow', 'font', 'Test Font'),
+            ('main', 'EditorWindow', 'font-size', '10'),
+            ('main', 'EditorWindow', 'font-bold', False)]
+        self.assertEqual(changes, expected)
+        changes.clear()
+        configure.fontSize.set(12)
+        expected = [
+            ('main', 'EditorWindow', 'font', 'Test Font'),
+            ('main', 'EditorWindow', 'font-size', '12'),
+            ('main', 'EditorWindow', 'font-bold', False)]
+        self.assertEqual(changes, expected)
+        changes.clear()
+        configure.fontBold.set(True)
+        expected = [
+            ('main', 'EditorWindow', 'font', 'Test Font'),
+            ('main', 'EditorWindow', 'font-size', '12'),
+            ('main', 'EditorWindow', 'font-bold', True)]
+        self.assertEqual(changes, expected)
+
+    #def test_sample(self): pass  # TODO
+
+    def test_tabspace(self):
+        configure.spaceNum.set(6)
+        self.assertEqual(changes, [('main', 'Indent', 'num-spaces', 6)])
+
+
+class HighlightTest(unittest.TestCase):
+
+    def setUp(self):
+        changes.clear()
+
+    #def test_colorchoose(self): pass  # TODO
+
+
+class KeysTest(unittest.TestCase):
+
+    def setUp(self):
+        changes.clear()
+
+
+class GeneralTest(unittest.TestCase):
+
+    def setUp(self):
+        changes.clear()
+
+    def test_startup(self):
+        configure.radioStartupEdit.invoke()
+        self.assertEqual(changes,
+                         [('main', 'General', 'editor-on-startup', 1)])
+
+    def test_autosave(self):
+        configure.radioSaveAuto.invoke()
+        self.assertEqual(changes, [('main', 'General', 'autosave', 1)])
+
+    def test_editor_size(self):
+        configure.entryWinHeight.insert(0, '1')
+        self.assertEqual(changes, [('main', 'EditorWindow', 'height', '140')])
+        changes.clear()
+        configure.entryWinWidth.insert(0, '1')
+        self.assertEqual(changes, [('main', 'EditorWindow', 'width', '180')])
+
+    #def test_help_sources(self): pass  # TODO
 
 
 if __name__ == '__main__':

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 17:15:31 2016
From: python-checkins at python.org (terry.reedy)
Date: Mon, 07 Nov 2016 22:15:31 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_with_3=2E6?=
Message-ID: <20161107221531.30750.82187.848FC1B1@psf.io>

https://hg.python.org/cpython/rev/d0e4440a68b3
changeset:   104958:d0e4440a68b3
parent:      104956:38c806f0943b
parent:      104957:d6440718eb30
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Mon Nov 07 17:15:16 2016 -0500
summary:
  Merge with 3.6

files:
  Lib/idlelib/config-main.def                |   68 ++--
  Lib/idlelib/config.py                      |   23 +-
  Lib/idlelib/configdialog.py                |   32 +-
  Lib/idlelib/idle_test/test_configdialog.py |  129 ++++++++-
  4 files changed, 179 insertions(+), 73 deletions(-)


diff --git a/Lib/idlelib/config-main.def b/Lib/idlelib/config-main.def
--- a/Lib/idlelib/config-main.def
+++ b/Lib/idlelib/config-main.def
@@ -4,44 +4,50 @@
 # When IDLE starts, it will look in
 # the following two sets of files, in order:
 #
-#     default configuration
-#     ---------------------
-#     config-main.def         the default general config file
-#     config-extensions.def   the default extension config file
-#     config-highlight.def    the default highlighting config file
-#     config-keys.def         the default keybinding config file
+#     default configuration files in idlelib
+#     --------------------------------------
+#     config-main.def         default general config file
+#     config-extensions.def   default extension config file
+#     config-highlight.def    default highlighting config file
+#     config-keys.def         default keybinding config file
 #
-#     user configuration
-#     -------------------
-#     ~/.idlerc/config-main.cfg            the user general config file
-#     ~/.idlerc/config-extensions.cfg      the user extension config file
-#     ~/.idlerc/config-highlight.cfg       the user highlighting config file
-#     ~/.idlerc/config-keys.cfg            the user keybinding config file
+#     user configuration files in ~/.idlerc
+#     -------------------------------------
+#     config-main.cfg         user general config file
+#     config-extensions.cfg   user extension config file
+#     config-highlight.cfg    user highlighting config file
+#     config-keys.cfg         user keybinding config file
 #
-# On Windows2000 and Windows XP the .idlerc directory is at
-#     Documents and Settings\<username>\.idlerc
-#
-# On Windows98 it is at c:\.idlerc
+# On Windows, the default location of the home directory ('~' above)
+# depends on the version.  For Windows 10, it is C:\Users\<username>.
 #
 # Any options the user saves through the config dialog will be saved to
-# the relevant user config file. Reverting any general setting to the
-# default causes that entry to be wiped from the user file and re-read
-# from the default file. User highlighting themes or keybinding sets are
-# retained unless specifically deleted within the config dialog. Choosing
-# one of the default themes or keysets just applies the relevant settings
-# from the default file.
+# the relevant user config file. Reverting any general or extension
+# setting to the default causes that entry to be wiped from the user
+# file and re-read from the default file.  This rule applies to each
+# item, except that the three editor font items are saved as a group.
 #
-# Additional help sources are listed in the [HelpFiles] section and must be
-# viewable by a web browser (or the Windows Help viewer in the case of .chm
-# files). These sources will be listed on the Help menu.  The pattern is
+# User highlighting themes and keybinding sets must have (section) names
+# distinct from the default names.  All items are added and saved as a
+# group. They are retained unless specifically deleted within the config
+# dialog. Choosing one of the default themes or keysets just applies the
+# relevant settings from the default file.
+#
+# Additional help sources are listed in the [HelpFiles] section below
+# and should be viewable by a web browser (or the Windows Help viewer in
+# the case of .chm files). These sources will be listed on the Help
+# menu.  The pattern, and two examples, are
+#
 # <sequence_number = menu item;/path/to/help/source>
-# You can't use a semi-colon in a menu item or path.  The path will be platform
-# specific because of path separators, drive specs etc.
+# 1 = IDLE;C:/Programs/Python36/Lib/idlelib/help.html
+# 2 = Pillow;https://pillow.readthedocs.io/en/latest/
 #
-# It is best to use the Configuration GUI to set up additional help sources!
-# Example:
-#1 = My Extra Help Source;/usr/share/doc/foo/index.html
-#2 = Another Help Source;/path/to/another.pdf
+# You can't use a semi-colon in a menu item or path.  The path will be
+# platform specific because of path separators, drive specs etc.
+#
+# The default files should not be edited except to add new sections to
+# config-extensions.def for added extensions .  The user files should be
+# modified through the Settings dialog.
 
 [General]
 editor-on-startup= 0
diff --git a/Lib/idlelib/config.py b/Lib/idlelib/config.py
--- a/Lib/idlelib/config.py
+++ b/Lib/idlelib/config.py
@@ -1,13 +1,20 @@
-"""Provides access to stored IDLE configuration information.
+"""idlelib.config -- Manage IDLE configuration information.
 
-Refer to the comments at the beginning of config-main.def for a description of
-the available configuration files and the design implemented to update user
-configuration information.  In particular, user configuration choices which
-duplicate the defaults will be removed from the user's configuration files,
-and if a file becomes empty, it will be deleted.
+The comments at the beginning of config-main.def describe the
+configuration files and the design implemented to update user
+configuration information.  In particular, user configuration choices
+which duplicate the defaults will be removed from the user's
+configuration files, and if a user file becomes empty, it will be
+deleted.
 
-The contents of the user files may be altered using the Options/Configure IDLE
-menu to access the configuration GUI (configdialog.py), or manually.
+The configuration database maps options to values.  Comceptually, the
+database keys are tuples (config-type, section, item).  As implemented,
+there are  separate dicts for default and user values.  Each has
+config-type keys 'main', 'extensions', 'highlight', and 'keys'.  The
+value for each key is a ConfigParser instance that maps section and item
+to values.  For 'main' and 'extenstons', user values override
+default values.  For 'highlight' and 'keys', user sections augment the
+default sections (and must, therefore, have distinct names).
 
 Throughout this module there is an emphasis on returning useable defaults
 when a problem occurs in returning a requested configuration value back to
diff --git a/Lib/idlelib/configdialog.py b/Lib/idlelib/configdialog.py
--- a/Lib/idlelib/configdialog.py
+++ b/Lib/idlelib/configdialog.py
@@ -392,28 +392,28 @@
                                text=' Additional Help Sources ')
         #frameRun
         labelRunChoiceTitle = Label(frameRun, text='At Startup')
-        radioStartupEdit = Radiobutton(
+        self.radioStartupEdit = Radiobutton(
                 frameRun, variable=self.startupEdit, value=1,
-                command=self.SetKeysType, text="Open Edit Window")
-        radioStartupShell = Radiobutton(
+                text="Open Edit Window")
+        self.radioStartupShell = Radiobutton(
                 frameRun, variable=self.startupEdit, value=0,
-                command=self.SetKeysType, text='Open Shell Window')
+                text='Open Shell Window')
         #frameSave
         labelRunSaveTitle = Label(frameSave, text='At Start of Run (F5)  ')
-        radioSaveAsk = Radiobutton(
+        self.radioSaveAsk = Radiobutton(
                 frameSave, variable=self.autoSave, value=0,
-                command=self.SetKeysType, text="Prompt to Save")
-        radioSaveAuto = Radiobutton(
+                text="Prompt to Save")
+        self.radioSaveAuto = Radiobutton(
                 frameSave, variable=self.autoSave, value=1,
-                command=self.SetKeysType, text='No Prompt')
+                text='No Prompt')
         #frameWinSize
         labelWinSizeTitle = Label(
                 frameWinSize, text='Initial Window Size  (in characters)')
         labelWinWidthTitle = Label(frameWinSize, text='Width')
-        entryWinWidth = Entry(
+        self.entryWinWidth = Entry(
                 frameWinSize, textvariable=self.winWidth, width=3)
         labelWinHeightTitle = Label(frameWinSize, text='Height')
-        entryWinHeight = Entry(
+        self.entryWinHeight = Entry(
                 frameWinSize, textvariable=self.winHeight, width=3)
         #frameHelp
         frameHelpList = Frame(frameHelp)
@@ -443,17 +443,17 @@
         frameHelp.pack(side=TOP, padx=5, pady=5, expand=TRUE, fill=BOTH)
         #frameRun
         labelRunChoiceTitle.pack(side=LEFT, anchor=W, padx=5, pady=5)
-        radioStartupShell.pack(side=RIGHT, anchor=W, padx=5, pady=5)
-        radioStartupEdit.pack(side=RIGHT, anchor=W, padx=5, pady=5)
+        self.radioStartupShell.pack(side=RIGHT, anchor=W, padx=5, pady=5)
+        self.radioStartupEdit.pack(side=RIGHT, anchor=W, padx=5, pady=5)
         #frameSave
         labelRunSaveTitle.pack(side=LEFT, anchor=W, padx=5, pady=5)
-        radioSaveAuto.pack(side=RIGHT, anchor=W, padx=5, pady=5)
-        radioSaveAsk.pack(side=RIGHT, anchor=W, padx=5, pady=5)
+        self.radioSaveAuto.pack(side=RIGHT, anchor=W, padx=5, pady=5)
+        self.radioSaveAsk.pack(side=RIGHT, anchor=W, padx=5, pady=5)
         #frameWinSize
         labelWinSizeTitle.pack(side=LEFT, anchor=W, padx=5, pady=5)
-        entryWinHeight.pack(side=RIGHT, anchor=E, padx=10, pady=5)
+        self.entryWinHeight.pack(side=RIGHT, anchor=E, padx=10, pady=5)
         labelWinHeightTitle.pack(side=RIGHT, anchor=E, pady=5)
-        entryWinWidth.pack(side=RIGHT, anchor=E, padx=10, pady=5)
+        self.entryWinWidth.pack(side=RIGHT, anchor=E, padx=10, pady=5)
         labelWinWidthTitle.pack(side=RIGHT, anchor=E, pady=5)
         #frameHelp
         frameHelpListButtons.pack(side=RIGHT, padx=5, pady=5, fill=Y)
diff --git a/Lib/idlelib/idle_test/test_configdialog.py b/Lib/idlelib/idle_test/test_configdialog.py
--- a/Lib/idlelib/idle_test/test_configdialog.py
+++ b/Lib/idlelib/idle_test/test_configdialog.py
@@ -1,30 +1,123 @@
-'''Test idlelib.configdialog.
+"""Test idlelib.configdialog.
 
-Coverage: 46% just by creating dialog.
-The other half is code for working with user customizations.
-'''
-from idlelib.configdialog import ConfigDialog  # always test import
+Half the class creates dialog, half works with user customizations.
+Coverage: 46% just by creating dialog, 56% with current tests.
+"""
+from idlelib.configdialog import ConfigDialog, idleConf  # test import
 from test.support import requires
 requires('gui')
 from tkinter import Tk
 import unittest
+import idlelib.config as config
 
-class ConfigDialogTest(unittest.TestCase):
+# Tests should not depend on fortuitous user configurations.
+# They must not affect actual user .cfg files.
+# Use solution from test_config: empty parsers with no filename.
+usercfg = idleConf.userCfg
+testcfg = {
+    'main': config.IdleUserConfParser(''),
+    'highlight': config.IdleUserConfParser(''),
+    'keys': config.IdleUserConfParser(''),
+    'extensions': config.IdleUserConfParser(''),
+}
 
-    @classmethod
-    def setUpClass(cls):
-        cls.root = Tk()
-        cls.root.withdraw()
+# ConfigDialog.changedItems is a 3-level hierarchical dictionary of
+# pending changes that mirrors the multilevel user config dict.
+# For testing, record args in a list for comparison with expected.
+changes = []
+class TestDialog(ConfigDialog):
+    def AddChangedItem(self, *args):
+        changes.append(args)
 
-    @classmethod
-    def tearDownClass(cls):
-        cls.root.update_idletasks()
-        cls.root.destroy()
-        del cls.root
+def setUpModule():
+    global root, configure
+    idleConf.userCfg = testcfg
+    root = Tk()
+    root.withdraw()
+    configure = TestDialog(root, 'Test', _utest=True)
 
-    def test_configdialog(self):
-        d = ConfigDialog(self.root, 'Test', _utest=True)
-        d.remove_var_callbacks()
+
+def tearDownModule():
+    global root, configure
+    idleConf.userCfg = testcfg
+    configure.remove_var_callbacks()
+    del configure
+    root.update_idletasks()
+    root.destroy()
+    del root
+
+
+class FontTabTest(unittest.TestCase):
+
+
+    def setUp(self):
+        changes.clear()
+
+    def test_font(self):
+        configure.fontName.set('Test Font')
+        expected = [
+            ('main', 'EditorWindow', 'font', 'Test Font'),
+            ('main', 'EditorWindow', 'font-size', '10'),
+            ('main', 'EditorWindow', 'font-bold', False)]
+        self.assertEqual(changes, expected)
+        changes.clear()
+        configure.fontSize.set(12)
+        expected = [
+            ('main', 'EditorWindow', 'font', 'Test Font'),
+            ('main', 'EditorWindow', 'font-size', '12'),
+            ('main', 'EditorWindow', 'font-bold', False)]
+        self.assertEqual(changes, expected)
+        changes.clear()
+        configure.fontBold.set(True)
+        expected = [
+            ('main', 'EditorWindow', 'font', 'Test Font'),
+            ('main', 'EditorWindow', 'font-size', '12'),
+            ('main', 'EditorWindow', 'font-bold', True)]
+        self.assertEqual(changes, expected)
+
+    #def test_sample(self): pass  # TODO
+
+    def test_tabspace(self):
+        configure.spaceNum.set(6)
+        self.assertEqual(changes, [('main', 'Indent', 'num-spaces', 6)])
+
+
+class HighlightTest(unittest.TestCase):
+
+    def setUp(self):
+        changes.clear()
+
+    #def test_colorchoose(self): pass  # TODO
+
+
+class KeysTest(unittest.TestCase):
+
+    def setUp(self):
+        changes.clear()
+
+
+class GeneralTest(unittest.TestCase):
+
+    def setUp(self):
+        changes.clear()
+
+    def test_startup(self):
+        configure.radioStartupEdit.invoke()
+        self.assertEqual(changes,
+                         [('main', 'General', 'editor-on-startup', 1)])
+
+    def test_autosave(self):
+        configure.radioSaveAuto.invoke()
+        self.assertEqual(changes, [('main', 'General', 'autosave', 1)])
+
+    def test_editor_size(self):
+        configure.entryWinHeight.insert(0, '1')
+        self.assertEqual(changes, [('main', 'EditorWindow', 'height', '140')])
+        changes.clear()
+        configure.entryWinWidth.insert(0, '1')
+        self.assertEqual(changes, [('main', 'EditorWindow', 'width', '180')])
+
+    #def test_help_sources(self): pass  # TODO
 
 
 if __name__ == '__main__':

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 17:58:09 2016
From: python-checkins at python.org (eric.smith)
Date: Mon, 07 Nov 2016 22:58:09 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Fixed_issue_?=
 =?utf-8?q?=2328633=3A_segfault_when_concatenating_bytes_literal_and_f-str?=
 =?utf-8?q?ing=2E?=
Message-ID: <20161107225808.111534.40998.C4F3C2CD@psf.io>

https://hg.python.org/cpython/rev/31543f7cbdf4
changeset:   104959:31543f7cbdf4
branch:      3.6
parent:      104957:d6440718eb30
user:        Eric V. Smith <eric at trueblade.com>
date:        Mon Nov 07 17:54:01 2016 -0500
summary:
  Fixed issue #28633: segfault when concatenating bytes literal and f-string.

files:
  Lib/test/test_fstring.py |  7 +++++++
  Python/ast.c             |  9 +++++----
  2 files changed, 12 insertions(+), 4 deletions(-)


diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py
--- a/Lib/test/test_fstring.py
+++ b/Lib/test/test_fstring.py
@@ -92,6 +92,13 @@
         exec(c)
         self.assertEqual(x[0], 'foo3')
 
+    def test_compile_time_concat_errors(self):
+        self.assertAllRaise(SyntaxError,
+                            'cannot mix bytes and nonbytes literals',
+                            [r"""f'' b''""",
+                             r"""b'' f''""",
+                             ])
+
     def test_literal(self):
         self.assertEqual(f'', '')
         self.assertEqual(f'a', 'a')
diff --git a/Python/ast.c b/Python/ast.c
--- a/Python/ast.c
+++ b/Python/ast.c
@@ -5147,7 +5147,8 @@
         /* Check that we're not mixing bytes with unicode. */
         if (i != 0 && bytesmode != this_bytesmode) {
             ast_error(c, n, "cannot mix bytes and nonbytes literals");
-            Py_DECREF(s);
+            /* s is NULL if the current string part is an f-string. */
+            Py_XDECREF(s);
             goto error;
         }
         bytesmode = this_bytesmode;
@@ -5161,11 +5162,12 @@
             if (result < 0)
                 goto error;
         } else {
+            /* A string or byte string. */
+            assert(s != NULL && fstr == NULL);
+
             assert(bytesmode ? PyBytes_CheckExact(s) :
                    PyUnicode_CheckExact(s));
 
-            /* A string or byte string. */
-            assert(s != NULL && fstr == NULL);
             if (bytesmode) {
                 /* For bytes, concat as we go. */
                 if (i == 0) {
@@ -5177,7 +5179,6 @@
                         goto error;
                 }
             } else {
-                assert(s != NULL && fstr == NULL);
                 /* This is a regular string. Concatenate it. */
                 if (FstringParser_ConcatAndDel(&state, s) < 0)
                     goto error;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 17:58:09 2016
From: python-checkins at python.org (eric.smith)
Date: Mon, 07 Nov 2016 22:58:09 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgZnJvbSAzLjYu?=
Message-ID: <20161107225809.102193.86142.88EA5FC4@psf.io>

https://hg.python.org/cpython/rev/983a8887a895
changeset:   104960:983a8887a895
parent:      104958:d0e4440a68b3
parent:      104959:31543f7cbdf4
user:        Eric V. Smith <eric at trueblade.com>
date:        Mon Nov 07 17:57:48 2016 -0500
summary:
  Merge from 3.6.

files:
  Lib/test/test_fstring.py |  7 +++++++
  Python/ast.c             |  9 +++++----
  2 files changed, 12 insertions(+), 4 deletions(-)


diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py
--- a/Lib/test/test_fstring.py
+++ b/Lib/test/test_fstring.py
@@ -92,6 +92,13 @@
         exec(c)
         self.assertEqual(x[0], 'foo3')
 
+    def test_compile_time_concat_errors(self):
+        self.assertAllRaise(SyntaxError,
+                            'cannot mix bytes and nonbytes literals',
+                            [r"""f'' b''""",
+                             r"""b'' f''""",
+                             ])
+
     def test_literal(self):
         self.assertEqual(f'', '')
         self.assertEqual(f'a', 'a')
diff --git a/Python/ast.c b/Python/ast.c
--- a/Python/ast.c
+++ b/Python/ast.c
@@ -5147,7 +5147,8 @@
         /* Check that we're not mixing bytes with unicode. */
         if (i != 0 && bytesmode != this_bytesmode) {
             ast_error(c, n, "cannot mix bytes and nonbytes literals");
-            Py_DECREF(s);
+            /* s is NULL if the current string part is an f-string. */
+            Py_XDECREF(s);
             goto error;
         }
         bytesmode = this_bytesmode;
@@ -5161,11 +5162,12 @@
             if (result < 0)
                 goto error;
         } else {
+            /* A string or byte string. */
+            assert(s != NULL && fstr == NULL);
+
             assert(bytesmode ? PyBytes_CheckExact(s) :
                    PyUnicode_CheckExact(s));
 
-            /* A string or byte string. */
-            assert(s != NULL && fstr == NULL);
             if (bytesmode) {
                 /* For bytes, concat as we go. */
                 if (i == 0) {
@@ -5177,7 +5179,6 @@
                         goto error;
                 }
             } else {
-                assert(s != NULL && fstr == NULL);
                 /* This is a regular string. Concatenate it. */
                 if (FstringParser_ConcatAndDel(&state, s) < 0)
                     goto error;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 18:46:27 2016
From: python-checkins at python.org (victor.stinner)
Date: Mon, 07 Nov 2016 23:46:27 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NjM3?=
 =?utf-8?q?=3A_Revert_issue_=2328082=2C_don=27t_import_enum_in_re?=
Message-ID: <20161107234627.7081.74554.3149E5CE@psf.io>

https://hg.python.org/cpython/rev/d903a243c281
changeset:   104961:d903a243c281
branch:      3.6
parent:      104959:31543f7cbdf4
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 08 00:42:46 2016 +0100
summary:
  Issue #28637: Revert issue #28082, don't import enum in re

Importing the enum module in the re module slows down Python startup by 34%
when Python is run from a virtual environment, or more generally when the re
module is imported at startup but not the enum module.

files:
  Lib/re.py |  33 ++++++++++++---------------------
  1 files changed, 12 insertions(+), 21 deletions(-)


diff --git a/Lib/re.py b/Lib/re.py
--- a/Lib/re.py
+++ b/Lib/re.py
@@ -119,7 +119,6 @@
 
 """
 
-import enum
 import sre_compile
 import sre_parse
 import functools
@@ -139,26 +138,18 @@
 
 __version__ = "2.2.1"
 
-class RegexFlag(enum.IntFlag):
-    ASCII = sre_compile.SRE_FLAG_ASCII # assume ascii "locale"
-    IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case
-    LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale
-    UNICODE = sre_compile.SRE_FLAG_UNICODE # assume unicode "locale"
-    MULTILINE = sre_compile.SRE_FLAG_MULTILINE # make anchors look for newline
-    DOTALL = sre_compile.SRE_FLAG_DOTALL # make dot match newline
-    VERBOSE = sre_compile.SRE_FLAG_VERBOSE # ignore whitespace and comments
-    A = ASCII
-    I = IGNORECASE
-    L = LOCALE
-    U = UNICODE
-    M = MULTILINE
-    S = DOTALL
-    X = VERBOSE
-    # sre extensions (experimental, don't rely on these)
-    TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking
-    T = TEMPLATE
-    DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation
-globals().update(RegexFlag.__members__)
+# flags
+A = ASCII = sre_compile.SRE_FLAG_ASCII # assume ascii "locale"
+I = IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case
+L = LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale
+U = UNICODE = sre_compile.SRE_FLAG_UNICODE # assume unicode "locale"
+M = MULTILINE = sre_compile.SRE_FLAG_MULTILINE # make anchors look for newline
+S = DOTALL = sre_compile.SRE_FLAG_DOTALL # make dot match newline
+X = VERBOSE = sre_compile.SRE_FLAG_VERBOSE # ignore whitespace and comments
+
+# sre extensions (experimental, don't rely on these)
+T = TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking
+DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation
 
 # sre exception
 error = sre_compile.error

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 18:46:28 2016
From: python-checkins at python.org (victor.stinner)
Date: Mon, 07 Nov 2016 23:46:28 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42?=
Message-ID: <20161107234627.83454.38931.1FB23632@psf.io>

https://hg.python.org/cpython/rev/90e5aa508a89
changeset:   104962:90e5aa508a89
parent:      104960:983a8887a895
parent:      104961:d903a243c281
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 08 00:44:53 2016 +0100
summary:
  Merge 3.6

files:
  Lib/re.py |  33 ++++++++++++---------------------
  1 files changed, 12 insertions(+), 21 deletions(-)


diff --git a/Lib/re.py b/Lib/re.py
--- a/Lib/re.py
+++ b/Lib/re.py
@@ -119,7 +119,6 @@
 
 """
 
-import enum
 import sre_compile
 import sre_parse
 import functools
@@ -139,26 +138,18 @@
 
 __version__ = "2.2.1"
 
-class RegexFlag(enum.IntFlag):
-    ASCII = sre_compile.SRE_FLAG_ASCII # assume ascii "locale"
-    IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case
-    LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale
-    UNICODE = sre_compile.SRE_FLAG_UNICODE # assume unicode "locale"
-    MULTILINE = sre_compile.SRE_FLAG_MULTILINE # make anchors look for newline
-    DOTALL = sre_compile.SRE_FLAG_DOTALL # make dot match newline
-    VERBOSE = sre_compile.SRE_FLAG_VERBOSE # ignore whitespace and comments
-    A = ASCII
-    I = IGNORECASE
-    L = LOCALE
-    U = UNICODE
-    M = MULTILINE
-    S = DOTALL
-    X = VERBOSE
-    # sre extensions (experimental, don't rely on these)
-    TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking
-    T = TEMPLATE
-    DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation
-globals().update(RegexFlag.__members__)
+# flags
+A = ASCII = sre_compile.SRE_FLAG_ASCII # assume ascii "locale"
+I = IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case
+L = LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale
+U = UNICODE = sre_compile.SRE_FLAG_UNICODE # assume unicode "locale"
+M = MULTILINE = sre_compile.SRE_FLAG_MULTILINE # make anchors look for newline
+S = DOTALL = sre_compile.SRE_FLAG_DOTALL # make dot match newline
+X = VERBOSE = sre_compile.SRE_FLAG_VERBOSE # ignore whitespace and comments
+
+# sre extensions (experimental, don't rely on these)
+T = TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking
+DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation
 
 # sre exception
 error = sre_compile.error

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 19:01:32 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 08 Nov 2016 00:01:32 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjEz?=
 =?utf-8?b?OiBFeHBvc2UgYXN5bmNpby5fZ2V0X3J1bm5pbmdfbG9vcCgpIGFuZCBfc2V0?=
 =?utf-8?b?X3J1bm5pbmdfbG9vcCgp?=
Message-ID: <20161108000132.102333.38968.A2767BE9@psf.io>

https://hg.python.org/cpython/rev/abad0b9a35b3
changeset:   104963:abad0b9a35b3
branch:      3.5
parent:      104952:0669dcf1eb36
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 07 19:00:46 2016 -0500
summary:
  Issue #28613: Expose asyncio._get_running_loop() and _set_running_loop()

files:
  Lib/asyncio/events.py                |  1 +
  Lib/test/test_asyncio/test_events.py |  4 ++++
  2 files changed, 5 insertions(+), 0 deletions(-)


diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py
--- a/Lib/asyncio/events.py
+++ b/Lib/asyncio/events.py
@@ -6,6 +6,7 @@
            'get_event_loop_policy', 'set_event_loop_policy',
            'get_event_loop', 'set_event_loop', 'new_event_loop',
            'get_child_watcher', 'set_child_watcher',
+           '_set_running_loop', '_get_running_loop',
            ]
 
 import functools
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -2732,9 +2732,11 @@
         try:
             asyncio.set_event_loop_policy(Policy())
             loop = asyncio.new_event_loop()
+            self.assertIs(asyncio._get_running_loop(), None)
 
             async def func():
                 self.assertIs(asyncio.get_event_loop(), loop)
+                self.assertIs(asyncio._get_running_loop(), loop)
 
             loop.run_until_complete(func())
         finally:
@@ -2742,6 +2744,8 @@
             if loop is not None:
                 loop.close()
 
+        self.assertIs(asyncio._get_running_loop(), None)
+
 
 if __name__ == '__main__':
     unittest.main()

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 19:01:32 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 08 Nov 2016 00:01:32 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5_=28issue_=2328613=29?=
Message-ID: <20161108000132.30045.96346.1DD24970@psf.io>

https://hg.python.org/cpython/rev/61a237f3bb07
changeset:   104964:61a237f3bb07
branch:      3.6
parent:      104961:d903a243c281
parent:      104963:abad0b9a35b3
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 07 19:01:14 2016 -0500
summary:
  Merge 3.5 (issue #28613)

files:
  Lib/asyncio/events.py                |  1 +
  Lib/test/test_asyncio/test_events.py |  4 ++++
  2 files changed, 5 insertions(+), 0 deletions(-)


diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py
--- a/Lib/asyncio/events.py
+++ b/Lib/asyncio/events.py
@@ -6,6 +6,7 @@
            'get_event_loop_policy', 'set_event_loop_policy',
            'get_event_loop', 'set_event_loop', 'new_event_loop',
            'get_child_watcher', 'set_child_watcher',
+           '_set_running_loop', '_get_running_loop',
            ]
 
 import functools
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -2732,9 +2732,11 @@
         try:
             asyncio.set_event_loop_policy(Policy())
             loop = asyncio.new_event_loop()
+            self.assertIs(asyncio._get_running_loop(), None)
 
             async def func():
                 self.assertIs(asyncio.get_event_loop(), loop)
+                self.assertIs(asyncio._get_running_loop(), loop)
 
             loop.run_until_complete(func())
         finally:
@@ -2742,6 +2744,8 @@
             if loop is not None:
                 loop.close()
 
+        self.assertIs(asyncio._get_running_loop(), None)
+
 
 if __name__ == '__main__':
     unittest.main()

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 19:01:33 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 08 Nov 2016 00:01:33 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg2MTMp?=
Message-ID: <20161108000132.29950.98577.17A3ABF7@psf.io>

https://hg.python.org/cpython/rev/cf7711887b4a
changeset:   104965:cf7711887b4a
parent:      104962:90e5aa508a89
parent:      104964:61a237f3bb07
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 07 19:01:27 2016 -0500
summary:
  Merge 3.6 (issue #28613)

files:
  Lib/asyncio/events.py                |  1 +
  Lib/test/test_asyncio/test_events.py |  4 ++++
  2 files changed, 5 insertions(+), 0 deletions(-)


diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py
--- a/Lib/asyncio/events.py
+++ b/Lib/asyncio/events.py
@@ -6,6 +6,7 @@
            'get_event_loop_policy', 'set_event_loop_policy',
            'get_event_loop', 'set_event_loop', 'new_event_loop',
            'get_child_watcher', 'set_child_watcher',
+           '_set_running_loop', '_get_running_loop',
            ]
 
 import functools
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -2732,9 +2732,11 @@
         try:
             asyncio.set_event_loop_policy(Policy())
             loop = asyncio.new_event_loop()
+            self.assertIs(asyncio._get_running_loop(), None)
 
             async def func():
                 self.assertIs(asyncio.get_event_loop(), loop)
+                self.assertIs(asyncio._get_running_loop(), loop)
 
             loop.run_until_complete(func())
         finally:
@@ -2742,6 +2744,8 @@
             if loop is not None:
                 loop.close()
 
+        self.assertIs(asyncio._get_running_loop(), None)
+
 
 if __name__ == '__main__':
     unittest.main()

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 23:15:25 2016
From: python-checkins at python.org (terry.reedy)
Date: Tue, 08 Nov 2016 04:15:25 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NTcy?=
 =?utf-8?q?=3A_Use_system-specific_values_for_configdialog_font_test?=
Message-ID: <20161108041525.22961.45451.365FB588@psf.io>

https://hg.python.org/cpython/rev/f604b6ebd802
changeset:   104966:f604b6ebd802
branch:      3.6
parent:      104964:61a237f3bb07
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Mon Nov 07 23:14:53 2016 -0500
summary:
  Issue #28572: Use system-specific values for configdialog font test

files:
  Lib/idlelib/config-main.def                |   2 +
  Lib/idlelib/idle_test/test_configdialog.py |  20 ++++++----
  2 files changed, 14 insertions(+), 8 deletions(-)


diff --git a/Lib/idlelib/config-main.def b/Lib/idlelib/config-main.def
--- a/Lib/idlelib/config-main.def
+++ b/Lib/idlelib/config-main.def
@@ -60,6 +60,8 @@
 width= 80
 height= 40
 font= TkFixedFont
+# For TkFixedFont, the actual size and boldness are obtained from tk
+# and override 10 and 0.  See idlelib.config.IdleConf.GetFont
 font-size= 10
 font-bold= 0
 encoding= none
diff --git a/Lib/idlelib/idle_test/test_configdialog.py b/Lib/idlelib/idle_test/test_configdialog.py
--- a/Lib/idlelib/idle_test/test_configdialog.py
+++ b/Lib/idlelib/idle_test/test_configdialog.py
@@ -54,25 +54,29 @@
         changes.clear()
 
     def test_font(self):
+        # Set values guaranteed not to be defaults.
+        dfont = idleConf.GetFont(root, 'main', 'EditorWindow')
+        dsize = str(dfont[1])
+        dbold = dfont[2] == 'bold'
         configure.fontName.set('Test Font')
         expected = [
             ('main', 'EditorWindow', 'font', 'Test Font'),
-            ('main', 'EditorWindow', 'font-size', '10'),
-            ('main', 'EditorWindow', 'font-bold', False)]
+            ('main', 'EditorWindow', 'font-size', dsize),
+            ('main', 'EditorWindow', 'font-bold', dbold)]
         self.assertEqual(changes, expected)
         changes.clear()
-        configure.fontSize.set(12)
+        configure.fontSize.set(20)
         expected = [
             ('main', 'EditorWindow', 'font', 'Test Font'),
-            ('main', 'EditorWindow', 'font-size', '12'),
-            ('main', 'EditorWindow', 'font-bold', False)]
+            ('main', 'EditorWindow', 'font-size', '20'),
+            ('main', 'EditorWindow', 'font-bold', dbold)]
         self.assertEqual(changes, expected)
         changes.clear()
-        configure.fontBold.set(True)
+        configure.fontBold.set(not dbold)
         expected = [
             ('main', 'EditorWindow', 'font', 'Test Font'),
-            ('main', 'EditorWindow', 'font-size', '12'),
-            ('main', 'EditorWindow', 'font-bold', True)]
+            ('main', 'EditorWindow', 'font-size', '20'),
+            ('main', 'EditorWindow', 'font-bold', not dbold)]
         self.assertEqual(changes, expected)
 
     #def test_sample(self): pass  # TODO

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov  7 23:15:25 2016
From: python-checkins at python.org (terry.reedy)
Date: Tue, 08 Nov 2016 04:15:25 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_with_3=2E6?=
Message-ID: <20161108041525.111593.64990.FEB51D9B@psf.io>

https://hg.python.org/cpython/rev/e1dfecfcfa04
changeset:   104967:e1dfecfcfa04
parent:      104965:cf7711887b4a
parent:      104966:f604b6ebd802
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Mon Nov 07 23:15:10 2016 -0500
summary:
  Merge with 3.6

files:
  Lib/idlelib/config-main.def                |   2 +
  Lib/idlelib/idle_test/test_configdialog.py |  20 ++++++----
  2 files changed, 14 insertions(+), 8 deletions(-)


diff --git a/Lib/idlelib/config-main.def b/Lib/idlelib/config-main.def
--- a/Lib/idlelib/config-main.def
+++ b/Lib/idlelib/config-main.def
@@ -60,6 +60,8 @@
 width= 80
 height= 40
 font= TkFixedFont
+# For TkFixedFont, the actual size and boldness are obtained from tk
+# and override 10 and 0.  See idlelib.config.IdleConf.GetFont
 font-size= 10
 font-bold= 0
 encoding= none
diff --git a/Lib/idlelib/idle_test/test_configdialog.py b/Lib/idlelib/idle_test/test_configdialog.py
--- a/Lib/idlelib/idle_test/test_configdialog.py
+++ b/Lib/idlelib/idle_test/test_configdialog.py
@@ -54,25 +54,29 @@
         changes.clear()
 
     def test_font(self):
+        # Set values guaranteed not to be defaults.
+        dfont = idleConf.GetFont(root, 'main', 'EditorWindow')
+        dsize = str(dfont[1])
+        dbold = dfont[2] == 'bold'
         configure.fontName.set('Test Font')
         expected = [
             ('main', 'EditorWindow', 'font', 'Test Font'),
-            ('main', 'EditorWindow', 'font-size', '10'),
-            ('main', 'EditorWindow', 'font-bold', False)]
+            ('main', 'EditorWindow', 'font-size', dsize),
+            ('main', 'EditorWindow', 'font-bold', dbold)]
         self.assertEqual(changes, expected)
         changes.clear()
-        configure.fontSize.set(12)
+        configure.fontSize.set(20)
         expected = [
             ('main', 'EditorWindow', 'font', 'Test Font'),
-            ('main', 'EditorWindow', 'font-size', '12'),
-            ('main', 'EditorWindow', 'font-bold', False)]
+            ('main', 'EditorWindow', 'font-size', '20'),
+            ('main', 'EditorWindow', 'font-bold', dbold)]
         self.assertEqual(changes, expected)
         changes.clear()
-        configure.fontBold.set(True)
+        configure.fontBold.set(not dbold)
         expected = [
             ('main', 'EditorWindow', 'font', 'Test Font'),
-            ('main', 'EditorWindow', 'font-size', '12'),
-            ('main', 'EditorWindow', 'font-bold', True)]
+            ('main', 'EditorWindow', 'font-size', '20'),
+            ('main', 'EditorWindow', 'font-bold', not dbold)]
         self.assertEqual(changes, expected)
 
     #def test_sample(self): pass  # TODO

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 12:23:32 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 08 Nov 2016 17:23:32 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChkb2NzL2luc3BlY3Qp?=
Message-ID: <20161108172330.83458.20582.4DBC3D48@psf.io>

https://hg.python.org/cpython/rev/06700c13b640
changeset:   104969:06700c13b640
parent:      104967:e1dfecfcfa04
parent:      104968:f7f89a4e047d
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 12:23:26 2016 -0500
summary:
  Merge 3.6 (docs/inspect)

files:
  Lib/inspect.py |  9 +++++++--
  1 files changed, 7 insertions(+), 2 deletions(-)


diff --git a/Lib/inspect.py b/Lib/inspect.py
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -179,17 +179,22 @@
 def iscoroutinefunction(object):
     """Return true if the object is a coroutine function.
 
-    Coroutine functions are defined with "async def" syntax,
-    or generators decorated with "types.coroutine".
+    Coroutine functions are defined with "async def" syntax.
     """
     return bool((isfunction(object) or ismethod(object)) and
                 object.__code__.co_flags & CO_COROUTINE)
 
 def isasyncgenfunction(object):
+    """Return true if the object is an asynchronous generator function.
+
+    Asynchronous generator functions are defined with "async def"
+    syntax and have "yield" expressions in their body.
+    """
     return bool((isfunction(object) or ismethod(object)) and
                 object.__code__.co_flags & CO_ASYNC_GENERATOR)
 
 def isasyncgen(object):
+    """Return true if the object is an asynchronous generator."""
     return isinstance(object, types.AsyncGeneratorType)
 
 def isgenerator(object):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 12:23:43 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 08 Nov 2016 17:23:43 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_docs/inspect?=
 =?utf-8?q?=3A_clarify_iscoroutinefunction=3B_add_docs_for_isasyncgen*?=
Message-ID: <20161108172330.7760.44191.6A489ABE@psf.io>

https://hg.python.org/cpython/rev/f7f89a4e047d
changeset:   104968:f7f89a4e047d
branch:      3.6
parent:      104966:f604b6ebd802
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 12:23:09 2016 -0500
summary:
  docs/inspect: clarify iscoroutinefunction; add docs for isasyncgen*

files:
  Lib/inspect.py |  9 +++++++--
  1 files changed, 7 insertions(+), 2 deletions(-)


diff --git a/Lib/inspect.py b/Lib/inspect.py
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -179,17 +179,22 @@
 def iscoroutinefunction(object):
     """Return true if the object is a coroutine function.
 
-    Coroutine functions are defined with "async def" syntax,
-    or generators decorated with "types.coroutine".
+    Coroutine functions are defined with "async def" syntax.
     """
     return bool((isfunction(object) or ismethod(object)) and
                 object.__code__.co_flags & CO_COROUTINE)
 
 def isasyncgenfunction(object):
+    """Return true if the object is an asynchronous generator function.
+
+    Asynchronous generator functions are defined with "async def"
+    syntax and have "yield" expressions in their body.
+    """
     return bool((isfunction(object) or ismethod(object)) and
                 object.__code__.co_flags & CO_ASYNC_GENERATOR)
 
 def isasyncgen(object):
+    """Return true if the object is an asynchronous generator."""
     return isinstance(object, types.AsyncGeneratorType)
 
 def isgenerator(object):

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Tue Nov  8 12:30:20 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Tue, 8 Nov 2016 17:30:20 +0000
Subject: [Python-checkins] BAD Benchmark Results for Python Default
 2016-11-08
Message-ID: <d87cad75-d3d6-476a-8f5e-4445e3d458da@irsmsx151.ger.corp.intel.com>

Results for project Python default, build date 2016-11-08 03:01:58 +0000
commit:		cf7711887b4a
previous commit:	e60c1aef639a
revision date:	2016-11-08 00:01:27 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.21%          0.60%          6.27%            14.81%
:-)             pybench      0.07%          0.07%          5.30%             5.62%
:-(            regex_v8      2.75%         -0.23%         -3.10%             4.71%
:-)               nbody      0.09%         -0.06%          2.08%             4.07%
:-(        json_dump_v2      0.38%         -1.14%        -10.27%            15.23%
:-(      normal_startup      0.10%          0.10%         -2.04%             6.38%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-default-2016-11-08/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Tue Nov  8 12:30:00 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Tue, 8 Nov 2016 17:30:00 +0000
Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7
 2016-11-08
Message-ID: <6b966f22-968d-4efb-b876-61af04d846f4@irsmsx151.ger.corp.intel.com>

Results for project Python 2.7, build date 2016-11-08 03:48:26 +0000
commit:		1f61a5d2f448
previous commit:	2b58cde6a7f2
revision date:	2016-11-07 19:29:07 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.18%         -0.16%          4.93%             4.99%
:-)             pybench      0.10%          0.04%          5.83%             4.30%
:-(            regex_v8      0.68%         -0.09%         -2.16%            10.97%
:-)               nbody      0.07%          0.01%          7.97%             4.54%
:-)        json_dump_v2      0.29%         -0.06%          2.66%            10.71%
:-|      normal_startup      0.69%         -0.01%         -0.52%             2.42%
:-)             ssbench      0.21%         -0.05%          2.29%             2.24%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-11-08/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Tue Nov  8 13:18:34 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 08 Nov 2016 18:18:34 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328637=3A_No_longer_use_re_in_site=2Epy=2E?=
Message-ID: <20161108181834.29840.83424.8B70CC8D@psf.io>

https://hg.python.org/cpython/rev/a4ea837a7f84
changeset:   104971:a4ea837a7f84
parent:      104969:06700c13b640
parent:      104970:a822818ec74e
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 08 20:18:20 2016 +0200
summary:
  Issue #28637: No longer use re in site.py.
This makes Python startup from a virtual environment a little faster.

files:
  Lib/site.py |  13 ++++---------
  1 files changed, 4 insertions(+), 9 deletions(-)


diff --git a/Lib/site.py b/Lib/site.py
--- a/Lib/site.py
+++ b/Lib/site.py
@@ -422,8 +422,6 @@
 
     sys.__interactivehook__ = register_readline
 
-CONFIG_LINE = r'^(?P<key>(\w|[-_])+)\s*=\s*(?P<value>.*)\s*$'
-
 def venv(known_paths):
     global PREFIXES, ENABLE_USER_SITE
 
@@ -445,19 +443,16 @@
         ]
 
     if candidate_confs:
-        import re
-        config_line = re.compile(CONFIG_LINE)
         virtual_conf = candidate_confs[0]
         system_site = "true"
         # Issue 25185: Use UTF-8, as that's what the venv module uses when
         # writing the file.
         with open(virtual_conf, encoding='utf-8') as f:
             for line in f:
-                line = line.strip()
-                m = config_line.match(line)
-                if m:
-                    d = m.groupdict()
-                    key, value = d['key'].lower(), d['value']
+                if '=' in line:
+                    key, _, value = line.partition('=')
+                    key = key.strip().lower()
+                    value = value.strip()
                     if key == 'include-system-site-packages':
                         system_site = value.lower()
                     elif key == 'home':

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 13:18:36 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 08 Nov 2016 18:18:36 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NjM3?=
 =?utf-8?q?=3A_No_longer_use_re_in_site=2Epy=2E?=
Message-ID: <20161108181833.31027.43610.4608A14E@psf.io>

https://hg.python.org/cpython/rev/a822818ec74e
changeset:   104970:a822818ec74e
branch:      3.6
parent:      104968:f7f89a4e047d
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 08 20:17:35 2016 +0200
summary:
  Issue #28637: No longer use re in site.py.
This makes Python startup from a virtual environment a little faster.

files:
  Lib/site.py |  13 ++++---------
  1 files changed, 4 insertions(+), 9 deletions(-)


diff --git a/Lib/site.py b/Lib/site.py
--- a/Lib/site.py
+++ b/Lib/site.py
@@ -422,8 +422,6 @@
 
     sys.__interactivehook__ = register_readline
 
-CONFIG_LINE = r'^(?P<key>(\w|[-_])+)\s*=\s*(?P<value>.*)\s*$'
-
 def venv(known_paths):
     global PREFIXES, ENABLE_USER_SITE
 
@@ -445,19 +443,16 @@
         ]
 
     if candidate_confs:
-        import re
-        config_line = re.compile(CONFIG_LINE)
         virtual_conf = candidate_confs[0]
         system_site = "true"
         # Issue 25185: Use UTF-8, as that's what the venv module uses when
         # writing the file.
         with open(virtual_conf, encoding='utf-8') as f:
             for line in f:
-                line = line.strip()
-                m = config_line.match(line)
-                if m:
-                    d = m.groupdict()
-                    key, value = d['key'].lower(), d['value']
+                if '=' in line:
+                    key, _, value = line.partition('=')
+                    key = key.strip().lower()
+                    value = value.strip()
                     if key == 'include-system-site-packages':
                         system_site = value.lower()
                     elif key == 'home':

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 13:29:07 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 08 Nov 2016 18:29:07 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328585=3A_Restored_docstring_of_os=2E=5Fisdir=28?=
 =?utf-8?b?KS4=?=
Message-ID: <20161108182904.7609.51516.6E9E72F6@psf.io>

https://hg.python.org/cpython/rev/d77fe603cded
changeset:   104974:d77fe603cded
parent:      104971:a4ea837a7f84
parent:      104973:3da89b1678da
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 08 20:28:43 2016 +0200
summary:
  Issue #28585: Restored docstring of os._isdir().

files:
  Modules/clinic/posixmodule.c.h |  5 +++--
  Modules/posixmodule.c          |  6 ++----
  2 files changed, 5 insertions(+), 6 deletions(-)


diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h
--- a/Modules/clinic/posixmodule.c.h
+++ b/Modules/clinic/posixmodule.c.h
@@ -995,7 +995,8 @@
 PyDoc_STRVAR(os__isdir__doc__,
 "_isdir($module, path, /)\n"
 "--\n"
-"\n");
+"\n"
+"Return true if the pathname refers to an existing directory.");
 
 #define OS__ISDIR_METHODDEF    \
     {"_isdir", (PyCFunction)os__isdir, METH_O, os__isdir__doc__},
@@ -6351,4 +6352,4 @@
 #ifndef OS_GETRANDOM_METHODDEF
     #define OS_GETRANDOM_METHODDEF
 #endif /* !defined(OS_GETRANDOM_METHODDEF) */
-/*[clinic end generated code: output=e4a3bd36c7bb8356 input=a9049054013a1b77]*/
+/*[clinic end generated code: output=61abf6df195aa5f1 input=a9049054013a1b77]*/
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -3717,20 +3717,18 @@
     return result;
 }
 
-PyDoc_STRVAR(posix__isdir__doc__,
-"Return true if the pathname refers to an existing directory.");
-
 /*[clinic input]
 os._isdir
 
     path: path_t
     /
 
+Return true if the pathname refers to an existing directory.
 [clinic start generated code]*/
 
 static PyObject *
 os__isdir_impl(PyObject *module, path_t *path)
-/*[clinic end generated code: output=75f56f32720836cb input=e794f12faab62a2a]*/
+/*[clinic end generated code: output=75f56f32720836cb input=5e0800149c0ad95f]*/
 {
     DWORD attributes;
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 13:29:07 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 08 Nov 2016 18:29:07 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NTg1?=
 =?utf-8?q?=3A_Restored_docstring_of_os=2E=5Fisdir=28=29=2E?=
Message-ID: <20161108182903.3243.58288.ADFB17B9@psf.io>

https://hg.python.org/cpython/rev/5b4fa92dac43
changeset:   104972:5b4fa92dac43
branch:      3.5
parent:      104963:abad0b9a35b3
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 08 20:21:22 2016 +0200
summary:
  Issue #28585: Restored docstring of os._isdir().

files:
  Modules/clinic/posixmodule.c.h |  5 +++--
  Modules/posixmodule.c          |  6 ++----
  2 files changed, 5 insertions(+), 6 deletions(-)


diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h
--- a/Modules/clinic/posixmodule.c.h
+++ b/Modules/clinic/posixmodule.c.h
@@ -954,7 +954,8 @@
 PyDoc_STRVAR(os__isdir__doc__,
 "_isdir($module, path, /)\n"
 "--\n"
-"\n");
+"\n"
+"Return true if the pathname refers to an existing directory.");
 
 #define OS__ISDIR_METHODDEF    \
     {"_isdir", (PyCFunction)os__isdir, METH_O, os__isdir__doc__},
@@ -5784,4 +5785,4 @@
 #ifndef OS_SET_HANDLE_INHERITABLE_METHODDEF
     #define OS_SET_HANDLE_INHERITABLE_METHODDEF
 #endif /* !defined(OS_SET_HANDLE_INHERITABLE_METHODDEF) */
-/*[clinic end generated code: output=9d5f831b23145d1e input=a9049054013a1b77]*/
+/*[clinic end generated code: output=7690b72549d2524e input=a9049054013a1b77]*/
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -3865,20 +3865,18 @@
     return result;
 }
 
-PyDoc_STRVAR(posix__isdir__doc__,
-"Return true if the pathname refers to an existing directory.");
-
 /*[clinic input]
 os._isdir
 
     path: path_t
     /
 
+Return true if the pathname refers to an existing directory.
 [clinic start generated code]*/
 
 static PyObject *
 os__isdir_impl(PyObject *module, path_t *path)
-/*[clinic end generated code: output=75f56f32720836cb input=e794f12faab62a2a]*/
+/*[clinic end generated code: output=75f56f32720836cb input=5e0800149c0ad95f]*/
 {
     DWORD attributes;
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 13:29:07 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 08 Nov 2016 18:29:07 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328585=3A_Restored_docstring_of_os=2E=5Fisdir=28=29=2E?=
Message-ID: <20161108182904.30992.66060.AD795D11@psf.io>

https://hg.python.org/cpython/rev/3da89b1678da
changeset:   104973:3da89b1678da
branch:      3.6
parent:      104970:a822818ec74e
parent:      104972:5b4fa92dac43
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 08 20:26:18 2016 +0200
summary:
  Issue #28585: Restored docstring of os._isdir().

files:
  Modules/clinic/posixmodule.c.h |  5 +++--
  Modules/posixmodule.c          |  6 ++----
  2 files changed, 5 insertions(+), 6 deletions(-)


diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h
--- a/Modules/clinic/posixmodule.c.h
+++ b/Modules/clinic/posixmodule.c.h
@@ -995,7 +995,8 @@
 PyDoc_STRVAR(os__isdir__doc__,
 "_isdir($module, path, /)\n"
 "--\n"
-"\n");
+"\n"
+"Return true if the pathname refers to an existing directory.");
 
 #define OS__ISDIR_METHODDEF    \
     {"_isdir", (PyCFunction)os__isdir, METH_O, os__isdir__doc__},
@@ -6148,4 +6149,4 @@
 #ifndef OS_GETRANDOM_METHODDEF
     #define OS_GETRANDOM_METHODDEF
 #endif /* !defined(OS_GETRANDOM_METHODDEF) */
-/*[clinic end generated code: output=b9ed5703d2feb0d9 input=a9049054013a1b77]*/
+/*[clinic end generated code: output=50cfb7ebc44efb67 input=a9049054013a1b77]*/
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -3717,20 +3717,18 @@
     return result;
 }
 
-PyDoc_STRVAR(posix__isdir__doc__,
-"Return true if the pathname refers to an existing directory.");
-
 /*[clinic input]
 os._isdir
 
     path: path_t
     /
 
+Return true if the pathname refers to an existing directory.
 [clinic start generated code]*/
 
 static PyObject *
 os__isdir_impl(PyObject *module, path_t *path)
-/*[clinic end generated code: output=75f56f32720836cb input=e794f12faab62a2a]*/
+/*[clinic end generated code: output=75f56f32720836cb input=5e0800149c0ad95f]*/
 {
     DWORD attributes;
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 13:34:44 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 08 Nov 2016 18:34:44 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328621=3A_Sped_up_?=
 =?utf-8?q?converting_int_to_float_by_reusing_faster_bits_counting?=
Message-ID: <20161108183435.3153.57885.168F41EF@psf.io>

https://hg.python.org/cpython/rev/1940b72b0a02
changeset:   104975:1940b72b0a02
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 08 20:34:22 2016 +0200
summary:
  Issue #28621: Sped up converting int to float by reusing faster bits counting
implementation.  Patch by Adrian Wielgosik.

files:
  Misc/NEWS            |   3 +
  Objects/longobject.c |  60 ++++++++++++++-----------------
  2 files changed, 31 insertions(+), 32 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #28621: Sped up converting int to float by reusing faster bits counting
+  implementation.  Patch by Adrian Wielgosik.
+
 - Issue #28580: Optimize iterating split table values.
   Patch by Xiang Zhang.
 
diff --git a/Objects/longobject.c b/Objects/longobject.c
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -708,12 +708,33 @@
     return Py_SIZE(v) == 0 ? 0 : (Py_SIZE(v) < 0 ? -1 : 1);
 }
 
+/* bits_in_digit(d) returns the unique integer k such that 2**(k-1) <= d <
+   2**k if d is nonzero, else 0. */
+
+static const unsigned char BitLengthTable[32] = {
+    0, 1, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4,
+    5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5
+};
+
+static int
+bits_in_digit(digit d)
+{
+    int d_bits = 0;
+    while (d >= 32) {
+        d_bits += 6;
+        d >>= 6;
+    }
+    d_bits += (int)BitLengthTable[d];
+    return d_bits;
+}
+
 size_t
 _PyLong_NumBits(PyObject *vv)
 {
     PyLongObject *v = (PyLongObject *)vv;
     size_t result = 0;
     Py_ssize_t ndigits;
+    int msd_bits;
 
     assert(v != NULL);
     assert(PyLong_Check(v));
@@ -724,12 +745,10 @@
         if ((size_t)(ndigits - 1) > SIZE_MAX / (size_t)PyLong_SHIFT)
             goto Overflow;
         result = (size_t)(ndigits - 1) * (size_t)PyLong_SHIFT;
-        do {
-            ++result;
-            if (result == 0)
-                goto Overflow;
-            msd >>= 1;
-        } while (msd);
+        msd_bits = bits_in_digit(msd);
+        if (SIZE_MAX - msd_bits < result)
+            goto Overflow;
+        result += msd_bits;
     }
     return result;
 
@@ -1414,26 +1433,6 @@
             Py_RETURN_NOTIMPLEMENTED;                   \
     } while(0)
 
-/* bits_in_digit(d) returns the unique integer k such that 2**(k-1) <= d <
-   2**k if d is nonzero, else 0. */
-
-static const unsigned char BitLengthTable[32] = {
-    0, 1, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4,
-    5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5
-};
-
-static int
-bits_in_digit(digit d)
-{
-    int d_bits = 0;
-    while (d >= 32) {
-        d_bits += 6;
-        d >>= 6;
-    }
-    d_bits += (int)BitLengthTable[d];
-    return d_bits;
-}
-
 /* x[0:m] and y[0:n] are digit vectors, LSD first, m >= n required.  x[0:n]
  * is modified in place, by adding y to it.  Carries are propagated as far as
  * x[m-1], and the remaining carry (0 or 1) is returned.
@@ -5079,7 +5078,8 @@
 long_bit_length(PyLongObject *v)
 {
     PyLongObject *result, *x, *y;
-    Py_ssize_t ndigits, msd_bits = 0;
+    Py_ssize_t ndigits;
+    int msd_bits;
     digit msd;
 
     assert(v != NULL);
@@ -5090,11 +5090,7 @@
         return PyLong_FromLong(0);
 
     msd = v->ob_digit[ndigits-1];
-    while (msd >= 32) {
-        msd_bits += 6;
-        msd >>= 6;
-    }
-    msd_bits += (long)(BitLengthTable[msd]);
+    msd_bits = bits_in_digit(msd);
 
     if (ndigits <= PY_SSIZE_T_MAX/PyLong_SHIFT)
         return PyLong_FromSsize_t((ndigits-1)*PyLong_SHIFT + msd_bits);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 14:28:52 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 08 Nov 2016 19:28:52 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4NTYz?=
 =?utf-8?q?=3A_Fixed_possible_DoS_and_arbitrary_code_execution_when_handle?=
Message-ID: <20161108192852.2079.96442.ED4D7597@psf.io>

https://hg.python.org/cpython/rev/e0cc3fadd7b3
changeset:   104976:e0cc3fadd7b3
branch:      2.7
parent:      104945:1f61a5d2f448
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 08 21:15:55 2016 +0200
summary:
  Issue #28563: Fixed possible DoS and arbitrary code execution when handle
plural form selections in the gettext module.  The expression parser now
supports exact syntax supported by GNU gettext.

files:
  Lib/gettext.py           |  177 ++++++++++++++++++--------
  Lib/test/test_gettext.py |   85 ++++++++++++-
  Misc/NEWS                |    4 +
  3 files changed, 209 insertions(+), 57 deletions(-)


diff --git a/Lib/gettext.py b/Lib/gettext.py
--- a/Lib/gettext.py
+++ b/Lib/gettext.py
@@ -59,74 +59,139 @@
 
 _default_localedir = os.path.join(sys.prefix, 'share', 'locale')
 
+# Expression parsing for plural form selection.
+#
+# The gettext library supports a small subset of C syntax.  The only
+# incompatible difference is that integer literals starting with zero are
+# decimal.
+#
+# https://www.gnu.org/software/gettext/manual/gettext.html#Plural-forms
+# http://git.savannah.gnu.org/cgit/gettext.git/tree/gettext-runtime/intl/plural.y
 
-def test(condition, true, false):
-    """
-    Implements the C expression:
+_token_pattern = re.compile(r"""
+        (?P<WHITESPACES>[ \t]+)                    | # spaces and horizontal tabs
+        (?P<NUMBER>[0-9]+\b)                       | # decimal integer
+        (?P<NAME>n\b)                              | # only n is allowed
+        (?P<PARENTHESIS>[()])                      |
+        (?P<OPERATOR>[-*/%+?:]|[><!]=?|==|&&|\|\|) | # !, *, /, %, +, -, <, >,
+                                                     # <=, >=, ==, !=, &&, ||,
+                                                     # ? :
+                                                     # unary and bitwise ops
+                                                     # not allowed
+        (?P<INVALID>\w+|.)                           # invalid token
+    """, re.VERBOSE|re.DOTALL)
 
-      condition ? true : false
+def _tokenize(plural):
+    for mo in re.finditer(_token_pattern, plural):
+        kind = mo.lastgroup
+        if kind == 'WHITESPACES':
+            continue
+        value = mo.group(kind)
+        if kind == 'INVALID':
+            raise ValueError('invalid token in plural form: %s' % value)
+        yield value
+    yield ''
 
-    Required to correctly interpret plural forms.
-    """
-    if condition:
-        return true
+def _error(value):
+    if value:
+        return ValueError('unexpected token in plural form: %s' % value)
     else:
-        return false
+        return ValueError('unexpected end of plural form')
 
+_binary_ops = (
+    ('||',),
+    ('&&',),
+    ('==', '!='),
+    ('<', '>', '<=', '>='),
+    ('+', '-'),
+    ('*', '/', '%'),
+)
+_binary_ops = {op: i for i, ops in enumerate(_binary_ops, 1) for op in ops}
+_c2py_ops = {'||': 'or', '&&': 'and', '/': '//'}
+
+def _parse(tokens, priority=-1):
+    result = ''
+    nexttok = next(tokens)
+    while nexttok == '!':
+        result += 'not '
+        nexttok = next(tokens)
+
+    if nexttok == '(':
+        sub, nexttok = _parse(tokens)
+        result = '%s(%s)' % (result, sub)
+        if nexttok != ')':
+            raise ValueError('unbalanced parenthesis in plural form')
+    elif nexttok == 'n':
+        result = '%s%s' % (result, nexttok)
+    else:
+        try:
+            value = int(nexttok, 10)
+        except ValueError:
+            raise _error(nexttok)
+        result = '%s%d' % (result, value)
+    nexttok = next(tokens)
+
+    j = 100
+    while nexttok in _binary_ops:
+        i = _binary_ops[nexttok]
+        if i < priority:
+            break
+        # Break chained comparisons
+        if i in (3, 4) and j in (3, 4):  # '==', '!=', '<', '>', '<=', '>='
+            result = '(%s)' % result
+        # Replace some C operators by their Python equivalents
+        op = _c2py_ops.get(nexttok, nexttok)
+        right, nexttok = _parse(tokens, i + 1)
+        result = '%s %s %s' % (result, op, right)
+        j = i
+    if j == priority == 4:  # '<', '>', '<=', '>='
+        result = '(%s)' % result
+
+    if nexttok == '?' and priority <= 0:
+        if_true, nexttok = _parse(tokens, 0)
+        if nexttok != ':':
+            raise _error(nexttok)
+        if_false, nexttok = _parse(tokens)
+        result = '%s if %s else %s' % (if_true, result, if_false)
+        if priority == 0:
+            result = '(%s)' % result
+
+    return result, nexttok
 
 def c2py(plural):
     """Gets a C expression as used in PO files for plural forms and returns a
-    Python lambda function that implements an equivalent expression.
+    Python function that implements an equivalent expression.
     """
-    # Security check, allow only the "n" identifier
+
+    if len(plural) > 1000:
+        raise ValueError('plural form expression is too long')
     try:
-        from cStringIO import StringIO
-    except ImportError:
-        from StringIO import StringIO
-    import token, tokenize
-    tokens = tokenize.generate_tokens(StringIO(plural).readline)
-    try:
-        danger = [x for x in tokens if x[0] == token.NAME and x[1] != 'n']
-    except tokenize.TokenError:
-        raise ValueError, \
-              'plural forms expression error, maybe unbalanced parenthesis'
-    else:
-        if danger:
-            raise ValueError, 'plural forms expression could be dangerous'
+        result, nexttok = _parse(_tokenize(plural))
+        if nexttok:
+            raise _error(nexttok)
 
-    # Replace some C operators by their Python equivalents
-    plural = plural.replace('&&', ' and ')
-    plural = plural.replace('||', ' or ')
+        depth = 0
+        for c in result:
+            if c == '(':
+                depth += 1
+                if depth > 20:
+                    # Python compiler limit is about 90.
+                    # The most complex example has 2.
+                    raise ValueError('plural form expression is too complex')
+            elif c == ')':
+                depth -= 1
 
-    expr = re.compile(r'\!([^=])')
-    plural = expr.sub(' not \\1', plural)
-
-    # Regular expression and replacement function used to transform
-    # "a?b:c" to "test(a,b,c)".
-    expr = re.compile(r'(.*?)\?(.*?):(.*)')
-    def repl(x):
-        return "test(%s, %s, %s)" % (x.group(1), x.group(2),
-                                     expr.sub(repl, x.group(3)))
-
-    # Code to transform the plural expression, taking care of parentheses
-    stack = ['']
-    for c in plural:
-        if c == '(':
-            stack.append('')
-        elif c == ')':
-            if len(stack) == 1:
-                # Actually, we never reach this code, because unbalanced
-                # parentheses get caught in the security check at the
-                # beginning.
-                raise ValueError, 'unbalanced parenthesis in plural form'
-            s = expr.sub(repl, stack.pop())
-            stack[-1] += '(%s)' % s
-        else:
-            stack[-1] += c
-    plural = expr.sub(repl, stack.pop())
-
-    return eval('lambda n: int(%s)' % plural)
-
+        ns = {}
+        exec('''if 1:
+            def func(n):
+                if not isinstance(n, int):
+                    raise ValueError('Plural value must be an integer.')
+                return int(%s)
+            ''' % result, ns)
+        return ns['func']
+    except RuntimeError:
+        # Recursion error can be raised in _parse() or exec().
+        raise ValueError('plural form expression is too complex')
 
 
 def _expand_lang(locale):
diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py
--- a/Lib/test/test_gettext.py
+++ b/Lib/test/test_gettext.py
@@ -237,7 +237,9 @@
         x = t.ngettext('There is %s file', 'There are %s files', 2)
         eq(x, 'Hay %s ficheros')
 
-    def test_hu(self):
+    # Examples from http://www.gnu.org/software/gettext/manual/gettext.html
+
+    def test_ja(self):
         eq = self.assertEqual
         f = gettext.c2py('0')
         s = ''.join([ str(f(x)) for x in range(200) ])
@@ -255,6 +257,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "00111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111")
 
+    def test_lv(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n != 0 ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "20111111111111111111101111111110111111111011111111101111111110111111111011111111101111111110111111111011111111111111111110111111111011111111101111111110111111111011111111101111111110111111111011111111")
+
     def test_gd(self):
         eq = self.assertEqual
         f = gettext.c2py('n==1 ? 0 : n==2 ? 1 : 2')
@@ -268,6 +276,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "20122222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222")
 
+    def test_ro(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n==1 ? 0 : (n==0 || (n%100 > 0 && n%100 < 20)) ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "10111111111111111111222222222222222222222222222222222222222222222222222222222222222222222222222222222111111111111111111122222222222222222222222222222222222222222222222222222222222222222222222222222222")
+
     def test_lt(self):
         eq = self.assertEqual
         f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n%10>=2 && (n%100<10 || n%100>=20) ? 1 : 2')
@@ -280,6 +294,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "20111222222222222222201112222220111222222011122222201112222220111222222011122222201112222220111222222011122222222222222220111222222011122222201112222220111222222011122222201112222220111222222011122222")
 
+    def test_cs(self):
+        eq = self.assertEqual
+        f = gettext.c2py('(n==1) ? 0 : (n>=2 && n<=4) ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "20111222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222")
+
     def test_pl(self):
         eq = self.assertEqual
         f = gettext.c2py('n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2')
@@ -292,10 +312,73 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "30122333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333012233333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333")
 
+    def test_ar(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n==0 ? 0 : n==1 ? 1 : n==2 ? 2 : n%100>=3 && n%100<=10 ? 3 : n%100>=11 ? 4 : 5')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "01233333333444444444444444444444444444444444444444444444444444444444444444444444444444444444444444445553333333344444444444444444444444444444444444444444444444444444444444444444444444444444444444444444")
+
     def test_security(self):
         raises = self.assertRaises
         # Test for a dangerous expression
         raises(ValueError, gettext.c2py, "os.chmod('/etc/passwd',0777)")
+        # issue28563
+        raises(ValueError, gettext.c2py, '"(eval(foo) && ""')
+        raises(ValueError, gettext.c2py, 'f"{os.system(\'sh\')}"')
+        # Maximum recursion depth exceeded during compilation
+        raises(ValueError, gettext.c2py, 'n+'*10000 + 'n')
+        self.assertEqual(gettext.c2py('n+'*100 + 'n')(1), 101)
+        # MemoryError during compilation
+        raises(ValueError, gettext.c2py, '('*100 + 'n' + ')'*100)
+        # Maximum recursion depth exceeded in C to Python translator
+        raises(ValueError, gettext.c2py, '('*10000 + 'n' + ')'*10000)
+        self.assertEqual(gettext.c2py('('*20 + 'n' + ')'*20)(1), 1)
+
+    def test_chained_comparison(self):
+        # C doesn't chain comparison as Python so 2 == 2 == 2 gets different results
+        f = gettext.c2py('n == n == n')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '010')
+        f = gettext.c2py('1 < n == n')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '100')
+        f = gettext.c2py('n == n < 2')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '010')
+        f = gettext.c2py('0 < n < 2')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '111')
+
+    def test_decimal_number(self):
+        self.assertEqual(gettext.c2py('0123')(1), 123)
+
+    def test_invalid_syntax(self):
+        invalid_expressions = [
+            'x>1', '(n>1', 'n>1)', '42**42**42', '0xa', '1.0', '1e2',
+            'n>0x1', '+n', '-n', 'n()', 'n(1)', '1+', 'nn', 'n n',
+        ]
+        for expr in invalid_expressions:
+            with self.assertRaises(ValueError):
+                gettext.c2py(expr)
+
+    def test_nested_condition_operator(self):
+        self.assertEqual(gettext.c2py('n?1?2:3:4')(0), 4)
+        self.assertEqual(gettext.c2py('n?1?2:3:4')(1), 2)
+        self.assertEqual(gettext.c2py('n?1:3?4:5')(0), 4)
+        self.assertEqual(gettext.c2py('n?1:3?4:5')(1), 1)
+
+    def test_division(self):
+        f = gettext.c2py('2/n*3')
+        self.assertEqual(f(1), 6)
+        self.assertEqual(f(2), 3)
+        self.assertEqual(f(3), 0)
+        self.assertEqual(f(-1), -6)
+        self.assertRaises(ZeroDivisionError, f, 0)
+
+    def test_plural_number(self):
+        f = gettext.c2py('1')
+        self.assertEqual(f(1), 1)
+        self.assertRaises(ValueError, f, 1.0)
+        self.assertRaises(ValueError, f, '1')
+        self.assertRaises(ValueError, f, [])
+        self.assertRaises(ValueError, f, object())
+
 
 class GNUTranslationParsingTest(GettextBaseTest):
     def test_plural_form_error_issue17898(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -60,6 +60,10 @@
 Library
 -------
 
+- Issue #28563: Fixed possible DoS and arbitrary code execution when handle
+  plural form selections in the gettext module.  The expression parser now
+  supports exact syntax supported by GNU gettext.
+
 - Issue #28387: Fixed possible crash in _io.TextIOWrapper deallocator when
   the garbage collector is invoked in other thread.  Based on patch by
   Sebastian Cufre.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 14:28:53 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 08 Nov 2016 19:28:53 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzI4NTYz?=
 =?utf-8?q?=3A_Fixed_possible_DoS_and_arbitrary_code_execution_when_handle?=
Message-ID: <20161108192852.96061.57690.3CA5D6DB@psf.io>

https://hg.python.org/cpython/rev/7e66c5dc4218
changeset:   104977:7e66c5dc4218
branch:      3.3
parent:      103779:5ae8756a1ae0
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 08 21:17:46 2016 +0200
summary:
  Issue #28563: Fixed possible DoS and arbitrary code execution when handle
plural form selections in the gettext module.  The expression parser now
supports exact syntax supported by GNU gettext.

files:
  Lib/gettext.py           |  168 ++++++++++++++++++++------
  Lib/test/test_gettext.py |   85 +++++++++++++-
  Misc/NEWS                |    4 +
  3 files changed, 214 insertions(+), 43 deletions(-)


diff --git a/Lib/gettext.py b/Lib/gettext.py
--- a/Lib/gettext.py
+++ b/Lib/gettext.py
@@ -57,55 +57,139 @@
 
 _default_localedir = os.path.join(sys.base_prefix, 'share', 'locale')
 
+# Expression parsing for plural form selection.
+#
+# The gettext library supports a small subset of C syntax.  The only
+# incompatible difference is that integer literals starting with zero are
+# decimal.
+#
+# https://www.gnu.org/software/gettext/manual/gettext.html#Plural-forms
+# http://git.savannah.gnu.org/cgit/gettext.git/tree/gettext-runtime/intl/plural.y
+
+_token_pattern = re.compile(r"""
+        (?P<WHITESPACES>[ \t]+)                    | # spaces and horizontal tabs
+        (?P<NUMBER>[0-9]+\b)                       | # decimal integer
+        (?P<NAME>n\b)                              | # only n is allowed
+        (?P<PARENTHESIS>[()])                      |
+        (?P<OPERATOR>[-*/%+?:]|[><!]=?|==|&&|\|\|) | # !, *, /, %, +, -, <, >,
+                                                     # <=, >=, ==, !=, &&, ||,
+                                                     # ? :
+                                                     # unary and bitwise ops
+                                                     # not allowed
+        (?P<INVALID>\w+|.)                           # invalid token
+    """, re.VERBOSE|re.DOTALL)
+
+def _tokenize(plural):
+    for mo in re.finditer(_token_pattern, plural):
+        kind = mo.lastgroup
+        if kind == 'WHITESPACES':
+            continue
+        value = mo.group(kind)
+        if kind == 'INVALID':
+            raise ValueError('invalid token in plural form: %s' % value)
+        yield value
+    yield ''
+
+def _error(value):
+    if value:
+        return ValueError('unexpected token in plural form: %s' % value)
+    else:
+        return ValueError('unexpected end of plural form')
+
+_binary_ops = (
+    ('||',),
+    ('&&',),
+    ('==', '!='),
+    ('<', '>', '<=', '>='),
+    ('+', '-'),
+    ('*', '/', '%'),
+)
+_binary_ops = {op: i for i, ops in enumerate(_binary_ops, 1) for op in ops}
+_c2py_ops = {'||': 'or', '&&': 'and', '/': '//'}
+
+def _parse(tokens, priority=-1):
+    result = ''
+    nexttok = next(tokens)
+    while nexttok == '!':
+        result += 'not '
+        nexttok = next(tokens)
+
+    if nexttok == '(':
+        sub, nexttok = _parse(tokens)
+        result = '%s(%s)' % (result, sub)
+        if nexttok != ')':
+            raise ValueError('unbalanced parenthesis in plural form')
+    elif nexttok == 'n':
+        result = '%s%s' % (result, nexttok)
+    else:
+        try:
+            value = int(nexttok, 10)
+        except ValueError:
+            raise _error(nexttok) from None
+        result = '%s%d' % (result, value)
+    nexttok = next(tokens)
+
+    j = 100
+    while nexttok in _binary_ops:
+        i = _binary_ops[nexttok]
+        if i < priority:
+            break
+        # Break chained comparisons
+        if i in (3, 4) and j in (3, 4):  # '==', '!=', '<', '>', '<=', '>='
+            result = '(%s)' % result
+        # Replace some C operators by their Python equivalents
+        op = _c2py_ops.get(nexttok, nexttok)
+        right, nexttok = _parse(tokens, i + 1)
+        result = '%s %s %s' % (result, op, right)
+        j = i
+    if j == priority == 4:  # '<', '>', '<=', '>='
+        result = '(%s)' % result
+
+    if nexttok == '?' and priority <= 0:
+        if_true, nexttok = _parse(tokens, 0)
+        if nexttok != ':':
+            raise _error(nexttok)
+        if_false, nexttok = _parse(tokens)
+        result = '%s if %s else %s' % (if_true, result, if_false)
+        if priority == 0:
+            result = '(%s)' % result
+
+    return result, nexttok
 
 def c2py(plural):
     """Gets a C expression as used in PO files for plural forms and returns a
-    Python lambda function that implements an equivalent expression.
+    Python function that implements an equivalent expression.
     """
-    # Security check, allow only the "n" identifier
-    import token, tokenize
-    tokens = tokenize.generate_tokens(io.StringIO(plural).readline)
+
+    if len(plural) > 1000:
+        raise ValueError('plural form expression is too long')
     try:
-        danger = [x for x in tokens if x[0] == token.NAME and x[1] != 'n']
-    except tokenize.TokenError:
-        raise ValueError('plural forms expression error, maybe unbalanced parenthesis')
-    else:
-        if danger:
-            raise ValueError('plural forms expression could be dangerous')
+        result, nexttok = _parse(_tokenize(plural))
+        if nexttok:
+            raise _error(nexttok)
 
-    # Replace some C operators by their Python equivalents
-    plural = plural.replace('&&', ' and ')
-    plural = plural.replace('||', ' or ')
+        depth = 0
+        for c in result:
+            if c == '(':
+                depth += 1
+                if depth > 20:
+                    # Python compiler limit is about 90.
+                    # The most complex example has 2.
+                    raise ValueError('plural form expression is too complex')
+            elif c == ')':
+                depth -= 1
 
-    expr = re.compile(r'\!([^=])')
-    plural = expr.sub(' not \\1', plural)
-
-    # Regular expression and replacement function used to transform
-    # "a?b:c" to "b if a else c".
-    expr = re.compile(r'(.*?)\?(.*?):(.*)')
-    def repl(x):
-        return "(%s if %s else %s)" % (x.group(2), x.group(1),
-                                       expr.sub(repl, x.group(3)))
-
-    # Code to transform the plural expression, taking care of parentheses
-    stack = ['']
-    for c in plural:
-        if c == '(':
-            stack.append('')
-        elif c == ')':
-            if len(stack) == 1:
-                # Actually, we never reach this code, because unbalanced
-                # parentheses get caught in the security check at the
-                # beginning.
-                raise ValueError('unbalanced parenthesis in plural form')
-            s = expr.sub(repl, stack.pop())
-            stack[-1] += '(%s)' % s
-        else:
-            stack[-1] += c
-    plural = expr.sub(repl, stack.pop())
-
-    return eval('lambda n: int(%s)' % plural)
-
+        ns = {}
+        exec('''if True:
+            def func(n):
+                if not isinstance(n, int):
+                    raise ValueError('Plural value must be an integer.')
+                return int(%s)
+            ''' % result, ns)
+        return ns['func']
+    except RuntimeError:
+        # Recursion error can be raised in _parse() or exec().
+        raise ValueError('plural form expression is too complex')
 
 
 def _expand_lang(loc):
diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py
--- a/Lib/test/test_gettext.py
+++ b/Lib/test/test_gettext.py
@@ -230,7 +230,9 @@
         x = t.ngettext('There is %s file', 'There are %s files', 2)
         eq(x, 'Hay %s ficheros')
 
-    def test_hu(self):
+    # Examples from http://www.gnu.org/software/gettext/manual/gettext.html
+
+    def test_ja(self):
         eq = self.assertEqual
         f = gettext.c2py('0')
         s = ''.join([ str(f(x)) for x in range(200) ])
@@ -248,6 +250,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "00111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111")
 
+    def test_lv(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n != 0 ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "20111111111111111111101111111110111111111011111111101111111110111111111011111111101111111110111111111011111111111111111110111111111011111111101111111110111111111011111111101111111110111111111011111111")
+
     def test_gd(self):
         eq = self.assertEqual
         f = gettext.c2py('n==1 ? 0 : n==2 ? 1 : 2')
@@ -261,6 +269,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "20122222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222")
 
+    def test_ro(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n==1 ? 0 : (n==0 || (n%100 > 0 && n%100 < 20)) ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "10111111111111111111222222222222222222222222222222222222222222222222222222222222222222222222222222222111111111111111111122222222222222222222222222222222222222222222222222222222222222222222222222222222")
+
     def test_lt(self):
         eq = self.assertEqual
         f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n%10>=2 && (n%100<10 || n%100>=20) ? 1 : 2')
@@ -273,6 +287,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "20111222222222222222201112222220111222222011122222201112222220111222222011122222201112222220111222222011122222222222222220111222222011122222201112222220111222222011122222201112222220111222222011122222")
 
+    def test_cs(self):
+        eq = self.assertEqual
+        f = gettext.c2py('(n==1) ? 0 : (n>=2 && n<=4) ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "20111222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222")
+
     def test_pl(self):
         eq = self.assertEqual
         f = gettext.c2py('n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2')
@@ -285,10 +305,73 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "30122333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333012233333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333")
 
+    def test_ar(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n==0 ? 0 : n==1 ? 1 : n==2 ? 2 : n%100>=3 && n%100<=10 ? 3 : n%100>=11 ? 4 : 5')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "01233333333444444444444444444444444444444444444444444444444444444444444444444444444444444444444444445553333333344444444444444444444444444444444444444444444444444444444444444444444444444444444444444444")
+
     def test_security(self):
         raises = self.assertRaises
         # Test for a dangerous expression
         raises(ValueError, gettext.c2py, "os.chmod('/etc/passwd',0777)")
+        # issue28563
+        raises(ValueError, gettext.c2py, '"(eval(foo) && ""')
+        raises(ValueError, gettext.c2py, 'f"{os.system(\'sh\')}"')
+        # Maximum recursion depth exceeded during compilation
+        raises(ValueError, gettext.c2py, 'n+'*10000 + 'n')
+        self.assertEqual(gettext.c2py('n+'*100 + 'n')(1), 101)
+        # MemoryError during compilation
+        raises(ValueError, gettext.c2py, '('*100 + 'n' + ')'*100)
+        # Maximum recursion depth exceeded in C to Python translator
+        raises(ValueError, gettext.c2py, '('*10000 + 'n' + ')'*10000)
+        self.assertEqual(gettext.c2py('('*20 + 'n' + ')'*20)(1), 1)
+
+    def test_chained_comparison(self):
+        # C doesn't chain comparison as Python so 2 == 2 == 2 gets different results
+        f = gettext.c2py('n == n == n')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '010')
+        f = gettext.c2py('1 < n == n')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '100')
+        f = gettext.c2py('n == n < 2')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '010')
+        f = gettext.c2py('0 < n < 2')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '111')
+
+    def test_decimal_number(self):
+        self.assertEqual(gettext.c2py('0123')(1), 123)
+
+    def test_invalid_syntax(self):
+        invalid_expressions = [
+            'x>1', '(n>1', 'n>1)', '42**42**42', '0xa', '1.0', '1e2',
+            'n>0x1', '+n', '-n', 'n()', 'n(1)', '1+', 'nn', 'n n',
+        ]
+        for expr in invalid_expressions:
+            with self.assertRaises(ValueError):
+                gettext.c2py(expr)
+
+    def test_nested_condition_operator(self):
+        self.assertEqual(gettext.c2py('n?1?2:3:4')(0), 4)
+        self.assertEqual(gettext.c2py('n?1?2:3:4')(1), 2)
+        self.assertEqual(gettext.c2py('n?1:3?4:5')(0), 4)
+        self.assertEqual(gettext.c2py('n?1:3?4:5')(1), 1)
+
+    def test_division(self):
+        f = gettext.c2py('2/n*3')
+        self.assertEqual(f(1), 6)
+        self.assertEqual(f(2), 3)
+        self.assertEqual(f(3), 0)
+        self.assertEqual(f(-1), -6)
+        self.assertRaises(ZeroDivisionError, f, 0)
+
+    def test_plural_number(self):
+        f = gettext.c2py('1')
+        self.assertEqual(f(1), 1)
+        self.assertRaises(ValueError, f, 1.0)
+        self.assertRaises(ValueError, f, '1')
+        self.assertRaises(ValueError, f, [])
+        self.assertRaises(ValueError, f, object())
+
 
 
 class UnicodeTranslationsTest(GettextBaseTest):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -32,6 +32,10 @@
 Library
 -------
 
+- Issue #28563: Fixed possible DoS and arbitrary code execution when handle
+  plural form selections in the gettext module.  The expression parser now
+  supports exact syntax supported by GNU gettext.
+
 - Issue #27783: Fix possible usage of uninitialized memory in operator.methodcaller.
 
 - Issue #27774: Fix possible Py_DECREF on unowned object in _sre.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 14:28:53 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 08 Nov 2016 19:28:53 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?=
 =?utf-8?q?_Issue_=2328563=3A_Fixed_possible_DoS_and_arbitrary_code_execut?=
 =?utf-8?q?ion_when_handle?=
Message-ID: <20161108192852.1955.1004.F63B8DF8@psf.io>

https://hg.python.org/cpython/rev/730cf8cdf02c
changeset:   104978:730cf8cdf02c
branch:      3.4
parent:      104878:fc4d1fb3f795
parent:      104977:7e66c5dc4218
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 08 21:20:09 2016 +0200
summary:
  Issue #28563: Fixed possible DoS and arbitrary code execution when handle
plural form selections in the gettext module.  The expression parser now
supports exact syntax supported by GNU gettext.

files:
  Lib/gettext.py           |  168 ++++++++++++++++++++------
  Lib/test/test_gettext.py |   85 +++++++++++++-
  Misc/NEWS                |    4 +
  3 files changed, 214 insertions(+), 43 deletions(-)


diff --git a/Lib/gettext.py b/Lib/gettext.py
--- a/Lib/gettext.py
+++ b/Lib/gettext.py
@@ -59,55 +59,139 @@
 
 _default_localedir = os.path.join(sys.base_prefix, 'share', 'locale')
 
+# Expression parsing for plural form selection.
+#
+# The gettext library supports a small subset of C syntax.  The only
+# incompatible difference is that integer literals starting with zero are
+# decimal.
+#
+# https://www.gnu.org/software/gettext/manual/gettext.html#Plural-forms
+# http://git.savannah.gnu.org/cgit/gettext.git/tree/gettext-runtime/intl/plural.y
+
+_token_pattern = re.compile(r"""
+        (?P<WHITESPACES>[ \t]+)                    | # spaces and horizontal tabs
+        (?P<NUMBER>[0-9]+\b)                       | # decimal integer
+        (?P<NAME>n\b)                              | # only n is allowed
+        (?P<PARENTHESIS>[()])                      |
+        (?P<OPERATOR>[-*/%+?:]|[><!]=?|==|&&|\|\|) | # !, *, /, %, +, -, <, >,
+                                                     # <=, >=, ==, !=, &&, ||,
+                                                     # ? :
+                                                     # unary and bitwise ops
+                                                     # not allowed
+        (?P<INVALID>\w+|.)                           # invalid token
+    """, re.VERBOSE|re.DOTALL)
+
+def _tokenize(plural):
+    for mo in re.finditer(_token_pattern, plural):
+        kind = mo.lastgroup
+        if kind == 'WHITESPACES':
+            continue
+        value = mo.group(kind)
+        if kind == 'INVALID':
+            raise ValueError('invalid token in plural form: %s' % value)
+        yield value
+    yield ''
+
+def _error(value):
+    if value:
+        return ValueError('unexpected token in plural form: %s' % value)
+    else:
+        return ValueError('unexpected end of plural form')
+
+_binary_ops = (
+    ('||',),
+    ('&&',),
+    ('==', '!='),
+    ('<', '>', '<=', '>='),
+    ('+', '-'),
+    ('*', '/', '%'),
+)
+_binary_ops = {op: i for i, ops in enumerate(_binary_ops, 1) for op in ops}
+_c2py_ops = {'||': 'or', '&&': 'and', '/': '//'}
+
+def _parse(tokens, priority=-1):
+    result = ''
+    nexttok = next(tokens)
+    while nexttok == '!':
+        result += 'not '
+        nexttok = next(tokens)
+
+    if nexttok == '(':
+        sub, nexttok = _parse(tokens)
+        result = '%s(%s)' % (result, sub)
+        if nexttok != ')':
+            raise ValueError('unbalanced parenthesis in plural form')
+    elif nexttok == 'n':
+        result = '%s%s' % (result, nexttok)
+    else:
+        try:
+            value = int(nexttok, 10)
+        except ValueError:
+            raise _error(nexttok) from None
+        result = '%s%d' % (result, value)
+    nexttok = next(tokens)
+
+    j = 100
+    while nexttok in _binary_ops:
+        i = _binary_ops[nexttok]
+        if i < priority:
+            break
+        # Break chained comparisons
+        if i in (3, 4) and j in (3, 4):  # '==', '!=', '<', '>', '<=', '>='
+            result = '(%s)' % result
+        # Replace some C operators by their Python equivalents
+        op = _c2py_ops.get(nexttok, nexttok)
+        right, nexttok = _parse(tokens, i + 1)
+        result = '%s %s %s' % (result, op, right)
+        j = i
+    if j == priority == 4:  # '<', '>', '<=', '>='
+        result = '(%s)' % result
+
+    if nexttok == '?' and priority <= 0:
+        if_true, nexttok = _parse(tokens, 0)
+        if nexttok != ':':
+            raise _error(nexttok)
+        if_false, nexttok = _parse(tokens)
+        result = '%s if %s else %s' % (if_true, result, if_false)
+        if priority == 0:
+            result = '(%s)' % result
+
+    return result, nexttok
 
 def c2py(plural):
     """Gets a C expression as used in PO files for plural forms and returns a
-    Python lambda function that implements an equivalent expression.
+    Python function that implements an equivalent expression.
     """
-    # Security check, allow only the "n" identifier
-    import token, tokenize
-    tokens = tokenize.generate_tokens(io.StringIO(plural).readline)
+
+    if len(plural) > 1000:
+        raise ValueError('plural form expression is too long')
     try:
-        danger = [x for x in tokens if x[0] == token.NAME and x[1] != 'n']
-    except tokenize.TokenError:
-        raise ValueError('plural forms expression error, maybe unbalanced parenthesis')
-    else:
-        if danger:
-            raise ValueError('plural forms expression could be dangerous')
+        result, nexttok = _parse(_tokenize(plural))
+        if nexttok:
+            raise _error(nexttok)
 
-    # Replace some C operators by their Python equivalents
-    plural = plural.replace('&&', ' and ')
-    plural = plural.replace('||', ' or ')
+        depth = 0
+        for c in result:
+            if c == '(':
+                depth += 1
+                if depth > 20:
+                    # Python compiler limit is about 90.
+                    # The most complex example has 2.
+                    raise ValueError('plural form expression is too complex')
+            elif c == ')':
+                depth -= 1
 
-    expr = re.compile(r'\!([^=])')
-    plural = expr.sub(' not \\1', plural)
-
-    # Regular expression and replacement function used to transform
-    # "a?b:c" to "b if a else c".
-    expr = re.compile(r'(.*?)\?(.*?):(.*)')
-    def repl(x):
-        return "(%s if %s else %s)" % (x.group(2), x.group(1),
-                                       expr.sub(repl, x.group(3)))
-
-    # Code to transform the plural expression, taking care of parentheses
-    stack = ['']
-    for c in plural:
-        if c == '(':
-            stack.append('')
-        elif c == ')':
-            if len(stack) == 1:
-                # Actually, we never reach this code, because unbalanced
-                # parentheses get caught in the security check at the
-                # beginning.
-                raise ValueError('unbalanced parenthesis in plural form')
-            s = expr.sub(repl, stack.pop())
-            stack[-1] += '(%s)' % s
-        else:
-            stack[-1] += c
-    plural = expr.sub(repl, stack.pop())
-
-    return eval('lambda n: int(%s)' % plural)
-
+        ns = {}
+        exec('''if True:
+            def func(n):
+                if not isinstance(n, int):
+                    raise ValueError('Plural value must be an integer.')
+                return int(%s)
+            ''' % result, ns)
+        return ns['func']
+    except RuntimeError:
+        # Recursion error can be raised in _parse() or exec().
+        raise ValueError('plural form expression is too complex')
 
 
 def _expand_lang(loc):
diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py
--- a/Lib/test/test_gettext.py
+++ b/Lib/test/test_gettext.py
@@ -236,7 +236,9 @@
         x = t.ngettext('There is %s file', 'There are %s files', 2)
         eq(x, 'Hay %s ficheros')
 
-    def test_hu(self):
+    # Examples from http://www.gnu.org/software/gettext/manual/gettext.html
+
+    def test_ja(self):
         eq = self.assertEqual
         f = gettext.c2py('0')
         s = ''.join([ str(f(x)) for x in range(200) ])
@@ -254,6 +256,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "00111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111")
 
+    def test_lv(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n != 0 ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "20111111111111111111101111111110111111111011111111101111111110111111111011111111101111111110111111111011111111111111111110111111111011111111101111111110111111111011111111101111111110111111111011111111")
+
     def test_gd(self):
         eq = self.assertEqual
         f = gettext.c2py('n==1 ? 0 : n==2 ? 1 : 2')
@@ -267,6 +275,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "20122222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222")
 
+    def test_ro(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n==1 ? 0 : (n==0 || (n%100 > 0 && n%100 < 20)) ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "10111111111111111111222222222222222222222222222222222222222222222222222222222222222222222222222222222111111111111111111122222222222222222222222222222222222222222222222222222222222222222222222222222222")
+
     def test_lt(self):
         eq = self.assertEqual
         f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n%10>=2 && (n%100<10 || n%100>=20) ? 1 : 2')
@@ -279,6 +293,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "20111222222222222222201112222220111222222011122222201112222220111222222011122222201112222220111222222011122222222222222220111222222011122222201112222220111222222011122222201112222220111222222011122222")
 
+    def test_cs(self):
+        eq = self.assertEqual
+        f = gettext.c2py('(n==1) ? 0 : (n>=2 && n<=4) ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "20111222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222")
+
     def test_pl(self):
         eq = self.assertEqual
         f = gettext.c2py('n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2')
@@ -291,10 +311,73 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "30122333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333012233333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333")
 
+    def test_ar(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n==0 ? 0 : n==1 ? 1 : n==2 ? 2 : n%100>=3 && n%100<=10 ? 3 : n%100>=11 ? 4 : 5')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "01233333333444444444444444444444444444444444444444444444444444444444444444444444444444444444444444445553333333344444444444444444444444444444444444444444444444444444444444444444444444444444444444444444")
+
     def test_security(self):
         raises = self.assertRaises
         # Test for a dangerous expression
         raises(ValueError, gettext.c2py, "os.chmod('/etc/passwd',0777)")
+        # issue28563
+        raises(ValueError, gettext.c2py, '"(eval(foo) && ""')
+        raises(ValueError, gettext.c2py, 'f"{os.system(\'sh\')}"')
+        # Maximum recursion depth exceeded during compilation
+        raises(ValueError, gettext.c2py, 'n+'*10000 + 'n')
+        self.assertEqual(gettext.c2py('n+'*100 + 'n')(1), 101)
+        # MemoryError during compilation
+        raises(ValueError, gettext.c2py, '('*100 + 'n' + ')'*100)
+        # Maximum recursion depth exceeded in C to Python translator
+        raises(ValueError, gettext.c2py, '('*10000 + 'n' + ')'*10000)
+        self.assertEqual(gettext.c2py('('*20 + 'n' + ')'*20)(1), 1)
+
+    def test_chained_comparison(self):
+        # C doesn't chain comparison as Python so 2 == 2 == 2 gets different results
+        f = gettext.c2py('n == n == n')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '010')
+        f = gettext.c2py('1 < n == n')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '100')
+        f = gettext.c2py('n == n < 2')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '010')
+        f = gettext.c2py('0 < n < 2')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '111')
+
+    def test_decimal_number(self):
+        self.assertEqual(gettext.c2py('0123')(1), 123)
+
+    def test_invalid_syntax(self):
+        invalid_expressions = [
+            'x>1', '(n>1', 'n>1)', '42**42**42', '0xa', '1.0', '1e2',
+            'n>0x1', '+n', '-n', 'n()', 'n(1)', '1+', 'nn', 'n n',
+        ]
+        for expr in invalid_expressions:
+            with self.assertRaises(ValueError):
+                gettext.c2py(expr)
+
+    def test_nested_condition_operator(self):
+        self.assertEqual(gettext.c2py('n?1?2:3:4')(0), 4)
+        self.assertEqual(gettext.c2py('n?1?2:3:4')(1), 2)
+        self.assertEqual(gettext.c2py('n?1:3?4:5')(0), 4)
+        self.assertEqual(gettext.c2py('n?1:3?4:5')(1), 1)
+
+    def test_division(self):
+        f = gettext.c2py('2/n*3')
+        self.assertEqual(f(1), 6)
+        self.assertEqual(f(2), 3)
+        self.assertEqual(f(3), 0)
+        self.assertEqual(f(-1), -6)
+        self.assertRaises(ZeroDivisionError, f, 0)
+
+    def test_plural_number(self):
+        f = gettext.c2py('1')
+        self.assertEqual(f(1), 1)
+        self.assertRaises(ValueError, f, 1.0)
+        self.assertRaises(ValueError, f, '1')
+        self.assertRaises(ValueError, f, [])
+        self.assertRaises(ValueError, f, object())
+
 
 class GNUTranslationParsingTest(GettextBaseTest):
     def test_plural_form_error_issue17898(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -16,6 +16,10 @@
 Library
 -------
 
+- Issue #28563: Fixed possible DoS and arbitrary code execution when handle
+  plural form selections in the gettext module.  The expression parser now
+  supports exact syntax supported by GNU gettext.
+
 - In the curses module, raise an error if window.getstr() or window.instr() is
   passed a negative value.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 14:28:53 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 08 Nov 2016 19:28:53 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?=
 =?utf-8?q?_Issue_=2328563=3A_Fixed_possible_DoS_and_arbitrary_code_execut?=
 =?utf-8?q?ion_when_handle?=
Message-ID: <20161108192852.2214.82115.30749291@psf.io>

https://hg.python.org/cpython/rev/e0dd4cc9661a
changeset:   104979:e0dd4cc9661a
branch:      3.5
parent:      104972:5b4fa92dac43
parent:      104978:730cf8cdf02c
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 08 21:26:14 2016 +0200
summary:
  Issue #28563: Fixed possible DoS and arbitrary code execution when handle
plural form selections in the gettext module.  The expression parser now
supports exact syntax supported by GNU gettext.

files:
  Lib/gettext.py           |  168 ++++++++++++++++++++------
  Lib/test/test_gettext.py |   85 +++++++++++++-
  2 files changed, 210 insertions(+), 43 deletions(-)


diff --git a/Lib/gettext.py b/Lib/gettext.py
--- a/Lib/gettext.py
+++ b/Lib/gettext.py
@@ -59,55 +59,139 @@
 
 _default_localedir = os.path.join(sys.base_prefix, 'share', 'locale')
 
+# Expression parsing for plural form selection.
+#
+# The gettext library supports a small subset of C syntax.  The only
+# incompatible difference is that integer literals starting with zero are
+# decimal.
+#
+# https://www.gnu.org/software/gettext/manual/gettext.html#Plural-forms
+# http://git.savannah.gnu.org/cgit/gettext.git/tree/gettext-runtime/intl/plural.y
+
+_token_pattern = re.compile(r"""
+        (?P<WHITESPACES>[ \t]+)                    | # spaces and horizontal tabs
+        (?P<NUMBER>[0-9]+\b)                       | # decimal integer
+        (?P<NAME>n\b)                              | # only n is allowed
+        (?P<PARENTHESIS>[()])                      |
+        (?P<OPERATOR>[-*/%+?:]|[><!]=?|==|&&|\|\|) | # !, *, /, %, +, -, <, >,
+                                                     # <=, >=, ==, !=, &&, ||,
+                                                     # ? :
+                                                     # unary and bitwise ops
+                                                     # not allowed
+        (?P<INVALID>\w+|.)                           # invalid token
+    """, re.VERBOSE|re.DOTALL)
+
+def _tokenize(plural):
+    for mo in re.finditer(_token_pattern, plural):
+        kind = mo.lastgroup
+        if kind == 'WHITESPACES':
+            continue
+        value = mo.group(kind)
+        if kind == 'INVALID':
+            raise ValueError('invalid token in plural form: %s' % value)
+        yield value
+    yield ''
+
+def _error(value):
+    if value:
+        return ValueError('unexpected token in plural form: %s' % value)
+    else:
+        return ValueError('unexpected end of plural form')
+
+_binary_ops = (
+    ('||',),
+    ('&&',),
+    ('==', '!='),
+    ('<', '>', '<=', '>='),
+    ('+', '-'),
+    ('*', '/', '%'),
+)
+_binary_ops = {op: i for i, ops in enumerate(_binary_ops, 1) for op in ops}
+_c2py_ops = {'||': 'or', '&&': 'and', '/': '//'}
+
+def _parse(tokens, priority=-1):
+    result = ''
+    nexttok = next(tokens)
+    while nexttok == '!':
+        result += 'not '
+        nexttok = next(tokens)
+
+    if nexttok == '(':
+        sub, nexttok = _parse(tokens)
+        result = '%s(%s)' % (result, sub)
+        if nexttok != ')':
+            raise ValueError('unbalanced parenthesis in plural form')
+    elif nexttok == 'n':
+        result = '%s%s' % (result, nexttok)
+    else:
+        try:
+            value = int(nexttok, 10)
+        except ValueError:
+            raise _error(nexttok) from None
+        result = '%s%d' % (result, value)
+    nexttok = next(tokens)
+
+    j = 100
+    while nexttok in _binary_ops:
+        i = _binary_ops[nexttok]
+        if i < priority:
+            break
+        # Break chained comparisons
+        if i in (3, 4) and j in (3, 4):  # '==', '!=', '<', '>', '<=', '>='
+            result = '(%s)' % result
+        # Replace some C operators by their Python equivalents
+        op = _c2py_ops.get(nexttok, nexttok)
+        right, nexttok = _parse(tokens, i + 1)
+        result = '%s %s %s' % (result, op, right)
+        j = i
+    if j == priority == 4:  # '<', '>', '<=', '>='
+        result = '(%s)' % result
+
+    if nexttok == '?' and priority <= 0:
+        if_true, nexttok = _parse(tokens, 0)
+        if nexttok != ':':
+            raise _error(nexttok)
+        if_false, nexttok = _parse(tokens)
+        result = '%s if %s else %s' % (if_true, result, if_false)
+        if priority == 0:
+            result = '(%s)' % result
+
+    return result, nexttok
 
 def c2py(plural):
     """Gets a C expression as used in PO files for plural forms and returns a
-    Python lambda function that implements an equivalent expression.
+    Python function that implements an equivalent expression.
     """
-    # Security check, allow only the "n" identifier
-    import token, tokenize
-    tokens = tokenize.generate_tokens(io.StringIO(plural).readline)
+
+    if len(plural) > 1000:
+        raise ValueError('plural form expression is too long')
     try:
-        danger = [x for x in tokens if x[0] == token.NAME and x[1] != 'n']
-    except tokenize.TokenError:
-        raise ValueError('plural forms expression error, maybe unbalanced parenthesis')
-    else:
-        if danger:
-            raise ValueError('plural forms expression could be dangerous')
+        result, nexttok = _parse(_tokenize(plural))
+        if nexttok:
+            raise _error(nexttok)
 
-    # Replace some C operators by their Python equivalents
-    plural = plural.replace('&&', ' and ')
-    plural = plural.replace('||', ' or ')
+        depth = 0
+        for c in result:
+            if c == '(':
+                depth += 1
+                if depth > 20:
+                    # Python compiler limit is about 90.
+                    # The most complex example has 2.
+                    raise ValueError('plural form expression is too complex')
+            elif c == ')':
+                depth -= 1
 
-    expr = re.compile(r'\!([^=])')
-    plural = expr.sub(' not \\1', plural)
-
-    # Regular expression and replacement function used to transform
-    # "a?b:c" to "b if a else c".
-    expr = re.compile(r'(.*?)\?(.*?):(.*)')
-    def repl(x):
-        return "(%s if %s else %s)" % (x.group(2), x.group(1),
-                                       expr.sub(repl, x.group(3)))
-
-    # Code to transform the plural expression, taking care of parentheses
-    stack = ['']
-    for c in plural:
-        if c == '(':
-            stack.append('')
-        elif c == ')':
-            if len(stack) == 1:
-                # Actually, we never reach this code, because unbalanced
-                # parentheses get caught in the security check at the
-                # beginning.
-                raise ValueError('unbalanced parenthesis in plural form')
-            s = expr.sub(repl, stack.pop())
-            stack[-1] += '(%s)' % s
-        else:
-            stack[-1] += c
-    plural = expr.sub(repl, stack.pop())
-
-    return eval('lambda n: int(%s)' % plural)
-
+        ns = {}
+        exec('''if True:
+            def func(n):
+                if not isinstance(n, int):
+                    raise ValueError('Plural value must be an integer.')
+                return int(%s)
+            ''' % result, ns)
+        return ns['func']
+    except RecursionError:
+        # Recursion error can be raised in _parse() or exec().
+        raise ValueError('plural form expression is too complex')
 
 
 def _expand_lang(loc):
diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py
--- a/Lib/test/test_gettext.py
+++ b/Lib/test/test_gettext.py
@@ -306,7 +306,9 @@
         x = t.ngettext('There is %s file', 'There are %s files', 2)
         eq(x, 'Hay %s ficheros')
 
-    def test_hu(self):
+    # Examples from http://www.gnu.org/software/gettext/manual/gettext.html
+
+    def test_ja(self):
         eq = self.assertEqual
         f = gettext.c2py('0')
         s = ''.join([ str(f(x)) for x in range(200) ])
@@ -324,6 +326,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "00111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111")
 
+    def test_lv(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n != 0 ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "20111111111111111111101111111110111111111011111111101111111110111111111011111111101111111110111111111011111111111111111110111111111011111111101111111110111111111011111111101111111110111111111011111111")
+
     def test_gd(self):
         eq = self.assertEqual
         f = gettext.c2py('n==1 ? 0 : n==2 ? 1 : 2')
@@ -337,6 +345,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "20122222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222")
 
+    def test_ro(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n==1 ? 0 : (n==0 || (n%100 > 0 && n%100 < 20)) ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "10111111111111111111222222222222222222222222222222222222222222222222222222222222222222222222222222222111111111111111111122222222222222222222222222222222222222222222222222222222222222222222222222222222")
+
     def test_lt(self):
         eq = self.assertEqual
         f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n%10>=2 && (n%100<10 || n%100>=20) ? 1 : 2')
@@ -349,6 +363,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "20111222222222222222201112222220111222222011122222201112222220111222222011122222201112222220111222222011122222222222222220111222222011122222201112222220111222222011122222201112222220111222222011122222")
 
+    def test_cs(self):
+        eq = self.assertEqual
+        f = gettext.c2py('(n==1) ? 0 : (n>=2 && n<=4) ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "20111222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222")
+
     def test_pl(self):
         eq = self.assertEqual
         f = gettext.c2py('n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2')
@@ -361,10 +381,73 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "30122333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333012233333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333")
 
+    def test_ar(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n==0 ? 0 : n==1 ? 1 : n==2 ? 2 : n%100>=3 && n%100<=10 ? 3 : n%100>=11 ? 4 : 5')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "01233333333444444444444444444444444444444444444444444444444444444444444444444444444444444444444444445553333333344444444444444444444444444444444444444444444444444444444444444444444444444444444444444444")
+
     def test_security(self):
         raises = self.assertRaises
         # Test for a dangerous expression
         raises(ValueError, gettext.c2py, "os.chmod('/etc/passwd',0777)")
+        # issue28563
+        raises(ValueError, gettext.c2py, '"(eval(foo) && ""')
+        raises(ValueError, gettext.c2py, 'f"{os.system(\'sh\')}"')
+        # Maximum recursion depth exceeded during compilation
+        raises(ValueError, gettext.c2py, 'n+'*10000 + 'n')
+        self.assertEqual(gettext.c2py('n+'*100 + 'n')(1), 101)
+        # MemoryError during compilation
+        raises(ValueError, gettext.c2py, '('*100 + 'n' + ')'*100)
+        # Maximum recursion depth exceeded in C to Python translator
+        raises(ValueError, gettext.c2py, '('*10000 + 'n' + ')'*10000)
+        self.assertEqual(gettext.c2py('('*20 + 'n' + ')'*20)(1), 1)
+
+    def test_chained_comparison(self):
+        # C doesn't chain comparison as Python so 2 == 2 == 2 gets different results
+        f = gettext.c2py('n == n == n')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '010')
+        f = gettext.c2py('1 < n == n')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '100')
+        f = gettext.c2py('n == n < 2')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '010')
+        f = gettext.c2py('0 < n < 2')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '111')
+
+    def test_decimal_number(self):
+        self.assertEqual(gettext.c2py('0123')(1), 123)
+
+    def test_invalid_syntax(self):
+        invalid_expressions = [
+            'x>1', '(n>1', 'n>1)', '42**42**42', '0xa', '1.0', '1e2',
+            'n>0x1', '+n', '-n', 'n()', 'n(1)', '1+', 'nn', 'n n',
+        ]
+        for expr in invalid_expressions:
+            with self.assertRaises(ValueError):
+                gettext.c2py(expr)
+
+    def test_nested_condition_operator(self):
+        self.assertEqual(gettext.c2py('n?1?2:3:4')(0), 4)
+        self.assertEqual(gettext.c2py('n?1?2:3:4')(1), 2)
+        self.assertEqual(gettext.c2py('n?1:3?4:5')(0), 4)
+        self.assertEqual(gettext.c2py('n?1:3?4:5')(1), 1)
+
+    def test_division(self):
+        f = gettext.c2py('2/n*3')
+        self.assertEqual(f(1), 6)
+        self.assertEqual(f(2), 3)
+        self.assertEqual(f(3), 0)
+        self.assertEqual(f(-1), -6)
+        self.assertRaises(ZeroDivisionError, f, 0)
+
+    def test_plural_number(self):
+        f = gettext.c2py('1')
+        self.assertEqual(f(1), 1)
+        self.assertRaises(ValueError, f, 1.0)
+        self.assertRaises(ValueError, f, '1')
+        self.assertRaises(ValueError, f, [])
+        self.assertRaises(ValueError, f, object())
+
 
 class GNUTranslationParsingTest(GettextBaseTest):
     def test_plural_form_error_issue17898(self):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 14:28:53 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 08 Nov 2016 19:28:53 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328563=3A_Fixed_possible_DoS_and_arbitrary_code_execut?=
 =?utf-8?q?ion_when_handle?=
Message-ID: <20161108192853.50112.66015.B8EDE675@psf.io>

https://hg.python.org/cpython/rev/7ea64ab9a5c8
changeset:   104980:7ea64ab9a5c8
branch:      3.6
parent:      104973:3da89b1678da
parent:      104979:e0dd4cc9661a
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 08 21:27:17 2016 +0200
summary:
  Issue #28563: Fixed possible DoS and arbitrary code execution when handle
plural form selections in the gettext module.  The expression parser now
supports exact syntax supported by GNU gettext.

files:
  Lib/gettext.py           |  168 ++++++++++++++++++++------
  Lib/test/test_gettext.py |   85 +++++++++++++-
  Misc/NEWS                |    4 +
  3 files changed, 214 insertions(+), 43 deletions(-)


diff --git a/Lib/gettext.py b/Lib/gettext.py
--- a/Lib/gettext.py
+++ b/Lib/gettext.py
@@ -59,55 +59,139 @@
 
 _default_localedir = os.path.join(sys.base_prefix, 'share', 'locale')
 
+# Expression parsing for plural form selection.
+#
+# The gettext library supports a small subset of C syntax.  The only
+# incompatible difference is that integer literals starting with zero are
+# decimal.
+#
+# https://www.gnu.org/software/gettext/manual/gettext.html#Plural-forms
+# http://git.savannah.gnu.org/cgit/gettext.git/tree/gettext-runtime/intl/plural.y
+
+_token_pattern = re.compile(r"""
+        (?P<WHITESPACES>[ \t]+)                    | # spaces and horizontal tabs
+        (?P<NUMBER>[0-9]+\b)                       | # decimal integer
+        (?P<NAME>n\b)                              | # only n is allowed
+        (?P<PARENTHESIS>[()])                      |
+        (?P<OPERATOR>[-*/%+?:]|[><!]=?|==|&&|\|\|) | # !, *, /, %, +, -, <, >,
+                                                     # <=, >=, ==, !=, &&, ||,
+                                                     # ? :
+                                                     # unary and bitwise ops
+                                                     # not allowed
+        (?P<INVALID>\w+|.)                           # invalid token
+    """, re.VERBOSE|re.DOTALL)
+
+def _tokenize(plural):
+    for mo in re.finditer(_token_pattern, plural):
+        kind = mo.lastgroup
+        if kind == 'WHITESPACES':
+            continue
+        value = mo.group(kind)
+        if kind == 'INVALID':
+            raise ValueError('invalid token in plural form: %s' % value)
+        yield value
+    yield ''
+
+def _error(value):
+    if value:
+        return ValueError('unexpected token in plural form: %s' % value)
+    else:
+        return ValueError('unexpected end of plural form')
+
+_binary_ops = (
+    ('||',),
+    ('&&',),
+    ('==', '!='),
+    ('<', '>', '<=', '>='),
+    ('+', '-'),
+    ('*', '/', '%'),
+)
+_binary_ops = {op: i for i, ops in enumerate(_binary_ops, 1) for op in ops}
+_c2py_ops = {'||': 'or', '&&': 'and', '/': '//'}
+
+def _parse(tokens, priority=-1):
+    result = ''
+    nexttok = next(tokens)
+    while nexttok == '!':
+        result += 'not '
+        nexttok = next(tokens)
+
+    if nexttok == '(':
+        sub, nexttok = _parse(tokens)
+        result = '%s(%s)' % (result, sub)
+        if nexttok != ')':
+            raise ValueError('unbalanced parenthesis in plural form')
+    elif nexttok == 'n':
+        result = '%s%s' % (result, nexttok)
+    else:
+        try:
+            value = int(nexttok, 10)
+        except ValueError:
+            raise _error(nexttok) from None
+        result = '%s%d' % (result, value)
+    nexttok = next(tokens)
+
+    j = 100
+    while nexttok in _binary_ops:
+        i = _binary_ops[nexttok]
+        if i < priority:
+            break
+        # Break chained comparisons
+        if i in (3, 4) and j in (3, 4):  # '==', '!=', '<', '>', '<=', '>='
+            result = '(%s)' % result
+        # Replace some C operators by their Python equivalents
+        op = _c2py_ops.get(nexttok, nexttok)
+        right, nexttok = _parse(tokens, i + 1)
+        result = '%s %s %s' % (result, op, right)
+        j = i
+    if j == priority == 4:  # '<', '>', '<=', '>='
+        result = '(%s)' % result
+
+    if nexttok == '?' and priority <= 0:
+        if_true, nexttok = _parse(tokens, 0)
+        if nexttok != ':':
+            raise _error(nexttok)
+        if_false, nexttok = _parse(tokens)
+        result = '%s if %s else %s' % (if_true, result, if_false)
+        if priority == 0:
+            result = '(%s)' % result
+
+    return result, nexttok
 
 def c2py(plural):
     """Gets a C expression as used in PO files for plural forms and returns a
-    Python lambda function that implements an equivalent expression.
+    Python function that implements an equivalent expression.
     """
-    # Security check, allow only the "n" identifier
-    import token, tokenize
-    tokens = tokenize.generate_tokens(io.StringIO(plural).readline)
+
+    if len(plural) > 1000:
+        raise ValueError('plural form expression is too long')
     try:
-        danger = [x for x in tokens if x[0] == token.NAME and x[1] != 'n']
-    except tokenize.TokenError:
-        raise ValueError('plural forms expression error, maybe unbalanced parenthesis')
-    else:
-        if danger:
-            raise ValueError('plural forms expression could be dangerous')
+        result, nexttok = _parse(_tokenize(plural))
+        if nexttok:
+            raise _error(nexttok)
 
-    # Replace some C operators by their Python equivalents
-    plural = plural.replace('&&', ' and ')
-    plural = plural.replace('||', ' or ')
+        depth = 0
+        for c in result:
+            if c == '(':
+                depth += 1
+                if depth > 20:
+                    # Python compiler limit is about 90.
+                    # The most complex example has 2.
+                    raise ValueError('plural form expression is too complex')
+            elif c == ')':
+                depth -= 1
 
-    expr = re.compile(r'\!([^=])')
-    plural = expr.sub(' not \\1', plural)
-
-    # Regular expression and replacement function used to transform
-    # "a?b:c" to "b if a else c".
-    expr = re.compile(r'(.*?)\?(.*?):(.*)')
-    def repl(x):
-        return "(%s if %s else %s)" % (x.group(2), x.group(1),
-                                       expr.sub(repl, x.group(3)))
-
-    # Code to transform the plural expression, taking care of parentheses
-    stack = ['']
-    for c in plural:
-        if c == '(':
-            stack.append('')
-        elif c == ')':
-            if len(stack) == 1:
-                # Actually, we never reach this code, because unbalanced
-                # parentheses get caught in the security check at the
-                # beginning.
-                raise ValueError('unbalanced parenthesis in plural form')
-            s = expr.sub(repl, stack.pop())
-            stack[-1] += '(%s)' % s
-        else:
-            stack[-1] += c
-    plural = expr.sub(repl, stack.pop())
-
-    return eval('lambda n: int(%s)' % plural)
-
+        ns = {}
+        exec('''if True:
+            def func(n):
+                if not isinstance(n, int):
+                    raise ValueError('Plural value must be an integer.')
+                return int(%s)
+            ''' % result, ns)
+        return ns['func']
+    except RecursionError:
+        # Recursion error can be raised in _parse() or exec().
+        raise ValueError('plural form expression is too complex')
 
 
 def _expand_lang(loc):
diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py
--- a/Lib/test/test_gettext.py
+++ b/Lib/test/test_gettext.py
@@ -305,7 +305,9 @@
         x = t.ngettext('There is %s file', 'There are %s files', 2)
         eq(x, 'Hay %s ficheros')
 
-    def test_hu(self):
+    # Examples from http://www.gnu.org/software/gettext/manual/gettext.html
+
+    def test_ja(self):
         eq = self.assertEqual
         f = gettext.c2py('0')
         s = ''.join([ str(f(x)) for x in range(200) ])
@@ -323,6 +325,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "00111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111")
 
+    def test_lv(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n != 0 ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "20111111111111111111101111111110111111111011111111101111111110111111111011111111101111111110111111111011111111111111111110111111111011111111101111111110111111111011111111101111111110111111111011111111")
+
     def test_gd(self):
         eq = self.assertEqual
         f = gettext.c2py('n==1 ? 0 : n==2 ? 1 : 2')
@@ -336,6 +344,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "20122222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222")
 
+    def test_ro(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n==1 ? 0 : (n==0 || (n%100 > 0 && n%100 < 20)) ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "10111111111111111111222222222222222222222222222222222222222222222222222222222222222222222222222222222111111111111111111122222222222222222222222222222222222222222222222222222222222222222222222222222222")
+
     def test_lt(self):
         eq = self.assertEqual
         f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n%10>=2 && (n%100<10 || n%100>=20) ? 1 : 2')
@@ -348,6 +362,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "20111222222222222222201112222220111222222011122222201112222220111222222011122222201112222220111222222011122222222222222220111222222011122222201112222220111222222011122222201112222220111222222011122222")
 
+    def test_cs(self):
+        eq = self.assertEqual
+        f = gettext.c2py('(n==1) ? 0 : (n>=2 && n<=4) ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "20111222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222")
+
     def test_pl(self):
         eq = self.assertEqual
         f = gettext.c2py('n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2')
@@ -360,10 +380,73 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "30122333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333012233333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333")
 
+    def test_ar(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n==0 ? 0 : n==1 ? 1 : n==2 ? 2 : n%100>=3 && n%100<=10 ? 3 : n%100>=11 ? 4 : 5')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "01233333333444444444444444444444444444444444444444444444444444444444444444444444444444444444444444445553333333344444444444444444444444444444444444444444444444444444444444444444444444444444444444444444")
+
     def test_security(self):
         raises = self.assertRaises
         # Test for a dangerous expression
         raises(ValueError, gettext.c2py, "os.chmod('/etc/passwd',0777)")
+        # issue28563
+        raises(ValueError, gettext.c2py, '"(eval(foo) && ""')
+        raises(ValueError, gettext.c2py, 'f"{os.system(\'sh\')}"')
+        # Maximum recursion depth exceeded during compilation
+        raises(ValueError, gettext.c2py, 'n+'*10000 + 'n')
+        self.assertEqual(gettext.c2py('n+'*100 + 'n')(1), 101)
+        # MemoryError during compilation
+        raises(ValueError, gettext.c2py, '('*100 + 'n' + ')'*100)
+        # Maximum recursion depth exceeded in C to Python translator
+        raises(ValueError, gettext.c2py, '('*10000 + 'n' + ')'*10000)
+        self.assertEqual(gettext.c2py('('*20 + 'n' + ')'*20)(1), 1)
+
+    def test_chained_comparison(self):
+        # C doesn't chain comparison as Python so 2 == 2 == 2 gets different results
+        f = gettext.c2py('n == n == n')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '010')
+        f = gettext.c2py('1 < n == n')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '100')
+        f = gettext.c2py('n == n < 2')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '010')
+        f = gettext.c2py('0 < n < 2')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '111')
+
+    def test_decimal_number(self):
+        self.assertEqual(gettext.c2py('0123')(1), 123)
+
+    def test_invalid_syntax(self):
+        invalid_expressions = [
+            'x>1', '(n>1', 'n>1)', '42**42**42', '0xa', '1.0', '1e2',
+            'n>0x1', '+n', '-n', 'n()', 'n(1)', '1+', 'nn', 'n n',
+        ]
+        for expr in invalid_expressions:
+            with self.assertRaises(ValueError):
+                gettext.c2py(expr)
+
+    def test_nested_condition_operator(self):
+        self.assertEqual(gettext.c2py('n?1?2:3:4')(0), 4)
+        self.assertEqual(gettext.c2py('n?1?2:3:4')(1), 2)
+        self.assertEqual(gettext.c2py('n?1:3?4:5')(0), 4)
+        self.assertEqual(gettext.c2py('n?1:3?4:5')(1), 1)
+
+    def test_division(self):
+        f = gettext.c2py('2/n*3')
+        self.assertEqual(f(1), 6)
+        self.assertEqual(f(2), 3)
+        self.assertEqual(f(3), 0)
+        self.assertEqual(f(-1), -6)
+        self.assertRaises(ZeroDivisionError, f, 0)
+
+    def test_plural_number(self):
+        f = gettext.c2py('1')
+        self.assertEqual(f(1), 1)
+        self.assertRaises(ValueError, f, 1.0)
+        self.assertRaises(ValueError, f, '1')
+        self.assertRaises(ValueError, f, [])
+        self.assertRaises(ValueError, f, object())
+
 
 class GNUTranslationParsingTest(GettextBaseTest):
     def test_plural_form_error_issue17898(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -16,6 +16,10 @@
 Library
 -------
 
+- Issue #28563: Fixed possible DoS and arbitrary code execution when handle
+  plural form selections in the gettext module.  The expression parser now
+  supports exact syntax supported by GNU gettext.
+
 - Issue #28387: Fixed possible crash in _io.TextIOWrapper deallocator when
   the garbage collector is invoked in other thread.  Based on patch by
   Sebastian Cufre.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 14:28:53 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 08 Nov 2016 19:28:53 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328563=3A_Fixed_possible_DoS_and_arbitrary_code_?=
 =?utf-8?q?execution_when_handle?=
Message-ID: <20161108192853.6258.67161.32BB2AD6@psf.io>

https://hg.python.org/cpython/rev/4ca9a4f96edc
changeset:   104981:4ca9a4f96edc
parent:      104975:1940b72b0a02
parent:      104980:7ea64ab9a5c8
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 08 21:28:30 2016 +0200
summary:
  Issue #28563: Fixed possible DoS and arbitrary code execution when handle
plural form selections in the gettext module.  The expression parser now
supports exact syntax supported by GNU gettext.

files:
  Lib/gettext.py           |  168 ++++++++++++++++++++------
  Lib/test/test_gettext.py |   85 +++++++++++++-
  Misc/NEWS                |    4 +
  3 files changed, 214 insertions(+), 43 deletions(-)


diff --git a/Lib/gettext.py b/Lib/gettext.py
--- a/Lib/gettext.py
+++ b/Lib/gettext.py
@@ -59,55 +59,139 @@
 
 _default_localedir = os.path.join(sys.base_prefix, 'share', 'locale')
 
+# Expression parsing for plural form selection.
+#
+# The gettext library supports a small subset of C syntax.  The only
+# incompatible difference is that integer literals starting with zero are
+# decimal.
+#
+# https://www.gnu.org/software/gettext/manual/gettext.html#Plural-forms
+# http://git.savannah.gnu.org/cgit/gettext.git/tree/gettext-runtime/intl/plural.y
+
+_token_pattern = re.compile(r"""
+        (?P<WHITESPACES>[ \t]+)                    | # spaces and horizontal tabs
+        (?P<NUMBER>[0-9]+\b)                       | # decimal integer
+        (?P<NAME>n\b)                              | # only n is allowed
+        (?P<PARENTHESIS>[()])                      |
+        (?P<OPERATOR>[-*/%+?:]|[><!]=?|==|&&|\|\|) | # !, *, /, %, +, -, <, >,
+                                                     # <=, >=, ==, !=, &&, ||,
+                                                     # ? :
+                                                     # unary and bitwise ops
+                                                     # not allowed
+        (?P<INVALID>\w+|.)                           # invalid token
+    """, re.VERBOSE|re.DOTALL)
+
+def _tokenize(plural):
+    for mo in re.finditer(_token_pattern, plural):
+        kind = mo.lastgroup
+        if kind == 'WHITESPACES':
+            continue
+        value = mo.group(kind)
+        if kind == 'INVALID':
+            raise ValueError('invalid token in plural form: %s' % value)
+        yield value
+    yield ''
+
+def _error(value):
+    if value:
+        return ValueError('unexpected token in plural form: %s' % value)
+    else:
+        return ValueError('unexpected end of plural form')
+
+_binary_ops = (
+    ('||',),
+    ('&&',),
+    ('==', '!='),
+    ('<', '>', '<=', '>='),
+    ('+', '-'),
+    ('*', '/', '%'),
+)
+_binary_ops = {op: i for i, ops in enumerate(_binary_ops, 1) for op in ops}
+_c2py_ops = {'||': 'or', '&&': 'and', '/': '//'}
+
+def _parse(tokens, priority=-1):
+    result = ''
+    nexttok = next(tokens)
+    while nexttok == '!':
+        result += 'not '
+        nexttok = next(tokens)
+
+    if nexttok == '(':
+        sub, nexttok = _parse(tokens)
+        result = '%s(%s)' % (result, sub)
+        if nexttok != ')':
+            raise ValueError('unbalanced parenthesis in plural form')
+    elif nexttok == 'n':
+        result = '%s%s' % (result, nexttok)
+    else:
+        try:
+            value = int(nexttok, 10)
+        except ValueError:
+            raise _error(nexttok) from None
+        result = '%s%d' % (result, value)
+    nexttok = next(tokens)
+
+    j = 100
+    while nexttok in _binary_ops:
+        i = _binary_ops[nexttok]
+        if i < priority:
+            break
+        # Break chained comparisons
+        if i in (3, 4) and j in (3, 4):  # '==', '!=', '<', '>', '<=', '>='
+            result = '(%s)' % result
+        # Replace some C operators by their Python equivalents
+        op = _c2py_ops.get(nexttok, nexttok)
+        right, nexttok = _parse(tokens, i + 1)
+        result = '%s %s %s' % (result, op, right)
+        j = i
+    if j == priority == 4:  # '<', '>', '<=', '>='
+        result = '(%s)' % result
+
+    if nexttok == '?' and priority <= 0:
+        if_true, nexttok = _parse(tokens, 0)
+        if nexttok != ':':
+            raise _error(nexttok)
+        if_false, nexttok = _parse(tokens)
+        result = '%s if %s else %s' % (if_true, result, if_false)
+        if priority == 0:
+            result = '(%s)' % result
+
+    return result, nexttok
 
 def c2py(plural):
     """Gets a C expression as used in PO files for plural forms and returns a
-    Python lambda function that implements an equivalent expression.
+    Python function that implements an equivalent expression.
     """
-    # Security check, allow only the "n" identifier
-    import token, tokenize
-    tokens = tokenize.generate_tokens(io.StringIO(plural).readline)
+
+    if len(plural) > 1000:
+        raise ValueError('plural form expression is too long')
     try:
-        danger = [x for x in tokens if x[0] == token.NAME and x[1] != 'n']
-    except tokenize.TokenError:
-        raise ValueError('plural forms expression error, maybe unbalanced parenthesis')
-    else:
-        if danger:
-            raise ValueError('plural forms expression could be dangerous')
+        result, nexttok = _parse(_tokenize(plural))
+        if nexttok:
+            raise _error(nexttok)
 
-    # Replace some C operators by their Python equivalents
-    plural = plural.replace('&&', ' and ')
-    plural = plural.replace('||', ' or ')
+        depth = 0
+        for c in result:
+            if c == '(':
+                depth += 1
+                if depth > 20:
+                    # Python compiler limit is about 90.
+                    # The most complex example has 2.
+                    raise ValueError('plural form expression is too complex')
+            elif c == ')':
+                depth -= 1
 
-    expr = re.compile(r'\!([^=])')
-    plural = expr.sub(' not \\1', plural)
-
-    # Regular expression and replacement function used to transform
-    # "a?b:c" to "b if a else c".
-    expr = re.compile(r'(.*?)\?(.*?):(.*)')
-    def repl(x):
-        return "(%s if %s else %s)" % (x.group(2), x.group(1),
-                                       expr.sub(repl, x.group(3)))
-
-    # Code to transform the plural expression, taking care of parentheses
-    stack = ['']
-    for c in plural:
-        if c == '(':
-            stack.append('')
-        elif c == ')':
-            if len(stack) == 1:
-                # Actually, we never reach this code, because unbalanced
-                # parentheses get caught in the security check at the
-                # beginning.
-                raise ValueError('unbalanced parenthesis in plural form')
-            s = expr.sub(repl, stack.pop())
-            stack[-1] += '(%s)' % s
-        else:
-            stack[-1] += c
-    plural = expr.sub(repl, stack.pop())
-
-    return eval('lambda n: int(%s)' % plural)
-
+        ns = {}
+        exec('''if True:
+            def func(n):
+                if not isinstance(n, int):
+                    raise ValueError('Plural value must be an integer.')
+                return int(%s)
+            ''' % result, ns)
+        return ns['func']
+    except RecursionError:
+        # Recursion error can be raised in _parse() or exec().
+        raise ValueError('plural form expression is too complex')
 
 
 def _expand_lang(loc):
diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py
--- a/Lib/test/test_gettext.py
+++ b/Lib/test/test_gettext.py
@@ -305,7 +305,9 @@
         x = t.ngettext('There is %s file', 'There are %s files', 2)
         eq(x, 'Hay %s ficheros')
 
-    def test_hu(self):
+    # Examples from http://www.gnu.org/software/gettext/manual/gettext.html
+
+    def test_ja(self):
         eq = self.assertEqual
         f = gettext.c2py('0')
         s = ''.join([ str(f(x)) for x in range(200) ])
@@ -323,6 +325,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "00111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111")
 
+    def test_lv(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n != 0 ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "20111111111111111111101111111110111111111011111111101111111110111111111011111111101111111110111111111011111111111111111110111111111011111111101111111110111111111011111111101111111110111111111011111111")
+
     def test_gd(self):
         eq = self.assertEqual
         f = gettext.c2py('n==1 ? 0 : n==2 ? 1 : 2')
@@ -336,6 +344,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "20122222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222")
 
+    def test_ro(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n==1 ? 0 : (n==0 || (n%100 > 0 && n%100 < 20)) ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "10111111111111111111222222222222222222222222222222222222222222222222222222222222222222222222222222222111111111111111111122222222222222222222222222222222222222222222222222222222222222222222222222222222")
+
     def test_lt(self):
         eq = self.assertEqual
         f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n%10>=2 && (n%100<10 || n%100>=20) ? 1 : 2')
@@ -348,6 +362,12 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "20111222222222222222201112222220111222222011122222201112222220111222222011122222201112222220111222222011122222222222222220111222222011122222201112222220111222222011122222201112222220111222222011122222")
 
+    def test_cs(self):
+        eq = self.assertEqual
+        f = gettext.c2py('(n==1) ? 0 : (n>=2 && n<=4) ? 1 : 2')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "20111222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222")
+
     def test_pl(self):
         eq = self.assertEqual
         f = gettext.c2py('n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2')
@@ -360,10 +380,73 @@
         s = ''.join([ str(f(x)) for x in range(200) ])
         eq(s, "30122333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333012233333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333")
 
+    def test_ar(self):
+        eq = self.assertEqual
+        f = gettext.c2py('n==0 ? 0 : n==1 ? 1 : n==2 ? 2 : n%100>=3 && n%100<=10 ? 3 : n%100>=11 ? 4 : 5')
+        s = ''.join([ str(f(x)) for x in range(200) ])
+        eq(s, "01233333333444444444444444444444444444444444444444444444444444444444444444444444444444444444444444445553333333344444444444444444444444444444444444444444444444444444444444444444444444444444444444444444")
+
     def test_security(self):
         raises = self.assertRaises
         # Test for a dangerous expression
         raises(ValueError, gettext.c2py, "os.chmod('/etc/passwd',0777)")
+        # issue28563
+        raises(ValueError, gettext.c2py, '"(eval(foo) && ""')
+        raises(ValueError, gettext.c2py, 'f"{os.system(\'sh\')}"')
+        # Maximum recursion depth exceeded during compilation
+        raises(ValueError, gettext.c2py, 'n+'*10000 + 'n')
+        self.assertEqual(gettext.c2py('n+'*100 + 'n')(1), 101)
+        # MemoryError during compilation
+        raises(ValueError, gettext.c2py, '('*100 + 'n' + ')'*100)
+        # Maximum recursion depth exceeded in C to Python translator
+        raises(ValueError, gettext.c2py, '('*10000 + 'n' + ')'*10000)
+        self.assertEqual(gettext.c2py('('*20 + 'n' + ')'*20)(1), 1)
+
+    def test_chained_comparison(self):
+        # C doesn't chain comparison as Python so 2 == 2 == 2 gets different results
+        f = gettext.c2py('n == n == n')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '010')
+        f = gettext.c2py('1 < n == n')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '100')
+        f = gettext.c2py('n == n < 2')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '010')
+        f = gettext.c2py('0 < n < 2')
+        self.assertEqual(''.join(str(f(x)) for x in range(3)), '111')
+
+    def test_decimal_number(self):
+        self.assertEqual(gettext.c2py('0123')(1), 123)
+
+    def test_invalid_syntax(self):
+        invalid_expressions = [
+            'x>1', '(n>1', 'n>1)', '42**42**42', '0xa', '1.0', '1e2',
+            'n>0x1', '+n', '-n', 'n()', 'n(1)', '1+', 'nn', 'n n',
+        ]
+        for expr in invalid_expressions:
+            with self.assertRaises(ValueError):
+                gettext.c2py(expr)
+
+    def test_nested_condition_operator(self):
+        self.assertEqual(gettext.c2py('n?1?2:3:4')(0), 4)
+        self.assertEqual(gettext.c2py('n?1?2:3:4')(1), 2)
+        self.assertEqual(gettext.c2py('n?1:3?4:5')(0), 4)
+        self.assertEqual(gettext.c2py('n?1:3?4:5')(1), 1)
+
+    def test_division(self):
+        f = gettext.c2py('2/n*3')
+        self.assertEqual(f(1), 6)
+        self.assertEqual(f(2), 3)
+        self.assertEqual(f(3), 0)
+        self.assertEqual(f(-1), -6)
+        self.assertRaises(ZeroDivisionError, f, 0)
+
+    def test_plural_number(self):
+        f = gettext.c2py('1')
+        self.assertEqual(f(1), 1)
+        self.assertRaises(ValueError, f, 1.0)
+        self.assertRaises(ValueError, f, '1')
+        self.assertRaises(ValueError, f, [])
+        self.assertRaises(ValueError, f, object())
+
 
 class GNUTranslationParsingTest(GettextBaseTest):
     def test_plural_form_error_issue17898(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -112,6 +112,10 @@
 Library
 -------
 
+- Issue #28563: Fixed possible DoS and arbitrary code execution when handle
+  plural form selections in the gettext module.  The expression parser now
+  supports exact syntax supported by GNU gettext.
+
 - Issue #28387: Fixed possible crash in _io.TextIOWrapper deallocator when
   the garbage collector is invoked in other thread.  Based on patch by
   Sebastian Cufre.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 15:15:47 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 08 Nov 2016 20:15:47 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjcyNDMp?=
Message-ID: <20161108201547.49937.58899.906F7367@psf.io>

https://hg.python.org/cpython/rev/9550f0d22d27
changeset:   104983:9550f0d22d27
parent:      104981:4ca9a4f96edc
parent:      104982:a0d50aad7b02
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 15:15:42 2016 -0500
summary:
  Merge 3.6 (issue #27243)

files:
  Lib/test/test_coroutines.py |  18 +++++++++---------
  Python/ceval.c              |   2 +-
  2 files changed, 10 insertions(+), 10 deletions(-)


diff --git a/Lib/test/test_coroutines.py b/Lib/test/test_coroutines.py
--- a/Lib/test/test_coroutines.py
+++ b/Lib/test/test_coroutines.py
@@ -1398,7 +1398,7 @@
 
         buffer = []
         async def test1():
-            with self.assertWarnsRegex(PendingDeprecationWarning, "legacy"):
+            with self.assertWarnsRegex(DeprecationWarning, "legacy"):
                 async for i1, i2 in AsyncIter():
                     buffer.append(i1 + i2)
 
@@ -1412,7 +1412,7 @@
         buffer = []
         async def test2():
             nonlocal buffer
-            with self.assertWarnsRegex(PendingDeprecationWarning, "legacy"):
+            with self.assertWarnsRegex(DeprecationWarning, "legacy"):
                 async for i in AsyncIter():
                     buffer.append(i[0])
                     if i[0] == 20:
@@ -1431,7 +1431,7 @@
         buffer = []
         async def test3():
             nonlocal buffer
-            with self.assertWarnsRegex(PendingDeprecationWarning, "legacy"):
+            with self.assertWarnsRegex(DeprecationWarning, "legacy"):
                 async for i in AsyncIter():
                     if i[0] > 20:
                         continue
@@ -1514,7 +1514,7 @@
                 return 123
 
         async def foo():
-            with self.assertWarnsRegex(PendingDeprecationWarning, "legacy"):
+            with self.assertWarnsRegex(DeprecationWarning, "legacy"):
                 async for i in I():
                     print('never going to happen')
 
@@ -1623,7 +1623,7 @@
                 1/0
         async def foo():
             nonlocal CNT
-            with self.assertWarnsRegex(PendingDeprecationWarning, "legacy"):
+            with self.assertWarnsRegex(DeprecationWarning, "legacy"):
                 async for i in AI():
                     CNT += 1
             CNT += 10
@@ -1650,7 +1650,7 @@
         self.assertEqual(CNT, 0)
 
     def test_for_9(self):
-        # Test that PendingDeprecationWarning can safely be converted into
+        # Test that DeprecationWarning can safely be converted into
         # an exception (__aiter__ should not have a chance to raise
         # a ZeroDivisionError.)
         class AI:
@@ -1660,13 +1660,13 @@
             async for i in AI():
                 pass
 
-        with self.assertRaises(PendingDeprecationWarning):
+        with self.assertRaises(DeprecationWarning):
             with warnings.catch_warnings():
                 warnings.simplefilter("error")
                 run_async(foo())
 
     def test_for_10(self):
-        # Test that PendingDeprecationWarning can safely be converted into
+        # Test that DeprecationWarning can safely be converted into
         # an exception.
         class AI:
             async def __aiter__(self):
@@ -1675,7 +1675,7 @@
             async for i in AI():
                 pass
 
-        with self.assertRaises(PendingDeprecationWarning):
+        with self.assertRaises(DeprecationWarning):
             with warnings.catch_warnings():
                 warnings.simplefilter("error")
                 run_async(foo())
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -1917,7 +1917,7 @@
                 Py_DECREF(iter);
 
                 if (PyErr_WarnFormat(
-                        PyExc_PendingDeprecationWarning, 1,
+                        PyExc_DeprecationWarning, 1,
                         "'%.100s' implements legacy __aiter__ protocol; "
                         "__aiter__ should return an asynchronous "
                         "iterator, not awaitable",

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 15:15:47 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 08 Nov 2016 20:15:47 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI3MjQz?=
 =?utf-8?q?=3A_Change_PendingDeprecationWarning_-=3E_DeprecationWarning=2E?=
Message-ID: <20161108201546.85859.75444.512B8CDA@psf.io>

https://hg.python.org/cpython/rev/a0d50aad7b02
changeset:   104982:a0d50aad7b02
branch:      3.6
parent:      104980:7ea64ab9a5c8
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 15:13:07 2016 -0500
summary:
  Issue #27243: Change PendingDeprecationWarning -> DeprecationWarning.

As it was agreed in the issue, __aiter__ returning an awaitable
should result in PendingDeprecationWarning in 3.5 and in
DeprecationWarning in 3.6.

files:
  Lib/test/test_coroutines.py |  18 +++++++++---------
  Misc/NEWS                   |   5 +++++
  Python/ceval.c              |   2 +-
  3 files changed, 15 insertions(+), 10 deletions(-)


diff --git a/Lib/test/test_coroutines.py b/Lib/test/test_coroutines.py
--- a/Lib/test/test_coroutines.py
+++ b/Lib/test/test_coroutines.py
@@ -1398,7 +1398,7 @@
 
         buffer = []
         async def test1():
-            with self.assertWarnsRegex(PendingDeprecationWarning, "legacy"):
+            with self.assertWarnsRegex(DeprecationWarning, "legacy"):
                 async for i1, i2 in AsyncIter():
                     buffer.append(i1 + i2)
 
@@ -1412,7 +1412,7 @@
         buffer = []
         async def test2():
             nonlocal buffer
-            with self.assertWarnsRegex(PendingDeprecationWarning, "legacy"):
+            with self.assertWarnsRegex(DeprecationWarning, "legacy"):
                 async for i in AsyncIter():
                     buffer.append(i[0])
                     if i[0] == 20:
@@ -1431,7 +1431,7 @@
         buffer = []
         async def test3():
             nonlocal buffer
-            with self.assertWarnsRegex(PendingDeprecationWarning, "legacy"):
+            with self.assertWarnsRegex(DeprecationWarning, "legacy"):
                 async for i in AsyncIter():
                     if i[0] > 20:
                         continue
@@ -1514,7 +1514,7 @@
                 return 123
 
         async def foo():
-            with self.assertWarnsRegex(PendingDeprecationWarning, "legacy"):
+            with self.assertWarnsRegex(DeprecationWarning, "legacy"):
                 async for i in I():
                     print('never going to happen')
 
@@ -1623,7 +1623,7 @@
                 1/0
         async def foo():
             nonlocal CNT
-            with self.assertWarnsRegex(PendingDeprecationWarning, "legacy"):
+            with self.assertWarnsRegex(DeprecationWarning, "legacy"):
                 async for i in AI():
                     CNT += 1
             CNT += 10
@@ -1650,7 +1650,7 @@
         self.assertEqual(CNT, 0)
 
     def test_for_9(self):
-        # Test that PendingDeprecationWarning can safely be converted into
+        # Test that DeprecationWarning can safely be converted into
         # an exception (__aiter__ should not have a chance to raise
         # a ZeroDivisionError.)
         class AI:
@@ -1660,13 +1660,13 @@
             async for i in AI():
                 pass
 
-        with self.assertRaises(PendingDeprecationWarning):
+        with self.assertRaises(DeprecationWarning):
             with warnings.catch_warnings():
                 warnings.simplefilter("error")
                 run_async(foo())
 
     def test_for_10(self):
-        # Test that PendingDeprecationWarning can safely be converted into
+        # Test that DeprecationWarning can safely be converted into
         # an exception.
         class AI:
             async def __aiter__(self):
@@ -1675,7 +1675,7 @@
             async for i in AI():
                 pass
 
-        with self.assertRaises(PendingDeprecationWarning):
+        with self.assertRaises(DeprecationWarning):
             with warnings.catch_warnings():
                 warnings.simplefilter("error")
                 run_async(foo())
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -13,6 +13,11 @@
 - Issue #28583: PyDict_SetDefault didn't combine split table when needed.
   Patch by Xiang Zhang.
 
+- Issue #27243: Change PendingDeprecationWarning -> DeprecationWarning.
+  As it was agreed in the issue, __aiter__ returning an awaitable
+  should result in PendingDeprecationWarning in 3.5 and in
+  DeprecationWarning in 3.6.
+
 Library
 -------
 
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -1911,7 +1911,7 @@
                 Py_DECREF(iter);
 
                 if (PyErr_WarnFormat(
-                        PyExc_PendingDeprecationWarning, 1,
+                        PyExc_DeprecationWarning, 1,
                         "'%.100s' implements legacy __aiter__ protocol; "
                         "__aiter__ should return an asynchronous "
                         "iterator, not awaitable",

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 16:16:04 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 08 Nov 2016 21:16:04 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Fixed_possible_abort_in_ceval_loop_if_=5FPyUnicode=5FFromId=28?=
 =?utf-8?q?=29_fails=2E?=
Message-ID: <20161108211604.26317.78454.40DB155C@psf.io>

https://hg.python.org/cpython/rev/f7308820bb7a
changeset:   104985:f7308820bb7a
branch:      3.6
parent:      104982:a0d50aad7b02
parent:      104984:691004f9bf11
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 08 23:13:36 2016 +0200
summary:
  Fixed possible abort in ceval loop if _PyUnicode_FromId() fails.
Every opcode should end with DISPATCH() or goto error.

files:
  Python/ceval.c |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -2140,7 +2140,7 @@
             else {
                 PyObject *build_class_str = _PyUnicode_FromId(&PyId___build_class__);
                 if (build_class_str == NULL)
-                    break;
+                    goto error;
                 bc = PyObject_GetItem(f->f_builtins, build_class_str);
                 if (bc == NULL) {
                     if (PyErr_ExceptionMatches(PyExc_KeyError))
@@ -2652,7 +2652,7 @@
                 /* do the same if locals() is not a dict */
                 PyObject *ann_str = _PyUnicode_FromId(&PyId___annotations__);
                 if (ann_str == NULL) {
-                    break;
+                    goto error;
                 }
                 ann_dict = PyObject_GetItem(f->f_locals, ann_str);
                 if (ann_dict == NULL) {

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 16:16:04 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 08 Nov 2016 21:16:04 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Fixed_possible_abort_in_ceval_loop_if_=5FPyUnicode=5FFro?=
 =?utf-8?b?bUlkKCkgZmFpbHMu?=
Message-ID: <20161108211604.26426.37338.B44042D3@psf.io>

https://hg.python.org/cpython/rev/4d230008ca96
changeset:   104986:4d230008ca96
parent:      104983:9550f0d22d27
parent:      104985:f7308820bb7a
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 08 23:14:00 2016 +0200
summary:
  Fixed possible abort in ceval loop if _PyUnicode_FromId() fails.
Every opcode should end with DISPATCH() or goto error.

files:
  Python/ceval.c |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -2146,7 +2146,7 @@
             else {
                 PyObject *build_class_str = _PyUnicode_FromId(&PyId___build_class__);
                 if (build_class_str == NULL)
-                    break;
+                    goto error;
                 bc = PyObject_GetItem(f->f_builtins, build_class_str);
                 if (bc == NULL) {
                     if (PyErr_ExceptionMatches(PyExc_KeyError))
@@ -2658,7 +2658,7 @@
                 /* do the same if locals() is not a dict */
                 PyObject *ann_str = _PyUnicode_FromId(&PyId___annotations__);
                 if (ann_str == NULL) {
-                    break;
+                    goto error;
                 }
                 ann_dict = PyObject_GetItem(f->f_locals, ann_str);
                 if (ann_dict == NULL) {

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 16:16:05 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 08 Nov 2016 21:16:05 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fixed_possible?=
 =?utf-8?q?_abort_in_ceval_loop_if_=5FPyUnicode=5FFromId=28=29_fails=2E?=
Message-ID: <20161108211604.50088.98355.2ADAECD6@psf.io>

https://hg.python.org/cpython/rev/691004f9bf11
changeset:   104984:691004f9bf11
branch:      3.5
parent:      104979:e0dd4cc9661a
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 08 23:12:46 2016 +0200
summary:
  Fixed possible abort in ceval loop if _PyUnicode_FromId() fails.
Every opcode should end with DISPATCH() or goto error.

files:
  Python/ceval.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -2196,7 +2196,7 @@
             else {
                 PyObject *build_class_str = _PyUnicode_FromId(&PyId___build_class__);
                 if (build_class_str == NULL)
-                    break;
+                    goto error;
                 bc = PyObject_GetItem(f->f_builtins, build_class_str);
                 if (bc == NULL) {
                     if (PyErr_ExceptionMatches(PyExc_KeyError))

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 16:54:48 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 08 Nov 2016 21:54:48 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjYxODIp?=
Message-ID: <20161108215443.26452.94295.C9361C10@psf.io>

https://hg.python.org/cpython/rev/7b0e79e7f567
changeset:   104988:7b0e79e7f567
parent:      104986:4d230008ca96
parent:      104987:7a996e826f83
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 16:54:39 2016 -0500
summary:
  Merge 3.6 (issue #26182)

files:
  Misc/NEWS    |  2 ++
  Python/ast.c |  8 +++++---
  2 files changed, 7 insertions(+), 3 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -109,6 +109,8 @@
 - Issue #26182: Raise DeprecationWarning when async and await keywords are
   used as variable/attribute/class/function name.
 
+- Issue #26182: Fix ia refleak in code that raises DeprecationWarning.
+
 Library
 -------
 
diff --git a/Python/ast.c b/Python/ast.c
--- a/Python/ast.c
+++ b/Python/ast.c
@@ -944,17 +944,19 @@
         PyObject *message = PyUnicode_FromString(
             "'async' and 'await' will become reserved keywords"
             " in Python 3.7");
+        int ret;
         if (message == NULL) {
             return 1;
         }
-        if (PyErr_WarnExplicitObject(
+        ret = PyErr_WarnExplicitObject(
                 PyExc_DeprecationWarning,
                 message,
                 c->c_filename,
                 LINENO(n),
                 NULL,
-                NULL) < 0)
-        {
+                NULL);
+        Py_DECREF(message);
+        if (ret < 0) {
             return 1;
         }
     }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 16:54:48 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 08 Nov 2016 21:54:48 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI2MTgy?=
 =?utf-8?q?=3A_Fix_ia_refleak_in_code_that_raises_DeprecationWarning=2E?=
Message-ID: <20161108215443.26478.69734.4492E328@psf.io>

https://hg.python.org/cpython/rev/7a996e826f83
changeset:   104987:7a996e826f83
branch:      3.6
parent:      104985:f7308820bb7a
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 16:54:18 2016 -0500
summary:
  Issue #26182: Fix ia refleak in code that raises DeprecationWarning.

files:
  Misc/NEWS    |  2 ++
  Python/ast.c |  8 +++++---
  2 files changed, 7 insertions(+), 3 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -18,6 +18,8 @@
   should result in PendingDeprecationWarning in 3.5 and in
   DeprecationWarning in 3.6.
 
+- Issue #26182: Fix ia refleak in code that raises DeprecationWarning.
+
 Library
 -------
 
diff --git a/Python/ast.c b/Python/ast.c
--- a/Python/ast.c
+++ b/Python/ast.c
@@ -944,17 +944,19 @@
         PyObject *message = PyUnicode_FromString(
             "'async' and 'await' will become reserved keywords"
             " in Python 3.7");
+        int ret;
         if (message == NULL) {
             return 1;
         }
-        if (PyErr_WarnExplicitObject(
+        ret = PyErr_WarnExplicitObject(
                 PyExc_DeprecationWarning,
                 message,
                 c->c_filename,
                 LINENO(n),
                 NULL,
-                NULL) < 0)
-        {
+                NULL);
+        Py_DECREF(message);
+        if (ret < 0) {
             return 1;
         }
     }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 16:57:44 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 08 Nov 2016 21:57:44 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogbmV3czogRml4IGEg?=
 =?utf-8?q?typo?=
Message-ID: <20161108215744.26506.48059.8FE70855@psf.io>

https://hg.python.org/cpython/rev/4e38360f4fef
changeset:   104989:4e38360f4fef
branch:      3.6
parent:      104987:7a996e826f83
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 16:57:22 2016 -0500
summary:
  news: Fix a typo

files:
  Misc/NEWS |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -18,7 +18,7 @@
   should result in PendingDeprecationWarning in 3.5 and in
   DeprecationWarning in 3.6.
 
-- Issue #26182: Fix ia refleak in code that raises DeprecationWarning.
+- Issue #26182: Fix a refleak in code that raises DeprecationWarning.
 
 Library
 -------

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 16:57:45 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 08 Nov 2016 21:57:45 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42ICh0eXBvIGluIE5FV1Mp?=
Message-ID: <20161108215744.85438.65067.28EE9B23@psf.io>

https://hg.python.org/cpython/rev/48aac67269e3
changeset:   104990:48aac67269e3
parent:      104988:7b0e79e7f567
parent:      104989:4e38360f4fef
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 16:57:40 2016 -0500
summary:
  Merge 3.6 (typo in NEWS)

files:
  Misc/NEWS |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -109,7 +109,7 @@
 - Issue #26182: Raise DeprecationWarning when async and await keywords are
   used as variable/attribute/class/function name.
 
-- Issue #26182: Fix ia refleak in code that raises DeprecationWarning.
+- Issue #26182: Fix a refleak in code that raises DeprecationWarning.
 
 Library
 -------

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 19:05:46 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 00:05:46 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI2MDgx?=
 =?utf-8?b?OiBGaXggcmVmbGVhayBpbiBfYXN5bmNpby5GdXR1cmUuX19pdGVyX18oKS50?=
 =?utf-8?q?hrow=2E?=
Message-ID: <20161109000546.96300.23180.8D4CDCA3@psf.io>

https://hg.python.org/cpython/rev/345904bd0456
changeset:   104991:345904bd0456
branch:      3.6
parent:      104989:4e38360f4fef
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 19:04:57 2016 -0500
summary:
  Issue #26081: Fix refleak in _asyncio.Future.__iter__().throw.

files:
  Misc/NEWS                |  2 ++
  Modules/_asynciomodule.c |  4 +++-
  2 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -38,6 +38,8 @@
 
 - Issue #28634: Fix asyncio.isfuture() to support unittest.Mock.
 
+- Issue #26081: Fix refleak in _asyncio.Future.__iter__().throw.
+
 Documentation
 -------------
 
diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -1044,14 +1044,16 @@
     else {
         if (PyExceptionClass_Check(type)) {
             val = PyObject_CallObject(type, NULL);
+            PyErr_SetObject(type, val);
+            Py_DECREF(val);
         }
         else {
             val = type;
             assert (PyExceptionInstance_Check(val));
             type = (PyObject*)Py_TYPE(val);
             assert (PyExceptionClass_Check(type));
+            PyErr_SetObject(type, val);
         }
-        PyErr_SetObject(type, val);
     }
     return FutureIter_iternext(self);
 }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 19:05:46 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 00:05:46 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjYwODEp?=
Message-ID: <20161109000546.1858.85532.480570B8@psf.io>

https://hg.python.org/cpython/rev/b977775aa07d
changeset:   104992:b977775aa07d
parent:      104990:48aac67269e3
parent:      104991:345904bd0456
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 19:05:42 2016 -0500
summary:
  Merge 3.6 (issue #26081)

files:
  Modules/_asynciomodule.c |  4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -1044,14 +1044,16 @@
     else {
         if (PyExceptionClass_Check(type)) {
             val = PyObject_CallObject(type, NULL);
+            PyErr_SetObject(type, val);
+            Py_DECREF(val);
         }
         else {
             val = type;
             assert (PyExceptionInstance_Check(val));
             type = (PyObject*)Py_TYPE(val);
             assert (PyExceptionClass_Check(type));
+            PyErr_SetObject(type, val);
         }
-        PyErr_SetObject(type, val);
     }
     return FutureIter_iternext(self);
 }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 19:16:43 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 00:16:43 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_3=2E6_=28asyncio=29?=
Message-ID: <20161109001643.26527.18583.A99FE0F1@psf.io>

https://hg.python.org/cpython/rev/8e472e85fa5e
changeset:   104995:8e472e85fa5e
parent:      104992:b977775aa07d
parent:      104994:0943453c0210
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 19:16:37 2016 -0500
summary:
  Merge 3.6 (asyncio)

files:
  Lib/asyncio/coroutines.py            |  8 ++++++--
  Lib/test/test_asyncio/test_events.py |  8 ++++++--
  2 files changed, 12 insertions(+), 4 deletions(-)


diff --git a/Lib/asyncio/coroutines.py b/Lib/asyncio/coroutines.py
--- a/Lib/asyncio/coroutines.py
+++ b/Lib/asyncio/coroutines.py
@@ -262,8 +262,12 @@
     assert iscoroutine(coro)
 
     if not hasattr(coro, 'cr_code') and not hasattr(coro, 'gi_code'):
-        # Most likely a Cython coroutine.
-        coro_name = getattr(coro, '__qualname__', coro.__name__)
+        # Most likely a built-in type or a Cython coroutine.
+
+        # Built-in types might not have __qualname__ or __name__.
+        coro_name = getattr(
+            coro, '__qualname__',
+            getattr(coro, '__name__', type(coro).__name__))
         coro_name = '{}()'.format(coro_name)
 
         running = False
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -2384,8 +2384,6 @@
         # (such as ones compiled with Cython).
 
         class Coro:
-            __name__ = 'AAA'
-
             def send(self, v):
                 pass
 
@@ -2399,6 +2397,7 @@
                 pass
 
         coro = Coro()
+        coro.__name__ = 'AAA'
         self.assertTrue(asyncio.iscoroutine(coro))
         self.assertEqual(coroutines._format_coroutine(coro), 'AAA()')
 
@@ -2408,6 +2407,11 @@
         coro.cr_running = True
         self.assertEqual(coroutines._format_coroutine(coro), 'BBB() running')
 
+        coro = Coro()
+        # Some coroutines might not have '__name__', such as
+        # built-in async_gen.asend().
+        self.assertEqual(coroutines._format_coroutine(coro), 'Coro()')
+
 
 class TimerTests(unittest.TestCase):
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 19:16:43 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 00:16:43 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogYXN5bmNpbzogRml4?=
 =?utf-8?q?_=5Fformat=5Fcoroutine_for_coroutine-like_objects_w/o_=5F=5Fnam?=
 =?utf-8?b?ZV9f?=
Message-ID: <20161109001643.122585.25597.959C3BD7@psf.io>

https://hg.python.org/cpython/rev/251f881e4b53
changeset:   104993:251f881e4b53
branch:      3.5
parent:      104984:691004f9bf11
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 19:16:01 2016 -0500
summary:
  asyncio: Fix _format_coroutine for coroutine-like objects w/o __name__

Some built-in coroutine-like objects might not have __name__ or
__qualname__. A good example of such are 'asend', 'aclose' and
'athrow' coroutine methods of asynchronous generators.

files:
  Lib/asyncio/coroutines.py            |  8 ++++++--
  Lib/test/test_asyncio/test_events.py |  8 ++++++--
  2 files changed, 12 insertions(+), 4 deletions(-)


diff --git a/Lib/asyncio/coroutines.py b/Lib/asyncio/coroutines.py
--- a/Lib/asyncio/coroutines.py
+++ b/Lib/asyncio/coroutines.py
@@ -262,8 +262,12 @@
     assert iscoroutine(coro)
 
     if not hasattr(coro, 'cr_code') and not hasattr(coro, 'gi_code'):
-        # Most likely a Cython coroutine.
-        coro_name = getattr(coro, '__qualname__', coro.__name__)
+        # Most likely a built-in type or a Cython coroutine.
+
+        # Built-in types might not have __qualname__ or __name__.
+        coro_name = getattr(
+            coro, '__qualname__',
+            getattr(coro, '__name__', type(coro).__name__))
         coro_name = '{}()'.format(coro_name)
 
         running = False
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -2384,8 +2384,6 @@
         # (such as ones compiled with Cython).
 
         class Coro:
-            __name__ = 'AAA'
-
             def send(self, v):
                 pass
 
@@ -2399,6 +2397,7 @@
                 pass
 
         coro = Coro()
+        coro.__name__ = 'AAA'
         self.assertTrue(asyncio.iscoroutine(coro))
         self.assertEqual(coroutines._format_coroutine(coro), 'AAA()')
 
@@ -2408,6 +2407,11 @@
         coro.cr_running = True
         self.assertEqual(coroutines._format_coroutine(coro), 'BBB() running')
 
+        coro = Coro()
+        # Some coroutines might not have '__name__', such as
+        # built-in async_gen.asend().
+        self.assertEqual(coroutines._format_coroutine(coro), 'Coro()')
+
 
 class TimerTests(unittest.TestCase):
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 19:16:43 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 00:16:43 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5_=28asyncio=29?=
Message-ID: <20161109001643.5999.52799.779E8395@psf.io>

https://hg.python.org/cpython/rev/0943453c0210
changeset:   104994:0943453c0210
branch:      3.6
parent:      104991:345904bd0456
parent:      104993:251f881e4b53
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 19:16:15 2016 -0500
summary:
  Merge 3.5 (asyncio)

files:
  Lib/asyncio/coroutines.py            |  8 ++++++--
  Lib/test/test_asyncio/test_events.py |  8 ++++++--
  2 files changed, 12 insertions(+), 4 deletions(-)


diff --git a/Lib/asyncio/coroutines.py b/Lib/asyncio/coroutines.py
--- a/Lib/asyncio/coroutines.py
+++ b/Lib/asyncio/coroutines.py
@@ -262,8 +262,12 @@
     assert iscoroutine(coro)
 
     if not hasattr(coro, 'cr_code') and not hasattr(coro, 'gi_code'):
-        # Most likely a Cython coroutine.
-        coro_name = getattr(coro, '__qualname__', coro.__name__)
+        # Most likely a built-in type or a Cython coroutine.
+
+        # Built-in types might not have __qualname__ or __name__.
+        coro_name = getattr(
+            coro, '__qualname__',
+            getattr(coro, '__name__', type(coro).__name__))
         coro_name = '{}()'.format(coro_name)
 
         running = False
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -2384,8 +2384,6 @@
         # (such as ones compiled with Cython).
 
         class Coro:
-            __name__ = 'AAA'
-
             def send(self, v):
                 pass
 
@@ -2399,6 +2397,7 @@
                 pass
 
         coro = Coro()
+        coro.__name__ = 'AAA'
         self.assertTrue(asyncio.iscoroutine(coro))
         self.assertEqual(coroutines._format_coroutine(coro), 'AAA()')
 
@@ -2408,6 +2407,11 @@
         coro.cr_running = True
         self.assertEqual(coroutines._format_coroutine(coro), 'BBB() running')
 
+        coro = Coro()
+        # Some coroutines might not have '__name__', such as
+        # built-in async_gen.asend().
+        self.assertEqual(coroutines._format_coroutine(coro), 'Coro()')
+
 
 class TimerTests(unittest.TestCase):
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 19:20:27 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 00:20:27 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5?=
Message-ID: <20161109002027.49980.15567.66DEFF15@psf.io>

https://hg.python.org/cpython/rev/f86ad978e482
changeset:   104997:f86ad978e482
branch:      3.6
parent:      104994:0943453c0210
parent:      104996:4377f414f4a5
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 19:19:52 2016 -0500
summary:
  Merge 3.5

files:
  Objects/genobject.c |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Objects/genobject.c b/Objects/genobject.c
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -1123,7 +1123,7 @@
     0,                                          /* tp_init */
     0,                                          /* tp_alloc */
     0,                                          /* tp_new */
-    PyObject_Del,                               /* tp_free */
+    0,                                          /* tp_free */
 };
 
 PyObject *
@@ -1208,7 +1208,7 @@
     0,                                          /* tp_init */
     0,                                          /* tp_alloc */
     0,                                          /* tp_new */
-    PyObject_Del,                               /* tp_free */
+    0,                                          /* tp_free */
 };
 
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 19:20:27 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 00:20:27 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogZ2Vub2JqZWN0OiBS?=
 =?utf-8?q?emove_unnecessary_tp=5Ffree_slots_from_aiter=5Fwrapper_and_coro?=
 =?utf-8?q?=5Fwrapper?=
Message-ID: <20161109002027.50043.70874.830C5552@psf.io>

https://hg.python.org/cpython/rev/4377f414f4a5
changeset:   104996:4377f414f4a5
branch:      3.5
parent:      104993:251f881e4b53
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 19:19:28 2016 -0500
summary:
  genobject: Remove unnecessary tp_free slots from aiter_wrapper and coro_wrapper

files:
  Objects/genobject.c |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Objects/genobject.c b/Objects/genobject.c
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -1031,7 +1031,7 @@
     0,                                          /* tp_init */
     0,                                          /* tp_alloc */
     0,                                          /* tp_new */
-    PyObject_Del,                               /* tp_free */
+    0,                                          /* tp_free */
 };
 
 PyObject *
@@ -1116,7 +1116,7 @@
     0,                                          /* tp_init */
     0,                                          /* tp_alloc */
     0,                                          /* tp_new */
-    PyObject_Del,                               /* tp_free */
+    0,                                          /* tp_free */
 };
 
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 19:20:27 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 00:20:27 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42?=
Message-ID: <20161109002027.111365.35530.D39E3A1B@psf.io>

https://hg.python.org/cpython/rev/88f0be352cb8
changeset:   104998:88f0be352cb8
parent:      104995:8e472e85fa5e
parent:      104997:f86ad978e482
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 19:20:08 2016 -0500
summary:
  Merge 3.6

files:
  Objects/genobject.c |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Objects/genobject.c b/Objects/genobject.c
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -1123,7 +1123,7 @@
     0,                                          /* tp_init */
     0,                                          /* tp_alloc */
     0,                                          /* tp_new */
-    PyObject_Del,                               /* tp_free */
+    0,                                          /* tp_free */
 };
 
 PyObject *
@@ -1208,7 +1208,7 @@
     0,                                          /* tp_init */
     0,                                          /* tp_alloc */
     0,                                          /* tp_new */
-    PyObject_Del,                               /* tp_free */
+    0,                                          /* tp_free */
 };
 
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 19:46:45 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 00:46:45 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4MDAz?=
 =?utf-8?q?=3A_Make_WrappedVal=2C_ASend_and_AThrow_GC_types?=
Message-ID: <20161109004645.20330.37541.28D5F155@psf.io>

https://hg.python.org/cpython/rev/9f32ef6b210b
changeset:   104999:9f32ef6b210b
branch:      3.6
parent:      104997:f86ad978e482
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 19:46:22 2016 -0500
summary:
  Issue #28003: Make WrappedVal, ASend and AThrow GC types

files:
  Objects/genobject.c |  62 +++++++++++++++++++++++++-------
  1 files changed, 48 insertions(+), 14 deletions(-)


diff --git a/Objects/genobject.c b/Objects/genobject.c
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -1503,14 +1503,14 @@
         _PyAsyncGenWrappedValue *o;
         o = ag_value_freelist[--ag_value_freelist_free];
         assert(_PyAsyncGenWrappedValue_CheckExact(o));
-        PyObject_Del(o);
+        PyObject_GC_Del(o);
     }
 
     while (ag_asend_freelist_free) {
         PyAsyncGenASend *o;
         o = ag_asend_freelist[--ag_asend_freelist_free];
         assert(Py_TYPE(o) == &_PyAsyncGenASend_Type);
-        PyObject_Del(o);
+        PyObject_GC_Del(o);
     }
 
     return ret;
@@ -1557,16 +1557,25 @@
 static void
 async_gen_asend_dealloc(PyAsyncGenASend *o)
 {
+    _PyObject_GC_UNTRACK((PyObject *)o);
     Py_CLEAR(o->ags_gen);
     Py_CLEAR(o->ags_sendval);
     if (ag_asend_freelist_free < _PyAsyncGen_MAXFREELIST) {
         assert(PyAsyncGenASend_CheckExact(o));
         ag_asend_freelist[ag_asend_freelist_free++] = o;
     } else {
-        PyObject_Del(o);
+        PyObject_GC_Del(o);
     }
 }
 
+static int
+async_gen_asend_traverse(PyAsyncGenASend *o, visitproc visit, void *arg)
+{
+    Py_VISIT(o->ags_gen);
+    Py_VISIT(o->ags_sendval);
+    return 0;
+}
+
 
 static PyObject *
 async_gen_asend_send(PyAsyncGenASend *o, PyObject *arg)
@@ -1668,9 +1677,9 @@
     PyObject_GenericGetAttr,                    /* tp_getattro */
     0,                                          /* tp_setattro */
     0,                                          /* tp_as_buffer */
-    Py_TPFLAGS_DEFAULT,                         /* tp_flags */
+    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,    /* tp_flags */
     0,                                          /* tp_doc */
-    0,                                          /* tp_traverse */
+    (traverseproc)async_gen_asend_traverse,     /* tp_traverse */
     0,                                          /* tp_clear */
     0,                                          /* tp_richcompare */
     0,                                          /* tp_weaklistoffset */
@@ -1699,7 +1708,7 @@
         o = ag_asend_freelist[ag_asend_freelist_free];
         _Py_NewReference((PyObject *)o);
     } else {
-        o = PyObject_New(PyAsyncGenASend, &_PyAsyncGenASend_Type);
+        o = PyObject_GC_New(PyAsyncGenASend, &_PyAsyncGenASend_Type);
         if (o == NULL) {
             return NULL;
         }
@@ -1712,6 +1721,8 @@
     o->ags_sendval = sendval;
 
     o->ags_state = AWAITABLE_STATE_INIT;
+
+    _PyObject_GC_TRACK((PyObject*)o);
     return (PyObject*)o;
 }
 
@@ -1722,16 +1733,26 @@
 static void
 async_gen_wrapped_val_dealloc(_PyAsyncGenWrappedValue *o)
 {
+    _PyObject_GC_UNTRACK((PyObject *)o);
     Py_CLEAR(o->agw_val);
     if (ag_value_freelist_free < _PyAsyncGen_MAXFREELIST) {
         assert(_PyAsyncGenWrappedValue_CheckExact(o));
         ag_value_freelist[ag_value_freelist_free++] = o;
     } else {
-        PyObject_Del(o);
+        PyObject_GC_Del(o);
     }
 }
 
 
+static int
+async_gen_wrapped_val_traverse(_PyAsyncGenWrappedValue *o,
+                               visitproc visit, void *arg)
+{
+    Py_VISIT(o->agw_val);
+    return 0;
+}
+
+
 PyTypeObject _PyAsyncGenWrappedValue_Type = {
     PyVarObject_HEAD_INIT(&PyType_Type, 0)
     "async_generator_wrapped_value",            /* tp_name */
@@ -1753,9 +1774,9 @@
     PyObject_GenericGetAttr,                    /* tp_getattro */
     0,                                          /* tp_setattro */
     0,                                          /* tp_as_buffer */
-    Py_TPFLAGS_DEFAULT,                         /* tp_flags */
+    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,    /* tp_flags */
     0,                                          /* tp_doc */
-    0,                                          /* tp_traverse */
+    (traverseproc)async_gen_wrapped_val_traverse, /* tp_traverse */
     0,                                          /* tp_clear */
     0,                                          /* tp_richcompare */
     0,                                          /* tp_weaklistoffset */
@@ -1787,13 +1808,15 @@
         assert(_PyAsyncGenWrappedValue_CheckExact(o));
         _Py_NewReference((PyObject*)o);
     } else {
-        o = PyObject_New(_PyAsyncGenWrappedValue, &_PyAsyncGenWrappedValue_Type);
+        o = PyObject_GC_New(_PyAsyncGenWrappedValue,
+                            &_PyAsyncGenWrappedValue_Type);
         if (o == NULL) {
             return NULL;
         }
     }
     o->agw_val = val;
     Py_INCREF(val);
+    _PyObject_GC_TRACK((PyObject*)o);
     return (PyObject*)o;
 }
 
@@ -1804,9 +1827,19 @@
 static void
 async_gen_athrow_dealloc(PyAsyncGenAThrow *o)
 {
+    _PyObject_GC_UNTRACK((PyObject *)o);
     Py_CLEAR(o->agt_gen);
     Py_CLEAR(o->agt_args);
-    PyObject_Del(o);
+    PyObject_GC_Del(o);
+}
+
+
+static int
+async_gen_athrow_traverse(PyAsyncGenAThrow *o, visitproc visit, void *arg)
+{
+    Py_VISIT(o->agt_gen);
+    Py_VISIT(o->agt_args);
+    return 0;
 }
 
 
@@ -1990,9 +2023,9 @@
     PyObject_GenericGetAttr,                    /* tp_getattro */
     0,                                          /* tp_setattro */
     0,                                          /* tp_as_buffer */
-    Py_TPFLAGS_DEFAULT,                         /* tp_flags */
+    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,    /* tp_flags */
     0,                                          /* tp_doc */
-    0,                                          /* tp_traverse */
+    (traverseproc)async_gen_athrow_traverse,    /* tp_traverse */
     0,                                          /* tp_clear */
     0,                                          /* tp_richcompare */
     0,                                          /* tp_weaklistoffset */
@@ -2016,7 +2049,7 @@
 async_gen_athrow_new(PyAsyncGenObject *gen, PyObject *args)
 {
     PyAsyncGenAThrow *o;
-    o = PyObject_New(PyAsyncGenAThrow, &_PyAsyncGenAThrow_Type);
+    o = PyObject_GC_New(PyAsyncGenAThrow, &_PyAsyncGenAThrow_Type);
     if (o == NULL) {
         return NULL;
     }
@@ -2025,5 +2058,6 @@
     o->agt_state = AWAITABLE_STATE_INIT;
     Py_INCREF(gen);
     Py_XINCREF(args);
+    _PyObject_GC_TRACK((PyObject*)o);
     return (PyObject*)o;
 }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 19:46:45 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 00:46:45 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjgwMDMp?=
Message-ID: <20161109004645.28710.85782.71095B8B@psf.io>

https://hg.python.org/cpython/rev/6f51b495656c
changeset:   105000:6f51b495656c
parent:      104998:88f0be352cb8
parent:      104999:9f32ef6b210b
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 19:46:41 2016 -0500
summary:
  Merge 3.6 (issue #28003)

files:
  Objects/genobject.c |  62 +++++++++++++++++++++++++-------
  1 files changed, 48 insertions(+), 14 deletions(-)


diff --git a/Objects/genobject.c b/Objects/genobject.c
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -1503,14 +1503,14 @@
         _PyAsyncGenWrappedValue *o;
         o = ag_value_freelist[--ag_value_freelist_free];
         assert(_PyAsyncGenWrappedValue_CheckExact(o));
-        PyObject_Del(o);
+        PyObject_GC_Del(o);
     }
 
     while (ag_asend_freelist_free) {
         PyAsyncGenASend *o;
         o = ag_asend_freelist[--ag_asend_freelist_free];
         assert(Py_TYPE(o) == &_PyAsyncGenASend_Type);
-        PyObject_Del(o);
+        PyObject_GC_Del(o);
     }
 
     return ret;
@@ -1557,16 +1557,25 @@
 static void
 async_gen_asend_dealloc(PyAsyncGenASend *o)
 {
+    _PyObject_GC_UNTRACK((PyObject *)o);
     Py_CLEAR(o->ags_gen);
     Py_CLEAR(o->ags_sendval);
     if (ag_asend_freelist_free < _PyAsyncGen_MAXFREELIST) {
         assert(PyAsyncGenASend_CheckExact(o));
         ag_asend_freelist[ag_asend_freelist_free++] = o;
     } else {
-        PyObject_Del(o);
+        PyObject_GC_Del(o);
     }
 }
 
+static int
+async_gen_asend_traverse(PyAsyncGenASend *o, visitproc visit, void *arg)
+{
+    Py_VISIT(o->ags_gen);
+    Py_VISIT(o->ags_sendval);
+    return 0;
+}
+
 
 static PyObject *
 async_gen_asend_send(PyAsyncGenASend *o, PyObject *arg)
@@ -1668,9 +1677,9 @@
     PyObject_GenericGetAttr,                    /* tp_getattro */
     0,                                          /* tp_setattro */
     0,                                          /* tp_as_buffer */
-    Py_TPFLAGS_DEFAULT,                         /* tp_flags */
+    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,    /* tp_flags */
     0,                                          /* tp_doc */
-    0,                                          /* tp_traverse */
+    (traverseproc)async_gen_asend_traverse,     /* tp_traverse */
     0,                                          /* tp_clear */
     0,                                          /* tp_richcompare */
     0,                                          /* tp_weaklistoffset */
@@ -1699,7 +1708,7 @@
         o = ag_asend_freelist[ag_asend_freelist_free];
         _Py_NewReference((PyObject *)o);
     } else {
-        o = PyObject_New(PyAsyncGenASend, &_PyAsyncGenASend_Type);
+        o = PyObject_GC_New(PyAsyncGenASend, &_PyAsyncGenASend_Type);
         if (o == NULL) {
             return NULL;
         }
@@ -1712,6 +1721,8 @@
     o->ags_sendval = sendval;
 
     o->ags_state = AWAITABLE_STATE_INIT;
+
+    _PyObject_GC_TRACK((PyObject*)o);
     return (PyObject*)o;
 }
 
@@ -1722,16 +1733,26 @@
 static void
 async_gen_wrapped_val_dealloc(_PyAsyncGenWrappedValue *o)
 {
+    _PyObject_GC_UNTRACK((PyObject *)o);
     Py_CLEAR(o->agw_val);
     if (ag_value_freelist_free < _PyAsyncGen_MAXFREELIST) {
         assert(_PyAsyncGenWrappedValue_CheckExact(o));
         ag_value_freelist[ag_value_freelist_free++] = o;
     } else {
-        PyObject_Del(o);
+        PyObject_GC_Del(o);
     }
 }
 
 
+static int
+async_gen_wrapped_val_traverse(_PyAsyncGenWrappedValue *o,
+                               visitproc visit, void *arg)
+{
+    Py_VISIT(o->agw_val);
+    return 0;
+}
+
+
 PyTypeObject _PyAsyncGenWrappedValue_Type = {
     PyVarObject_HEAD_INIT(&PyType_Type, 0)
     "async_generator_wrapped_value",            /* tp_name */
@@ -1753,9 +1774,9 @@
     PyObject_GenericGetAttr,                    /* tp_getattro */
     0,                                          /* tp_setattro */
     0,                                          /* tp_as_buffer */
-    Py_TPFLAGS_DEFAULT,                         /* tp_flags */
+    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,    /* tp_flags */
     0,                                          /* tp_doc */
-    0,                                          /* tp_traverse */
+    (traverseproc)async_gen_wrapped_val_traverse, /* tp_traverse */
     0,                                          /* tp_clear */
     0,                                          /* tp_richcompare */
     0,                                          /* tp_weaklistoffset */
@@ -1787,13 +1808,15 @@
         assert(_PyAsyncGenWrappedValue_CheckExact(o));
         _Py_NewReference((PyObject*)o);
     } else {
-        o = PyObject_New(_PyAsyncGenWrappedValue, &_PyAsyncGenWrappedValue_Type);
+        o = PyObject_GC_New(_PyAsyncGenWrappedValue,
+                            &_PyAsyncGenWrappedValue_Type);
         if (o == NULL) {
             return NULL;
         }
     }
     o->agw_val = val;
     Py_INCREF(val);
+    _PyObject_GC_TRACK((PyObject*)o);
     return (PyObject*)o;
 }
 
@@ -1804,9 +1827,19 @@
 static void
 async_gen_athrow_dealloc(PyAsyncGenAThrow *o)
 {
+    _PyObject_GC_UNTRACK((PyObject *)o);
     Py_CLEAR(o->agt_gen);
     Py_CLEAR(o->agt_args);
-    PyObject_Del(o);
+    PyObject_GC_Del(o);
+}
+
+
+static int
+async_gen_athrow_traverse(PyAsyncGenAThrow *o, visitproc visit, void *arg)
+{
+    Py_VISIT(o->agt_gen);
+    Py_VISIT(o->agt_args);
+    return 0;
 }
 
 
@@ -1990,9 +2023,9 @@
     PyObject_GenericGetAttr,                    /* tp_getattro */
     0,                                          /* tp_setattro */
     0,                                          /* tp_as_buffer */
-    Py_TPFLAGS_DEFAULT,                         /* tp_flags */
+    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,    /* tp_flags */
     0,                                          /* tp_doc */
-    0,                                          /* tp_traverse */
+    (traverseproc)async_gen_athrow_traverse,    /* tp_traverse */
     0,                                          /* tp_clear */
     0,                                          /* tp_richcompare */
     0,                                          /* tp_weaklistoffset */
@@ -2016,7 +2049,7 @@
 async_gen_athrow_new(PyAsyncGenObject *gen, PyObject *args)
 {
     PyAsyncGenAThrow *o;
-    o = PyObject_New(PyAsyncGenAThrow, &_PyAsyncGenAThrow_Type);
+    o = PyObject_GC_New(PyAsyncGenAThrow, &_PyAsyncGenAThrow_Type);
     if (o == NULL) {
         return NULL;
     }
@@ -2025,5 +2058,6 @@
     o->agt_state = AWAITABLE_STATE_INIT;
     Py_INCREF(gen);
     Py_XINCREF(args);
+    _PyObject_GC_TRACK((PyObject*)o);
     return (PyObject*)o;
 }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 20:00:12 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 01:00:12 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5_=28issue_=2328639=29?=
Message-ID: <20161109010011.28459.62054.E7F870BB@psf.io>

https://hg.python.org/cpython/rev/6540adb8722a
changeset:   105002:6540adb8722a
branch:      3.6
parent:      104999:9f32ef6b210b
parent:      105001:bb6ad816a43c
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 19:59:29 2016 -0500
summary:
  Merge 3.5 (issue #28639)

files:
  Lib/inspect.py |  4 ++--
  Misc/NEWS      |  3 +++
  2 files changed, 5 insertions(+), 2 deletions(-)


diff --git a/Lib/inspect.py b/Lib/inspect.py
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -219,10 +219,10 @@
     return isinstance(object, types.CoroutineType)
 
 def isawaitable(object):
-    """Return true is object can be passed to an ``await`` expression."""
+    """Return true if object can be passed to an ``await`` expression."""
     return (isinstance(object, types.CoroutineType) or
             isinstance(object, types.GeneratorType) and
-                object.gi_code.co_flags & CO_ITERABLE_COROUTINE or
+                bool(object.gi_code.co_flags & CO_ITERABLE_COROUTINE) or
             isinstance(object, collections.abc.Awaitable))
 
 def istraceback(object):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -40,6 +40,9 @@
 
 - Issue #26081: Fix refleak in _asyncio.Future.__iter__().throw.
 
+- Issue #28639: Fix inspect.isawaitable to always return bool
+  Patch by Justin Mayfield.
+
 Documentation
 -------------
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 20:00:12 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 01:00:12 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjM5?=
 =?utf-8?q?=3A_Fix_inspect=2Eisawaitable_to_always_return_bool?=
Message-ID: <20161109010011.96061.26648.DA34391F@psf.io>

https://hg.python.org/cpython/rev/bb6ad816a43c
changeset:   105001:bb6ad816a43c
branch:      3.5
parent:      104996:4377f414f4a5
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 19:57:44 2016 -0500
summary:
  Issue #28639: Fix inspect.isawaitable to always return bool

Patch by Justin Mayfield.

files:
  Lib/inspect.py |  4 ++--
  Misc/NEWS      |  3 +++
  2 files changed, 5 insertions(+), 2 deletions(-)


diff --git a/Lib/inspect.py b/Lib/inspect.py
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -207,10 +207,10 @@
     return isinstance(object, types.CoroutineType)
 
 def isawaitable(object):
-    """Return true is object can be passed to an ``await`` expression."""
+    """Return true if object can be passed to an ``await`` expression."""
     return (isinstance(object, types.CoroutineType) or
             isinstance(object, types.GeneratorType) and
-                object.gi_code.co_flags & CO_ITERABLE_COROUTINE or
+                bool(object.gi_code.co_flags & CO_ITERABLE_COROUTINE) or
             isinstance(object, collections.abc.Awaitable))
 
 def istraceback(object):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -450,6 +450,9 @@
 - Issue #28613: Fix get_event_loop() return the current loop if 
   called from coroutines/callbacks.
 
+- Issue #28639: Fix inspect.isawaitable to always return bool
+  Patch by Justin Mayfield.
+
 IDLE
 ----
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 20:00:12 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 01:00:12 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg2Mzkp?=
Message-ID: <20161109010011.2058.5071.B2413AE4@psf.io>

https://hg.python.org/cpython/rev/8e3d359cc73b
changeset:   105003:8e3d359cc73b
parent:      105000:6f51b495656c
parent:      105002:6540adb8722a
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 08 19:59:57 2016 -0500
summary:
  Merge 3.6 (issue #28639)

files:
  Lib/inspect.py |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/inspect.py b/Lib/inspect.py
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -219,10 +219,10 @@
     return isinstance(object, types.CoroutineType)
 
 def isawaitable(object):
-    """Return true is object can be passed to an ``await`` expression."""
+    """Return true if object can be passed to an ``await`` expression."""
     return (isinstance(object, types.CoroutineType) or
             isinstance(object, types.GeneratorType) and
-                object.gi_code.co_flags & CO_ITERABLE_COROUTINE or
+                bool(object.gi_code.co_flags & CO_ITERABLE_COROUTINE) or
             isinstance(object, collections.abc.Awaitable))
 
 def istraceback(object):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 22:13:20 2016
From: python-checkins at python.org (andrew.kuchling)
Date: Wed, 09 Nov 2016 03:13:20 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Update_URL_for?=
 =?utf-8?q?_requests_documentation_after_IRC_request?=
Message-ID: <20161109031320.66431.51943.C1293BC7@psf.io>

https://hg.python.org/cpython/rev/78ec6302e06d
changeset:   105004:78ec6302e06d
branch:      3.5
parent:      105001:bb6ad816a43c
user:        Andrew Kuchling <amk at amk.ca>
date:        Tue Nov 08 22:07:43 2016 -0500
summary:
  Update URL for requests documentation after IRC request

files:
  Doc/library/http.client.rst    |  2 +-
  Doc/library/urllib.request.rst |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst
--- a/Doc/library/http.client.rst
+++ b/Doc/library/http.client.rst
@@ -20,7 +20,7 @@
 
 .. seealso::
 
-    The `Requests package <https://requests.readthedocs.org/>`_
+    The `Requests package <https://docs.python-requests.org/>`_
     is recommended for a higher-level HTTP client interface.
 
 .. note::
diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst
--- a/Doc/library/urllib.request.rst
+++ b/Doc/library/urllib.request.rst
@@ -18,7 +18,7 @@
 
 .. seealso::
 
-    The `Requests package <https://requests.readthedocs.org/>`_
+    The `Requests package <https://docs.python-requests.org/>`_
     is recommended for a higher-level HTTP client interface.
 
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 22:16:51 2016
From: python-checkins at python.org (andrew.kuchling)
Date: Wed, 09 Nov 2016 03:16:51 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_from_3=2E5?=
Message-ID: <20161109031651.20484.83069.1278B5B8@psf.io>

https://hg.python.org/cpython/rev/1d482fe040b7
changeset:   105005:1d482fe040b7
branch:      3.6
parent:      105002:6540adb8722a
parent:      105004:78ec6302e06d
user:        Andrew Kuchling <amk at amk.ca>
date:        Tue Nov 08 22:14:39 2016 -0500
summary:
  Merge from 3.5

files:
  Doc/library/http.client.rst    |  2 +-
  Doc/library/urllib.request.rst |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst
--- a/Doc/library/http.client.rst
+++ b/Doc/library/http.client.rst
@@ -20,7 +20,7 @@
 
 .. seealso::
 
-    The `Requests package <https://requests.readthedocs.org/>`_
+    The `Requests package <https://docs.python-requests.org/>`_
     is recommended for a higher-level HTTP client interface.
 
 .. note::
diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst
--- a/Doc/library/urllib.request.rst
+++ b/Doc/library/urllib.request.rst
@@ -18,7 +18,7 @@
 
 .. seealso::
 
-    The `Requests package <https://requests.readthedocs.org/>`_
+    The `Requests package <https://docs.python-requests.org/>`_
     is recommended for a higher-level HTTP client interface.
 
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 22:35:58 2016
From: python-checkins at python.org (andrew.kuchling)
Date: Wed, 09 Nov 2016 03:35:58 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Use_http_inste?=
 =?utf-8?q?ad_of_https=2C_due_to_certificate_error?=
Message-ID: <20161109033558.66539.17806.DDDAB5ED@psf.io>

https://hg.python.org/cpython/rev/952d8dfc0229
changeset:   105006:952d8dfc0229
branch:      3.5
parent:      105004:78ec6302e06d
user:        Andrew Kuchling <amk at amk.ca>
date:        Tue Nov 08 22:33:31 2016 -0500
summary:
  Use http instead of https, due to certificate error

files:
  Doc/library/http.client.rst    |  2 +-
  Doc/library/urllib.request.rst |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst
--- a/Doc/library/http.client.rst
+++ b/Doc/library/http.client.rst
@@ -20,7 +20,7 @@
 
 .. seealso::
 
-    The `Requests package <https://docs.python-requests.org/>`_
+    The `Requests package <http://docs.python-requests.org/>`_
     is recommended for a higher-level HTTP client interface.
 
 .. note::
diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst
--- a/Doc/library/urllib.request.rst
+++ b/Doc/library/urllib.request.rst
@@ -18,7 +18,7 @@
 
 .. seealso::
 
-    The `Requests package <https://docs.python-requests.org/>`_
+    The `Requests package <http://docs.python-requests.org/>`_
     is recommended for a higher-level HTTP client interface.
 
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 22:39:10 2016
From: python-checkins at python.org (andrew.kuchling)
Date: Wed, 09 Nov 2016 03:39:10 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_from_3=2E5?=
Message-ID: <20161109033910.49937.93320.222A7211@psf.io>

https://hg.python.org/cpython/rev/1f7baa4c8b43
changeset:   105007:1f7baa4c8b43
branch:      3.6
parent:      105005:1d482fe040b7
parent:      105006:952d8dfc0229
user:        Andrew Kuchling <amk at amk.ca>
date:        Tue Nov 08 22:38:54 2016 -0500
summary:
  Merge from 3.5

files:
  Doc/library/http.client.rst    |  2 +-
  Doc/library/urllib.request.rst |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst
--- a/Doc/library/http.client.rst
+++ b/Doc/library/http.client.rst
@@ -20,7 +20,7 @@
 
 .. seealso::
 
-    The `Requests package <https://docs.python-requests.org/>`_
+    The `Requests package <http://docs.python-requests.org/>`_
     is recommended for a higher-level HTTP client interface.
 
 .. note::
diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst
--- a/Doc/library/urllib.request.rst
+++ b/Doc/library/urllib.request.rst
@@ -18,7 +18,7 @@
 
 .. seealso::
 
-    The `Requests package <https://docs.python-requests.org/>`_
+    The `Requests package <http://docs.python-requests.org/>`_
     is recommended for a higher-level HTTP client interface.
 
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov  8 22:57:56 2016
From: python-checkins at python.org (andrew.kuchling)
Date: Wed, 09 Nov 2016 03:57:56 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42?=
Message-ID: <20161109035756.20291.2936.A30F0F45@psf.io>

https://hg.python.org/cpython/rev/b78574cb00ab
changeset:   105008:b78574cb00ab
parent:      105003:8e3d359cc73b
parent:      105007:1f7baa4c8b43
user:        Andrew Kuchling <amk at amk.ca>
date:        Tue Nov 08 22:53:59 2016 -0500
summary:
  Merge 3.6

files:
  Doc/library/http.client.rst    |  2 +-
  Doc/library/urllib.request.rst |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst
--- a/Doc/library/http.client.rst
+++ b/Doc/library/http.client.rst
@@ -20,7 +20,7 @@
 
 .. seealso::
 
-    The `Requests package <https://requests.readthedocs.org/>`_
+    The `Requests package <http://docs.python-requests.org/>`_
     is recommended for a higher-level HTTP client interface.
 
 .. note::
diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst
--- a/Doc/library/urllib.request.rst
+++ b/Doc/library/urllib.request.rst
@@ -18,7 +18,7 @@
 
 .. seealso::
 
-    The `Requests package <https://requests.readthedocs.org/>`_
+    The `Requests package <http://docs.python-requests.org/>`_
     is recommended for a higher-level HTTP client interface.
 
 

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Wed Nov  9 04:02:48 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Wed, 09 Nov 2016 09:02:48 +0000
Subject: [Python-checkins] Daily reference leaks (b78574cb00ab): sum=1120
Message-ID: <20161109090247.50140.35000.CEB8D404@psf.io>

results for b78574cb00ab on branch "default"
--------------------------------------------

test_ast leaked [98, 98, 98] references, sum=294
test_ast leaked [98, 98, 98] memory blocks, sum=294
test_asyncio leaked [3, 0, 0] memory blocks, sum=3
test_code leaked [2, 2, 2] references, sum=6
test_code leaked [2, 2, 2] memory blocks, sum=6
test_functools leaked [0, 3, 1] memory blocks, sum=4
test_pydoc leaked [106, 106, 106] references, sum=318
test_pydoc leaked [42, 42, 42] memory blocks, sum=126
test_trace leaked [12, 12, 12] references, sum=36
test_trace leaked [11, 11, 11] memory blocks, sum=33


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog3ML929', '--timeout', '7200']

From python-checkins at python.org  Wed Nov  9 09:43:43 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 14:43:43 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3OTQy?=
 =?utf-8?q?=3A_Fix_memory_leak_in_codeobject=2Ec?=
Message-ID: <20161109144343.6258.44167.C5C7715E@psf.io>

https://hg.python.org/cpython/rev/41613bb27f80
changeset:   105009:41613bb27f80
branch:      2.7
parent:      104976:e0cc3fadd7b3
user:        Yury Selivanov <yury at magic.io>
date:        Wed Nov 09 09:41:15 2016 -0500
summary:
  Issue #27942: Fix memory leak in codeobject.c

files:
  Objects/codeobject.c |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Objects/codeobject.c b/Objects/codeobject.c
--- a/Objects/codeobject.c
+++ b/Objects/codeobject.c
@@ -65,6 +65,7 @@
             intern_string_constants(v);
         }
         else if (PyFrozenSet_CheckExact(v)) {
+            PyObject *w = v;
             PyObject *tmp = PySequence_Tuple(v);
             if (tmp == NULL) {
                 PyErr_Clear();
@@ -77,6 +78,7 @@
                 }
                 else {
                     PyTuple_SET_ITEM(tuple, i, v);
+                    Py_DECREF(w);
                     modified = 1;
                 }
             }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 09:43:43 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 14:43:43 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogc3N1ZSAjMjc5NDI6?=
 =?utf-8?q?_Fix_memory_leak_in_codeobject=2Ec?=
Message-ID: <20161109144343.2058.68765.745DB598@psf.io>

https://hg.python.org/cpython/rev/2c6825c9ecfd
changeset:   105010:2c6825c9ecfd
branch:      3.5
parent:      105006:952d8dfc0229
user:        Yury Selivanov <yury at magic.io>
date:        Wed Nov 09 09:42:14 2016 -0500
summary:
  ssue #27942: Fix memory leak in codeobject.c

files:
  Misc/NEWS            |  2 ++
  Objects/codeobject.c |  2 ++
  2 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -110,6 +110,8 @@
 - Issue #27443: __length_hint__() of bytearray iterators no longer return a
   negative integer for a resized bytearray.
 
+- Issue #27942: Fix memory leak in codeobject.c
+
 Library
 -------
 
diff --git a/Objects/codeobject.c b/Objects/codeobject.c
--- a/Objects/codeobject.c
+++ b/Objects/codeobject.c
@@ -69,6 +69,7 @@
             intern_string_constants(v);
         }
         else if (PyFrozenSet_CheckExact(v)) {
+            PyObject *w = v;
             PyObject *tmp = PySequence_Tuple(v);
             if (tmp == NULL) {
                 PyErr_Clear();
@@ -81,6 +82,7 @@
                 }
                 else {
                     PyTuple_SET_ITEM(tuple, i, v);
+                    Py_DECREF(w);
                     modified = 1;
                 }
             }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 09:43:44 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 14:43:44 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjc5NDIp?=
Message-ID: <20161109144343.6164.16809.65AA5040@psf.io>

https://hg.python.org/cpython/rev/c27269c0d619
changeset:   105012:c27269c0d619
parent:      105008:b78574cb00ab
parent:      105011:b671ac7ae620
user:        Yury Selivanov <yury at magic.io>
date:        Wed Nov 09 09:43:37 2016 -0500
summary:
  Merge 3.6 (issue #27942)

files:
  Misc/NEWS            |  2 ++
  Objects/codeobject.c |  2 ++
  2 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -588,6 +588,8 @@
 
 - Issue #28008: Implement PEP 530 -- asynchronous comprehensions.
 
+- Issue #27942: Fix memory leak in codeobject.c
+
 Library
 -------
 
diff --git a/Objects/codeobject.c b/Objects/codeobject.c
--- a/Objects/codeobject.c
+++ b/Objects/codeobject.c
@@ -77,6 +77,7 @@
             intern_string_constants(v);
         }
         else if (PyFrozenSet_CheckExact(v)) {
+            PyObject *w = v;
             PyObject *tmp = PySequence_Tuple(v);
             if (tmp == NULL) {
                 PyErr_Clear();
@@ -89,6 +90,7 @@
                 }
                 else {
                     PyTuple_SET_ITEM(tuple, i, v);
+                    Py_DECREF(w);
                     modified = 1;
                 }
             }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 09:43:45 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 14:43:45 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5_=28issue_=2327942=29?=
Message-ID: <20161109144343.49911.80317.C63F49FC@psf.io>

https://hg.python.org/cpython/rev/b671ac7ae620
changeset:   105011:b671ac7ae620
branch:      3.6
parent:      105007:1f7baa4c8b43
parent:      105010:2c6825c9ecfd
user:        Yury Selivanov <yury at magic.io>
date:        Wed Nov 09 09:43:08 2016 -0500
summary:
  Merge 3.5 (issue #27942)

files:
  Misc/NEWS            |  2 ++
  Objects/codeobject.c |  2 ++
  2 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -588,6 +588,8 @@
 
 - Issue #28008: Implement PEP 530 -- asynchronous comprehensions.
 
+- Issue #27942: Fix memory leak in codeobject.c
+
 Library
 -------
 
diff --git a/Objects/codeobject.c b/Objects/codeobject.c
--- a/Objects/codeobject.c
+++ b/Objects/codeobject.c
@@ -77,6 +77,7 @@
             intern_string_constants(v);
         }
         else if (PyFrozenSet_CheckExact(v)) {
+            PyObject *w = v;
             PyObject *tmp = PySequence_Tuple(v);
             if (tmp == NULL) {
                 PyErr_Clear();
@@ -89,6 +90,7 @@
                 }
                 else {
                     PyTuple_SET_ITEM(tuple, i, v);
+                    Py_DECREF(w);
                     modified = 1;
                 }
             }

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Wed Nov  9 10:14:14 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Wed, 9 Nov 2016 15:14:14 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python Default
 2016-11-09
Message-ID: <3f7f25db-c46b-468a-a4dc-813b92083d60@irsmsx105.ger.corp.intel.com>

Results for project Python default, build date 2016-11-09 03:01:44 +0000
commit:		8e3d359cc73b
previous commit:	cf7711887b4a
revision date:	2016-11-09 00:59:57 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.19%          0.07%          6.34%            14.46%
:-)             pybench      0.19%         -0.10%          5.20%             4.13%
:-|            regex_v8      2.77%          1.22%         -1.84%             4.07%
:-)               nbody      0.11%          0.47%          2.54%             4.79%
:-(        json_dump_v2      0.30%          0.49%         -9.73%            14.05%
:-|      normal_startup      0.47%         -0.05%         -0.48%             6.72%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-11-09/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Wed Nov  9 10:13:28 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Wed, 9 Nov 2016 15:13:28 +0000
Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-11-09
Message-ID: <08be0d14-cd6a-4892-b7ba-5df4957665c2@irsmsx105.ger.corp.intel.com>

Results for project Python 2.7, build date 2016-11-09 03:48:01 +0000
commit:		e0cc3fadd7b3
previous commit:	1f61a5d2f448
revision date:	2016-11-08 19:15:55 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.24%         -1.37%          3.63%             6.64%
:-)             pybench      0.13%         -0.02%          5.82%             4.62%
:-(            regex_v8      0.93%         -0.39%         -2.56%            11.18%
:-)               nbody      0.12%          0.11%          8.07%             4.07%
:-)        json_dump_v2      0.34%         -0.40%          2.27%             9.45%
:-|      normal_startup      1.16%          0.90%          0.38%             1.40%
:-|             ssbench      0.16%         -0.74%          1.53%             1.80%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-11-09/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Wed Nov  9 15:48:28 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 20:48:28 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5_=28issue_=2328652=29?=
Message-ID: <20161109204828.1883.79125.E2BE14AA@psf.io>

https://hg.python.org/cpython/rev/1273f1a3ddf7
changeset:   105014:1273f1a3ddf7
branch:      3.6
parent:      105011:b671ac7ae620
parent:      105013:85022978d900
user:        Yury Selivanov <yury at magic.io>
date:        Wed Nov 09 15:47:47 2016 -0500
summary:
  Merge 3.5 (issue #28652)

files:
  Lib/asyncio/base_events.py                |  56 +++++++-
  Lib/asyncio/unix_events.py                |   4 +-
  Lib/test/test_asyncio/test_base_events.py |  63 +++++++++-
  Lib/test/test_asyncio/test_events.py      |  11 +-
  Lib/test/test_asyncio/test_unix_events.py |  27 ++++
  Misc/NEWS                                 |   2 +
  6 files changed, 139 insertions(+), 24 deletions(-)


diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -84,12 +84,26 @@
                              'SO_REUSEPORT defined but not implemented.')
 
 
-# Linux's sock.type is a bitmask that can include extra info about socket.
-_SOCKET_TYPE_MASK = 0
-if hasattr(socket, 'SOCK_NONBLOCK'):
-    _SOCKET_TYPE_MASK |= socket.SOCK_NONBLOCK
-if hasattr(socket, 'SOCK_CLOEXEC'):
-    _SOCKET_TYPE_MASK |= socket.SOCK_CLOEXEC
+def _is_stream_socket(sock):
+    # Linux's socket.type is a bitmask that can include extra info
+    # about socket, therefore we can't do simple
+    # `sock_type == socket.SOCK_STREAM`.
+    return (sock.type & socket.SOCK_STREAM) == socket.SOCK_STREAM
+
+
+def _is_dgram_socket(sock):
+    # Linux's socket.type is a bitmask that can include extra info
+    # about socket, therefore we can't do simple
+    # `sock_type == socket.SOCK_DGRAM`.
+    return (sock.type & socket.SOCK_DGRAM) == socket.SOCK_DGRAM
+
+
+def _is_ip_socket(sock):
+    if sock.family == socket.AF_INET:
+        return True
+    if hasattr(socket, 'AF_INET6') and sock.family == socket.AF_INET6:
+        return True
+    return False
 
 
 def _ipaddr_info(host, port, family, type, proto):
@@ -102,8 +116,12 @@
             host is None:
         return None
 
-    type &= ~_SOCKET_TYPE_MASK
     if type == socket.SOCK_STREAM:
+        # Linux only:
+        #    getaddrinfo() can raise when socket.type is a bit mask.
+        #    So if socket.type is a bit mask of SOCK_STREAM, and say
+        #    SOCK_NONBLOCK, we simply return None, which will trigger
+        #    a call to getaddrinfo() letting it process this request.
         proto = socket.IPPROTO_TCP
     elif type == socket.SOCK_DGRAM:
         proto = socket.IPPROTO_UDP
@@ -124,7 +142,9 @@
             return None
 
     if family == socket.AF_UNSPEC:
-        afs = [socket.AF_INET, socket.AF_INET6]
+        afs = [socket.AF_INET]
+        if hasattr(socket, 'AF_INET6'):
+            afs.append(socket.AF_INET6)
     else:
         afs = [family]
 
@@ -772,9 +792,13 @@
                     raise OSError('Multiple exceptions: {}'.format(
                         ', '.join(str(exc) for exc in exceptions)))
 
-        elif sock is None:
-            raise ValueError(
-                'host and port was not specified and no sock specified')
+        else:
+            if sock is None:
+                raise ValueError(
+                    'host and port was not specified and no sock specified')
+            if not _is_stream_socket(sock) or not _is_ip_socket(sock):
+                raise ValueError(
+                    'A TCP Stream Socket was expected, got {!r}'.format(sock))
 
         transport, protocol = yield from self._create_connection_transport(
             sock, protocol_factory, ssl, server_hostname)
@@ -818,6 +842,9 @@
                                  allow_broadcast=None, sock=None):
         """Create datagram connection."""
         if sock is not None:
+            if not _is_dgram_socket(sock):
+                raise ValueError(
+                    'A UDP Socket was expected, got {!r}'.format(sock))
             if (local_addr or remote_addr or
                     family or proto or flags or
                     reuse_address or reuse_port or allow_broadcast):
@@ -1028,6 +1055,9 @@
         else:
             if sock is None:
                 raise ValueError('Neither host/port nor sock were specified')
+            if not _is_stream_socket(sock) or not _is_ip_socket(sock):
+                raise ValueError(
+                    'A TCP Stream Socket was expected, got {!r}'.format(sock))
             sockets = [sock]
 
         server = Server(self, sockets)
@@ -1049,6 +1079,10 @@
         This method is a coroutine.  When completed, the coroutine
         returns a (transport, protocol) pair.
         """
+        if not _is_stream_socket(sock):
+            raise ValueError(
+                'A Stream Socket was expected, got {!r}'.format(sock))
+
         transport, protocol = yield from self._create_connection_transport(
             sock, protocol_factory, ssl, '', server_side=True)
         if self._debug:
diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py
--- a/Lib/asyncio/unix_events.py
+++ b/Lib/asyncio/unix_events.py
@@ -235,7 +235,7 @@
             if sock is None:
                 raise ValueError('no path and sock were specified')
             if (sock.family != socket.AF_UNIX or
-                    sock.type != socket.SOCK_STREAM):
+                    not base_events._is_stream_socket(sock)):
                 raise ValueError(
                     'A UNIX Domain Stream Socket was expected, got {!r}'
                     .format(sock))
@@ -289,7 +289,7 @@
                     'path was not specified, and no sock specified')
 
             if (sock.family != socket.AF_UNIX or
-                    sock.type != socket.SOCK_STREAM):
+                    not base_events._is_stream_socket(sock)):
                 raise ValueError(
                     'A UNIX Domain Stream Socket was expected, got {!r}'
                     .format(sock))
diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py
--- a/Lib/test/test_asyncio/test_base_events.py
+++ b/Lib/test/test_asyncio/test_base_events.py
@@ -116,6 +116,13 @@
         self.assertIsNone(
             base_events._ipaddr_info('::3%lo0', 1, INET6, STREAM, TCP))
 
+        if hasattr(socket, 'SOCK_NONBLOCK'):
+            self.assertEqual(
+                None,
+                base_events._ipaddr_info(
+                    '1.2.3.4', 1, INET, STREAM | socket.SOCK_NONBLOCK, TCP))
+
+
     def test_port_parameter_types(self):
         # Test obscure kinds of arguments for "port".
         INET = socket.AF_INET
@@ -1040,6 +1047,43 @@
             MyProto, 'example.com', 80, sock=object())
         self.assertRaises(ValueError, self.loop.run_until_complete, coro)
 
+    @unittest.skipUnless(hasattr(socket, 'AF_UNIX'), 'no Unix sockets')
+    def test_create_connection_wrong_sock(self):
+        sock = socket.socket(socket.AF_UNIX)
+        with sock:
+            coro = self.loop.create_connection(MyProto, sock=sock)
+            with self.assertRaisesRegex(ValueError,
+                                        'A TCP Stream Socket was expected'):
+                self.loop.run_until_complete(coro)
+
+    @unittest.skipUnless(hasattr(socket, 'AF_UNIX'), 'no Unix sockets')
+    def test_create_server_wrong_sock(self):
+        sock = socket.socket(socket.AF_UNIX)
+        with sock:
+            coro = self.loop.create_server(MyProto, sock=sock)
+            with self.assertRaisesRegex(ValueError,
+                                        'A TCP Stream Socket was expected'):
+                self.loop.run_until_complete(coro)
+
+    @unittest.skipUnless(hasattr(socket, 'SOCK_NONBLOCK'),
+                         'no socket.SOCK_NONBLOCK (linux only)')
+    def test_create_server_stream_bittype(self):
+        sock = socket.socket(
+            socket.AF_INET, socket.SOCK_STREAM | socket.SOCK_NONBLOCK)
+        with sock:
+            coro = self.loop.create_server(lambda: None, sock=sock)
+            srv = self.loop.run_until_complete(coro)
+            srv.close()
+            self.loop.run_until_complete(srv.wait_closed())
+
+    def test_create_datagram_endpoint_wrong_sock(self):
+        sock = socket.socket(socket.AF_INET)
+        with sock:
+            coro = self.loop.create_datagram_endpoint(MyProto, sock=sock)
+            with self.assertRaisesRegex(ValueError,
+                                        'A UDP Socket was expected'):
+                self.loop.run_until_complete(coro)
+
     def test_create_connection_no_host_port_sock(self):
         coro = self.loop.create_connection(MyProto)
         self.assertRaises(ValueError, self.loop.run_until_complete, coro)
@@ -1487,36 +1531,39 @@
         self.assertEqual('CLOSED', protocol.state)
 
     def test_create_datagram_endpoint_sock_sockopts(self):
+        class FakeSock:
+            type = socket.SOCK_DGRAM
+
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, local_addr=('127.0.0.1', 0), sock=object())
+            MyDatagramProto, local_addr=('127.0.0.1', 0), sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, remote_addr=('127.0.0.1', 0), sock=object())
+            MyDatagramProto, remote_addr=('127.0.0.1', 0), sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, family=1, sock=object())
+            MyDatagramProto, family=1, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, proto=1, sock=object())
+            MyDatagramProto, proto=1, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, flags=1, sock=object())
+            MyDatagramProto, flags=1, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, reuse_address=True, sock=object())
+            MyDatagramProto, reuse_address=True, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, reuse_port=True, sock=object())
+            MyDatagramProto, reuse_port=True, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, allow_broadcast=True, sock=object())
+            MyDatagramProto, allow_broadcast=True, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
     def test_create_datagram_endpoint_sockopts(self):
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -791,9 +791,9 @@
         conn, _ = lsock.accept()
         proto = MyProto(loop=loop)
         proto.loop = loop
-        f = loop.create_task(
+        loop.run_until_complete(
             loop.connect_accepted_socket(
-                (lambda : proto), conn, ssl=server_ssl))
+                (lambda: proto), conn, ssl=server_ssl))
         loop.run_forever()
         proto.transport.close()
         lsock.close()
@@ -1377,6 +1377,11 @@
         server.transport.close()
 
     def test_create_datagram_endpoint_sock(self):
+        if (sys.platform == 'win32' and
+                isinstance(self.loop, proactor_events.BaseProactorEventLoop)):
+            raise unittest.SkipTest(
+                'UDP is not supported with proactor event loops')
+
         sock = None
         local_address = ('127.0.0.1', 0)
         infos = self.loop.run_until_complete(
@@ -1394,7 +1399,7 @@
         else:
             assert False, 'Can not create socket.'
 
-        f = self.loop.create_connection(
+        f = self.loop.create_datagram_endpoint(
             lambda: MyDatagramProto(loop=self.loop), sock=sock)
         tr, pr = self.loop.run_until_complete(f)
         self.assertIsInstance(tr, asyncio.Transport)
diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py
--- a/Lib/test/test_asyncio/test_unix_events.py
+++ b/Lib/test/test_asyncio/test_unix_events.py
@@ -280,6 +280,33 @@
                                         'A UNIX Domain Stream.*was expected'):
                 self.loop.run_until_complete(coro)
 
+    def test_create_unix_server_path_dgram(self):
+        sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+        with sock:
+            coro = self.loop.create_unix_server(lambda: None, path=None,
+                                                sock=sock)
+            with self.assertRaisesRegex(ValueError,
+                                        'A UNIX Domain Stream.*was expected'):
+                self.loop.run_until_complete(coro)
+
+    @unittest.skipUnless(hasattr(socket, 'SOCK_NONBLOCK'),
+                         'no socket.SOCK_NONBLOCK (linux only)')
+    def test_create_unix_server_path_stream_bittype(self):
+        sock = socket.socket(
+            socket.AF_UNIX, socket.SOCK_STREAM | socket.SOCK_NONBLOCK)
+        with tempfile.NamedTemporaryFile() as file:
+            fn = file.name
+        try:
+            with sock:
+                sock.bind(fn)
+                coro = self.loop.create_unix_server(lambda: None, path=None,
+                                                    sock=sock)
+                srv = self.loop.run_until_complete(coro)
+                srv.close()
+                self.loop.run_until_complete(srv.wait_closed())
+        finally:
+            os.unlink(fn)
+
     def test_create_unix_connection_path_inetsock(self):
         sock = socket.socket()
         with sock:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -43,6 +43,8 @@
 - Issue #28639: Fix inspect.isawaitable to always return bool
   Patch by Justin Mayfield.
 
+- Issue #28652: Make loop methods reject socket kinds they do not support.
+
 Documentation
 -------------
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 15:48:28 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 20:48:28 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjUy?=
 =?utf-8?q?=3A_Make_loop_methods_reject_socket_kinds_they_do_not_support?=
 =?utf-8?q?=2E?=
Message-ID: <20161109204828.122460.79940.949D22BF@psf.io>

https://hg.python.org/cpython/rev/85022978d900
changeset:   105013:85022978d900
branch:      3.5
parent:      105010:2c6825c9ecfd
user:        Yury Selivanov <yury at magic.io>
date:        Wed Nov 09 15:47:00 2016 -0500
summary:
  Issue #28652: Make loop methods reject socket kinds they do not support.

files:
  Lib/asyncio/base_events.py                |  56 +++++++-
  Lib/asyncio/unix_events.py                |   4 +-
  Lib/test/test_asyncio/test_base_events.py |  63 +++++++++-
  Lib/test/test_asyncio/test_events.py      |  11 +-
  Lib/test/test_asyncio/test_unix_events.py |  27 ++++
  Misc/NEWS                                 |   2 +
  6 files changed, 139 insertions(+), 24 deletions(-)


diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -84,12 +84,26 @@
                              'SO_REUSEPORT defined but not implemented.')
 
 
-# Linux's sock.type is a bitmask that can include extra info about socket.
-_SOCKET_TYPE_MASK = 0
-if hasattr(socket, 'SOCK_NONBLOCK'):
-    _SOCKET_TYPE_MASK |= socket.SOCK_NONBLOCK
-if hasattr(socket, 'SOCK_CLOEXEC'):
-    _SOCKET_TYPE_MASK |= socket.SOCK_CLOEXEC
+def _is_stream_socket(sock):
+    # Linux's socket.type is a bitmask that can include extra info
+    # about socket, therefore we can't do simple
+    # `sock_type == socket.SOCK_STREAM`.
+    return (sock.type & socket.SOCK_STREAM) == socket.SOCK_STREAM
+
+
+def _is_dgram_socket(sock):
+    # Linux's socket.type is a bitmask that can include extra info
+    # about socket, therefore we can't do simple
+    # `sock_type == socket.SOCK_DGRAM`.
+    return (sock.type & socket.SOCK_DGRAM) == socket.SOCK_DGRAM
+
+
+def _is_ip_socket(sock):
+    if sock.family == socket.AF_INET:
+        return True
+    if hasattr(socket, 'AF_INET6') and sock.family == socket.AF_INET6:
+        return True
+    return False
 
 
 def _ipaddr_info(host, port, family, type, proto):
@@ -102,8 +116,12 @@
             host is None:
         return None
 
-    type &= ~_SOCKET_TYPE_MASK
     if type == socket.SOCK_STREAM:
+        # Linux only:
+        #    getaddrinfo() can raise when socket.type is a bit mask.
+        #    So if socket.type is a bit mask of SOCK_STREAM, and say
+        #    SOCK_NONBLOCK, we simply return None, which will trigger
+        #    a call to getaddrinfo() letting it process this request.
         proto = socket.IPPROTO_TCP
     elif type == socket.SOCK_DGRAM:
         proto = socket.IPPROTO_UDP
@@ -124,7 +142,9 @@
             return None
 
     if family == socket.AF_UNSPEC:
-        afs = [socket.AF_INET, socket.AF_INET6]
+        afs = [socket.AF_INET]
+        if hasattr(socket, 'AF_INET6'):
+            afs.append(socket.AF_INET6)
     else:
         afs = [family]
 
@@ -771,9 +791,13 @@
                     raise OSError('Multiple exceptions: {}'.format(
                         ', '.join(str(exc) for exc in exceptions)))
 
-        elif sock is None:
-            raise ValueError(
-                'host and port was not specified and no sock specified')
+        else:
+            if sock is None:
+                raise ValueError(
+                    'host and port was not specified and no sock specified')
+            if not _is_stream_socket(sock) or not _is_ip_socket(sock):
+                raise ValueError(
+                    'A TCP Stream Socket was expected, got {!r}'.format(sock))
 
         transport, protocol = yield from self._create_connection_transport(
             sock, protocol_factory, ssl, server_hostname)
@@ -817,6 +841,9 @@
                                  allow_broadcast=None, sock=None):
         """Create datagram connection."""
         if sock is not None:
+            if not _is_dgram_socket(sock):
+                raise ValueError(
+                    'A UDP Socket was expected, got {!r}'.format(sock))
             if (local_addr or remote_addr or
                     family or proto or flags or
                     reuse_address or reuse_port or allow_broadcast):
@@ -1027,6 +1054,9 @@
         else:
             if sock is None:
                 raise ValueError('Neither host/port nor sock were specified')
+            if not _is_stream_socket(sock) or not _is_ip_socket(sock):
+                raise ValueError(
+                    'A TCP Stream Socket was expected, got {!r}'.format(sock))
             sockets = [sock]
 
         server = Server(self, sockets)
@@ -1048,6 +1078,10 @@
         This method is a coroutine.  When completed, the coroutine
         returns a (transport, protocol) pair.
         """
+        if not _is_stream_socket(sock):
+            raise ValueError(
+                'A Stream Socket was expected, got {!r}'.format(sock))
+
         transport, protocol = yield from self._create_connection_transport(
             sock, protocol_factory, ssl, '', server_side=True)
         if self._debug:
diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py
--- a/Lib/asyncio/unix_events.py
+++ b/Lib/asyncio/unix_events.py
@@ -235,7 +235,7 @@
             if sock is None:
                 raise ValueError('no path and sock were specified')
             if (sock.family != socket.AF_UNIX or
-                    sock.type != socket.SOCK_STREAM):
+                    not base_events._is_stream_socket(sock)):
                 raise ValueError(
                     'A UNIX Domain Stream Socket was expected, got {!r}'
                     .format(sock))
@@ -289,7 +289,7 @@
                     'path was not specified, and no sock specified')
 
             if (sock.family != socket.AF_UNIX or
-                    sock.type != socket.SOCK_STREAM):
+                    not base_events._is_stream_socket(sock)):
                 raise ValueError(
                     'A UNIX Domain Stream Socket was expected, got {!r}'
                     .format(sock))
diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py
--- a/Lib/test/test_asyncio/test_base_events.py
+++ b/Lib/test/test_asyncio/test_base_events.py
@@ -116,6 +116,13 @@
         self.assertIsNone(
             base_events._ipaddr_info('::3%lo0', 1, INET6, STREAM, TCP))
 
+        if hasattr(socket, 'SOCK_NONBLOCK'):
+            self.assertEqual(
+                None,
+                base_events._ipaddr_info(
+                    '1.2.3.4', 1, INET, STREAM | socket.SOCK_NONBLOCK, TCP))
+
+
     def test_port_parameter_types(self):
         # Test obscure kinds of arguments for "port".
         INET = socket.AF_INET
@@ -1040,6 +1047,43 @@
             MyProto, 'example.com', 80, sock=object())
         self.assertRaises(ValueError, self.loop.run_until_complete, coro)
 
+    @unittest.skipUnless(hasattr(socket, 'AF_UNIX'), 'no Unix sockets')
+    def test_create_connection_wrong_sock(self):
+        sock = socket.socket(socket.AF_UNIX)
+        with sock:
+            coro = self.loop.create_connection(MyProto, sock=sock)
+            with self.assertRaisesRegex(ValueError,
+                                        'A TCP Stream Socket was expected'):
+                self.loop.run_until_complete(coro)
+
+    @unittest.skipUnless(hasattr(socket, 'AF_UNIX'), 'no Unix sockets')
+    def test_create_server_wrong_sock(self):
+        sock = socket.socket(socket.AF_UNIX)
+        with sock:
+            coro = self.loop.create_server(MyProto, sock=sock)
+            with self.assertRaisesRegex(ValueError,
+                                        'A TCP Stream Socket was expected'):
+                self.loop.run_until_complete(coro)
+
+    @unittest.skipUnless(hasattr(socket, 'SOCK_NONBLOCK'),
+                         'no socket.SOCK_NONBLOCK (linux only)')
+    def test_create_server_stream_bittype(self):
+        sock = socket.socket(
+            socket.AF_INET, socket.SOCK_STREAM | socket.SOCK_NONBLOCK)
+        with sock:
+            coro = self.loop.create_server(lambda: None, sock=sock)
+            srv = self.loop.run_until_complete(coro)
+            srv.close()
+            self.loop.run_until_complete(srv.wait_closed())
+
+    def test_create_datagram_endpoint_wrong_sock(self):
+        sock = socket.socket(socket.AF_INET)
+        with sock:
+            coro = self.loop.create_datagram_endpoint(MyProto, sock=sock)
+            with self.assertRaisesRegex(ValueError,
+                                        'A UDP Socket was expected'):
+                self.loop.run_until_complete(coro)
+
     def test_create_connection_no_host_port_sock(self):
         coro = self.loop.create_connection(MyProto)
         self.assertRaises(ValueError, self.loop.run_until_complete, coro)
@@ -1487,36 +1531,39 @@
         self.assertEqual('CLOSED', protocol.state)
 
     def test_create_datagram_endpoint_sock_sockopts(self):
+        class FakeSock:
+            type = socket.SOCK_DGRAM
+
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, local_addr=('127.0.0.1', 0), sock=object())
+            MyDatagramProto, local_addr=('127.0.0.1', 0), sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, remote_addr=('127.0.0.1', 0), sock=object())
+            MyDatagramProto, remote_addr=('127.0.0.1', 0), sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, family=1, sock=object())
+            MyDatagramProto, family=1, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, proto=1, sock=object())
+            MyDatagramProto, proto=1, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, flags=1, sock=object())
+            MyDatagramProto, flags=1, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, reuse_address=True, sock=object())
+            MyDatagramProto, reuse_address=True, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, reuse_port=True, sock=object())
+            MyDatagramProto, reuse_port=True, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, allow_broadcast=True, sock=object())
+            MyDatagramProto, allow_broadcast=True, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
     def test_create_datagram_endpoint_sockopts(self):
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -791,9 +791,9 @@
         conn, _ = lsock.accept()
         proto = MyProto(loop=loop)
         proto.loop = loop
-        f = loop.create_task(
+        loop.run_until_complete(
             loop.connect_accepted_socket(
-                (lambda : proto), conn, ssl=server_ssl))
+                (lambda: proto), conn, ssl=server_ssl))
         loop.run_forever()
         proto.transport.close()
         lsock.close()
@@ -1377,6 +1377,11 @@
         server.transport.close()
 
     def test_create_datagram_endpoint_sock(self):
+        if (sys.platform == 'win32' and
+                isinstance(self.loop, proactor_events.BaseProactorEventLoop)):
+            raise unittest.SkipTest(
+                'UDP is not supported with proactor event loops')
+
         sock = None
         local_address = ('127.0.0.1', 0)
         infos = self.loop.run_until_complete(
@@ -1394,7 +1399,7 @@
         else:
             assert False, 'Can not create socket.'
 
-        f = self.loop.create_connection(
+        f = self.loop.create_datagram_endpoint(
             lambda: MyDatagramProto(loop=self.loop), sock=sock)
         tr, pr = self.loop.run_until_complete(f)
         self.assertIsInstance(tr, asyncio.Transport)
diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py
--- a/Lib/test/test_asyncio/test_unix_events.py
+++ b/Lib/test/test_asyncio/test_unix_events.py
@@ -280,6 +280,33 @@
                                         'A UNIX Domain Stream.*was expected'):
                 self.loop.run_until_complete(coro)
 
+    def test_create_unix_server_path_dgram(self):
+        sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+        with sock:
+            coro = self.loop.create_unix_server(lambda: None, path=None,
+                                                sock=sock)
+            with self.assertRaisesRegex(ValueError,
+                                        'A UNIX Domain Stream.*was expected'):
+                self.loop.run_until_complete(coro)
+
+    @unittest.skipUnless(hasattr(socket, 'SOCK_NONBLOCK'),
+                         'no socket.SOCK_NONBLOCK (linux only)')
+    def test_create_unix_server_path_stream_bittype(self):
+        sock = socket.socket(
+            socket.AF_UNIX, socket.SOCK_STREAM | socket.SOCK_NONBLOCK)
+        with tempfile.NamedTemporaryFile() as file:
+            fn = file.name
+        try:
+            with sock:
+                sock.bind(fn)
+                coro = self.loop.create_unix_server(lambda: None, path=None,
+                                                    sock=sock)
+                srv = self.loop.run_until_complete(coro)
+                srv.close()
+                self.loop.run_until_complete(srv.wait_closed())
+        finally:
+            os.unlink(fn)
+
     def test_create_unix_connection_path_inetsock(self):
         sock = socket.socket()
         with sock:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -455,6 +455,8 @@
 - Issue #28639: Fix inspect.isawaitable to always return bool
   Patch by Justin Mayfield.
 
+- Issue #28652: Make loop methods reject socket kinds they do not support.
+
 IDLE
 ----
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 15:48:29 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 20:48:29 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg2NTIp?=
Message-ID: <20161109204829.85660.6183.B9242C74@psf.io>

https://hg.python.org/cpython/rev/719da54652c5
changeset:   105015:719da54652c5
parent:      105012:c27269c0d619
parent:      105014:1273f1a3ddf7
user:        Yury Selivanov <yury at magic.io>
date:        Wed Nov 09 15:48:22 2016 -0500
summary:
  Merge 3.6 (issue #28652)

files:
  Lib/asyncio/base_events.py                |  56 +++++++-
  Lib/asyncio/unix_events.py                |   4 +-
  Lib/test/test_asyncio/test_base_events.py |  63 +++++++++-
  Lib/test/test_asyncio/test_events.py      |  11 +-
  Lib/test/test_asyncio/test_unix_events.py |  27 ++++
  5 files changed, 137 insertions(+), 24 deletions(-)


diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -84,12 +84,26 @@
                              'SO_REUSEPORT defined but not implemented.')
 
 
-# Linux's sock.type is a bitmask that can include extra info about socket.
-_SOCKET_TYPE_MASK = 0
-if hasattr(socket, 'SOCK_NONBLOCK'):
-    _SOCKET_TYPE_MASK |= socket.SOCK_NONBLOCK
-if hasattr(socket, 'SOCK_CLOEXEC'):
-    _SOCKET_TYPE_MASK |= socket.SOCK_CLOEXEC
+def _is_stream_socket(sock):
+    # Linux's socket.type is a bitmask that can include extra info
+    # about socket, therefore we can't do simple
+    # `sock_type == socket.SOCK_STREAM`.
+    return (sock.type & socket.SOCK_STREAM) == socket.SOCK_STREAM
+
+
+def _is_dgram_socket(sock):
+    # Linux's socket.type is a bitmask that can include extra info
+    # about socket, therefore we can't do simple
+    # `sock_type == socket.SOCK_DGRAM`.
+    return (sock.type & socket.SOCK_DGRAM) == socket.SOCK_DGRAM
+
+
+def _is_ip_socket(sock):
+    if sock.family == socket.AF_INET:
+        return True
+    if hasattr(socket, 'AF_INET6') and sock.family == socket.AF_INET6:
+        return True
+    return False
 
 
 def _ipaddr_info(host, port, family, type, proto):
@@ -102,8 +116,12 @@
             host is None:
         return None
 
-    type &= ~_SOCKET_TYPE_MASK
     if type == socket.SOCK_STREAM:
+        # Linux only:
+        #    getaddrinfo() can raise when socket.type is a bit mask.
+        #    So if socket.type is a bit mask of SOCK_STREAM, and say
+        #    SOCK_NONBLOCK, we simply return None, which will trigger
+        #    a call to getaddrinfo() letting it process this request.
         proto = socket.IPPROTO_TCP
     elif type == socket.SOCK_DGRAM:
         proto = socket.IPPROTO_UDP
@@ -124,7 +142,9 @@
             return None
 
     if family == socket.AF_UNSPEC:
-        afs = [socket.AF_INET, socket.AF_INET6]
+        afs = [socket.AF_INET]
+        if hasattr(socket, 'AF_INET6'):
+            afs.append(socket.AF_INET6)
     else:
         afs = [family]
 
@@ -772,9 +792,13 @@
                     raise OSError('Multiple exceptions: {}'.format(
                         ', '.join(str(exc) for exc in exceptions)))
 
-        elif sock is None:
-            raise ValueError(
-                'host and port was not specified and no sock specified')
+        else:
+            if sock is None:
+                raise ValueError(
+                    'host and port was not specified and no sock specified')
+            if not _is_stream_socket(sock) or not _is_ip_socket(sock):
+                raise ValueError(
+                    'A TCP Stream Socket was expected, got {!r}'.format(sock))
 
         transport, protocol = yield from self._create_connection_transport(
             sock, protocol_factory, ssl, server_hostname)
@@ -818,6 +842,9 @@
                                  allow_broadcast=None, sock=None):
         """Create datagram connection."""
         if sock is not None:
+            if not _is_dgram_socket(sock):
+                raise ValueError(
+                    'A UDP Socket was expected, got {!r}'.format(sock))
             if (local_addr or remote_addr or
                     family or proto or flags or
                     reuse_address or reuse_port or allow_broadcast):
@@ -1028,6 +1055,9 @@
         else:
             if sock is None:
                 raise ValueError('Neither host/port nor sock were specified')
+            if not _is_stream_socket(sock) or not _is_ip_socket(sock):
+                raise ValueError(
+                    'A TCP Stream Socket was expected, got {!r}'.format(sock))
             sockets = [sock]
 
         server = Server(self, sockets)
@@ -1049,6 +1079,10 @@
         This method is a coroutine.  When completed, the coroutine
         returns a (transport, protocol) pair.
         """
+        if not _is_stream_socket(sock):
+            raise ValueError(
+                'A Stream Socket was expected, got {!r}'.format(sock))
+
         transport, protocol = yield from self._create_connection_transport(
             sock, protocol_factory, ssl, '', server_side=True)
         if self._debug:
diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py
--- a/Lib/asyncio/unix_events.py
+++ b/Lib/asyncio/unix_events.py
@@ -235,7 +235,7 @@
             if sock is None:
                 raise ValueError('no path and sock were specified')
             if (sock.family != socket.AF_UNIX or
-                    sock.type != socket.SOCK_STREAM):
+                    not base_events._is_stream_socket(sock)):
                 raise ValueError(
                     'A UNIX Domain Stream Socket was expected, got {!r}'
                     .format(sock))
@@ -289,7 +289,7 @@
                     'path was not specified, and no sock specified')
 
             if (sock.family != socket.AF_UNIX or
-                    sock.type != socket.SOCK_STREAM):
+                    not base_events._is_stream_socket(sock)):
                 raise ValueError(
                     'A UNIX Domain Stream Socket was expected, got {!r}'
                     .format(sock))
diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py
--- a/Lib/test/test_asyncio/test_base_events.py
+++ b/Lib/test/test_asyncio/test_base_events.py
@@ -116,6 +116,13 @@
         self.assertIsNone(
             base_events._ipaddr_info('::3%lo0', 1, INET6, STREAM, TCP))
 
+        if hasattr(socket, 'SOCK_NONBLOCK'):
+            self.assertEqual(
+                None,
+                base_events._ipaddr_info(
+                    '1.2.3.4', 1, INET, STREAM | socket.SOCK_NONBLOCK, TCP))
+
+
     def test_port_parameter_types(self):
         # Test obscure kinds of arguments for "port".
         INET = socket.AF_INET
@@ -1040,6 +1047,43 @@
             MyProto, 'example.com', 80, sock=object())
         self.assertRaises(ValueError, self.loop.run_until_complete, coro)
 
+    @unittest.skipUnless(hasattr(socket, 'AF_UNIX'), 'no Unix sockets')
+    def test_create_connection_wrong_sock(self):
+        sock = socket.socket(socket.AF_UNIX)
+        with sock:
+            coro = self.loop.create_connection(MyProto, sock=sock)
+            with self.assertRaisesRegex(ValueError,
+                                        'A TCP Stream Socket was expected'):
+                self.loop.run_until_complete(coro)
+
+    @unittest.skipUnless(hasattr(socket, 'AF_UNIX'), 'no Unix sockets')
+    def test_create_server_wrong_sock(self):
+        sock = socket.socket(socket.AF_UNIX)
+        with sock:
+            coro = self.loop.create_server(MyProto, sock=sock)
+            with self.assertRaisesRegex(ValueError,
+                                        'A TCP Stream Socket was expected'):
+                self.loop.run_until_complete(coro)
+
+    @unittest.skipUnless(hasattr(socket, 'SOCK_NONBLOCK'),
+                         'no socket.SOCK_NONBLOCK (linux only)')
+    def test_create_server_stream_bittype(self):
+        sock = socket.socket(
+            socket.AF_INET, socket.SOCK_STREAM | socket.SOCK_NONBLOCK)
+        with sock:
+            coro = self.loop.create_server(lambda: None, sock=sock)
+            srv = self.loop.run_until_complete(coro)
+            srv.close()
+            self.loop.run_until_complete(srv.wait_closed())
+
+    def test_create_datagram_endpoint_wrong_sock(self):
+        sock = socket.socket(socket.AF_INET)
+        with sock:
+            coro = self.loop.create_datagram_endpoint(MyProto, sock=sock)
+            with self.assertRaisesRegex(ValueError,
+                                        'A UDP Socket was expected'):
+                self.loop.run_until_complete(coro)
+
     def test_create_connection_no_host_port_sock(self):
         coro = self.loop.create_connection(MyProto)
         self.assertRaises(ValueError, self.loop.run_until_complete, coro)
@@ -1487,36 +1531,39 @@
         self.assertEqual('CLOSED', protocol.state)
 
     def test_create_datagram_endpoint_sock_sockopts(self):
+        class FakeSock:
+            type = socket.SOCK_DGRAM
+
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, local_addr=('127.0.0.1', 0), sock=object())
+            MyDatagramProto, local_addr=('127.0.0.1', 0), sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, remote_addr=('127.0.0.1', 0), sock=object())
+            MyDatagramProto, remote_addr=('127.0.0.1', 0), sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, family=1, sock=object())
+            MyDatagramProto, family=1, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, proto=1, sock=object())
+            MyDatagramProto, proto=1, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, flags=1, sock=object())
+            MyDatagramProto, flags=1, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, reuse_address=True, sock=object())
+            MyDatagramProto, reuse_address=True, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, reuse_port=True, sock=object())
+            MyDatagramProto, reuse_port=True, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
         fut = self.loop.create_datagram_endpoint(
-            MyDatagramProto, allow_broadcast=True, sock=object())
+            MyDatagramProto, allow_broadcast=True, sock=FakeSock())
         self.assertRaises(ValueError, self.loop.run_until_complete, fut)
 
     def test_create_datagram_endpoint_sockopts(self):
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -791,9 +791,9 @@
         conn, _ = lsock.accept()
         proto = MyProto(loop=loop)
         proto.loop = loop
-        f = loop.create_task(
+        loop.run_until_complete(
             loop.connect_accepted_socket(
-                (lambda : proto), conn, ssl=server_ssl))
+                (lambda: proto), conn, ssl=server_ssl))
         loop.run_forever()
         proto.transport.close()
         lsock.close()
@@ -1377,6 +1377,11 @@
         server.transport.close()
 
     def test_create_datagram_endpoint_sock(self):
+        if (sys.platform == 'win32' and
+                isinstance(self.loop, proactor_events.BaseProactorEventLoop)):
+            raise unittest.SkipTest(
+                'UDP is not supported with proactor event loops')
+
         sock = None
         local_address = ('127.0.0.1', 0)
         infos = self.loop.run_until_complete(
@@ -1394,7 +1399,7 @@
         else:
             assert False, 'Can not create socket.'
 
-        f = self.loop.create_connection(
+        f = self.loop.create_datagram_endpoint(
             lambda: MyDatagramProto(loop=self.loop), sock=sock)
         tr, pr = self.loop.run_until_complete(f)
         self.assertIsInstance(tr, asyncio.Transport)
diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py
--- a/Lib/test/test_asyncio/test_unix_events.py
+++ b/Lib/test/test_asyncio/test_unix_events.py
@@ -280,6 +280,33 @@
                                         'A UNIX Domain Stream.*was expected'):
                 self.loop.run_until_complete(coro)
 
+    def test_create_unix_server_path_dgram(self):
+        sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+        with sock:
+            coro = self.loop.create_unix_server(lambda: None, path=None,
+                                                sock=sock)
+            with self.assertRaisesRegex(ValueError,
+                                        'A UNIX Domain Stream.*was expected'):
+                self.loop.run_until_complete(coro)
+
+    @unittest.skipUnless(hasattr(socket, 'SOCK_NONBLOCK'),
+                         'no socket.SOCK_NONBLOCK (linux only)')
+    def test_create_unix_server_path_stream_bittype(self):
+        sock = socket.socket(
+            socket.AF_UNIX, socket.SOCK_STREAM | socket.SOCK_NONBLOCK)
+        with tempfile.NamedTemporaryFile() as file:
+            fn = file.name
+        try:
+            with sock:
+                sock.bind(fn)
+                coro = self.loop.create_unix_server(lambda: None, path=None,
+                                                    sock=sock)
+                srv = self.loop.run_until_complete(coro)
+                srv.close()
+                self.loop.run_until_complete(srv.wait_closed())
+        finally:
+            os.unlink(fn)
+
     def test_create_unix_connection_path_inetsock(self):
         sock = socket.socket()
         with sock:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 15:59:34 2016
From: python-checkins at python.org (steve.dower)
Date: Wed, 09 Nov 2016 20:59:34 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzE5NzE3?=
 =?utf-8?q?=3A_Makes_Path=2Eresolve=28=29_succeed_on_paths_that_do_not_exi?=
 =?utf-8?q?st_=28patch_by?=
Message-ID: <20161109205934.6278.98886.7697F24F@psf.io>

https://hg.python.org/cpython/rev/03bbee2b0d28
changeset:   105016:03bbee2b0d28
branch:      3.6
parent:      105014:1273f1a3ddf7
user:        Steve Dower <steve.dower at microsoft.com>
date:        Wed Nov 09 12:58:17 2016 -0800
summary:
  Issue #19717: Makes Path.resolve() succeed on paths that do not exist (patch by Vajrasky Kok)

files:
  Doc/library/pathlib.rst  |  12 ++++--
  Lib/pathlib.py           |  29 +++++++++++---
  Lib/test/test_pathlib.py |  54 ++++++++++++++++++++++++---
  Misc/NEWS                |   3 +
  4 files changed, 81 insertions(+), 17 deletions(-)


diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst
--- a/Doc/library/pathlib.rst
+++ b/Doc/library/pathlib.rst
@@ -919,7 +919,7 @@
    to an existing file or directory, it will be unconditionally replaced.
 
 
-.. method:: Path.resolve()
+.. method:: Path.resolve(strict=False)
 
    Make the path absolute, resolving any symlinks.  A new path object is
    returned::
@@ -936,10 +936,14 @@
       >>> p.resolve()
       PosixPath('/home/antoine/pathlib/setup.py')
 
-   If the path doesn't exist, :exc:`FileNotFoundError` is raised.  If an
-   infinite loop is encountered along the resolution path,
-   :exc:`RuntimeError` is raised.
+   If the path doesn't exist and *strict* is ``True``, :exc:`FileNotFoundError`
+   is raised.  If *strict* is ``False``, the path is resolved as far as possible
+   and any remainder is appended without checking whether it exists.  If an
+   infinite loop is encountered along the resolution path, :exc:`RuntimeError`
+   is raised.
 
+   .. versionadded:: 3.6
+      The *strict* argument.
 
 .. method:: Path.rglob(pattern)
 
diff --git a/Lib/pathlib.py b/Lib/pathlib.py
--- a/Lib/pathlib.py
+++ b/Lib/pathlib.py
@@ -178,12 +178,26 @@
     def casefold_parts(self, parts):
         return [p.lower() for p in parts]
 
-    def resolve(self, path):
+    def resolve(self, path, strict=False):
         s = str(path)
         if not s:
             return os.getcwd()
+        previous_s = None
         if _getfinalpathname is not None:
-            return self._ext_to_normal(_getfinalpathname(s))
+            if strict:
+                return self._ext_to_normal(_getfinalpathname(s))
+            else:
+                while True:
+                    try:
+                        s = self._ext_to_normal(_getfinalpathname(s))
+                    except FileNotFoundError:
+                        previous_s = s
+                        s = os.path.abspath(os.path.join(s, os.pardir))
+                    else:
+                        if previous_s is None:
+                            return s
+                        else:
+                            return s + os.path.sep + os.path.basename(previous_s)
         # Means fallback on absolute
         return None
 
@@ -285,7 +299,7 @@
     def casefold_parts(self, parts):
         return parts
 
-    def resolve(self, path):
+    def resolve(self, path, strict=False):
         sep = self.sep
         accessor = path._accessor
         seen = {}
@@ -315,7 +329,10 @@
                     target = accessor.readlink(newpath)
                 except OSError as e:
                     if e.errno != EINVAL:
-                        raise
+                        if strict:
+                            raise
+                        else:
+                            return newpath
                     # Not a symlink
                     path = newpath
                 else:
@@ -1092,7 +1109,7 @@
         obj._init(template=self)
         return obj
 
-    def resolve(self):
+    def resolve(self, strict=False):
         """
         Make the path absolute, resolving all symlinks on the way and also
         normalizing it (for example turning slashes into backslashes under
@@ -1100,7 +1117,7 @@
         """
         if self._closed:
             self._raise_closed()
-        s = self._flavour.resolve(self)
+        s = self._flavour.resolve(self, strict=strict)
         if s is None:
             # No symlink resolution => for consistency, raise an error if
             # the path doesn't exist or is forbidden
diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py
--- a/Lib/test/test_pathlib.py
+++ b/Lib/test/test_pathlib.py
@@ -1486,8 +1486,8 @@
         self.assertEqual(set(p.glob("../xyzzy")), set())
 
 
-    def _check_resolve(self, p, expected):
-        q = p.resolve()
+    def _check_resolve(self, p, expected, strict=True):
+        q = p.resolve(strict)
         self.assertEqual(q, expected)
 
     # this can be used to check both relative and absolute resolutions
@@ -1498,8 +1498,17 @@
         P = self.cls
         p = P(BASE, 'foo')
         with self.assertRaises(OSError) as cm:
-            p.resolve()
+            p.resolve(strict=True)
         self.assertEqual(cm.exception.errno, errno.ENOENT)
+        # Non-strict
+        self.assertEqual(str(p.resolve(strict=False)),
+                         os.path.join(BASE, 'foo'))
+        p = P(BASE, 'foo', 'in', 'spam')
+        self.assertEqual(str(p.resolve(strict=False)),
+                         os.path.join(BASE, 'foo'))
+        p = P(BASE, '..', 'foo', 'in', 'spam')
+        self.assertEqual(str(p.resolve(strict=False)),
+                         os.path.abspath(os.path.join('foo')))
         # These are all relative symlinks
         p = P(BASE, 'dirB', 'fileB')
         self._check_resolve_relative(p, p)
@@ -1509,6 +1518,18 @@
         self._check_resolve_relative(p, P(BASE, 'dirB', 'fileB'))
         p = P(BASE, 'dirB', 'linkD', 'fileB')
         self._check_resolve_relative(p, P(BASE, 'dirB', 'fileB'))
+        # Non-strict
+        p = P(BASE, 'dirA', 'linkC', 'fileB', 'foo', 'in', 'spam')
+        self._check_resolve_relative(p, P(BASE, 'dirB', 'fileB', 'foo'), False)
+        p = P(BASE, 'dirA', 'linkC', '..', 'foo', 'in', 'spam')
+        if os.name == 'nt':
+            # In Windows, if linkY points to dirB, 'dirA\linkY\..'
+            # resolves to 'dirA' without resolving linkY first.
+            self._check_resolve_relative(p, P(BASE, 'dirA', 'foo'), False)
+        else:
+            # In Posix, if linkY points to dirB, 'dirA/linkY/..'
+            # resolves to 'dirB/..' first before resolving to parent of dirB.
+            self._check_resolve_relative(p, P(BASE, 'foo'), False)
         # Now create absolute symlinks
         d = tempfile.mkdtemp(suffix='-dirD')
         self.addCleanup(support.rmtree, d)
@@ -1516,6 +1537,18 @@
         os.symlink(join('dirB'), os.path.join(d, 'linkY'))
         p = P(BASE, 'dirA', 'linkX', 'linkY', 'fileB')
         self._check_resolve_absolute(p, P(BASE, 'dirB', 'fileB'))
+        # Non-strict
+        p = P(BASE, 'dirA', 'linkX', 'linkY', 'foo', 'in', 'spam')
+        self._check_resolve_relative(p, P(BASE, 'dirB', 'foo'), False)
+        p = P(BASE, 'dirA', 'linkX', 'linkY', '..', 'foo', 'in', 'spam')
+        if os.name == 'nt':
+            # In Windows, if linkY points to dirB, 'dirA\linkY\..'
+            # resolves to 'dirA' without resolving linkY first.
+            self._check_resolve_relative(p, P(d, 'foo'), False)
+        else:
+            # In Posix, if linkY points to dirB, 'dirA/linkY/..'
+            # resolves to 'dirB/..' first before resolving to parent of dirB.
+            self._check_resolve_relative(p, P(BASE, 'foo'), False)
 
     @with_symlinks
     def test_resolve_dot(self):
@@ -1525,7 +1558,11 @@
         self.dirlink(os.path.join('0', '0'), join('1'))
         self.dirlink(os.path.join('1', '1'), join('2'))
         q = p / '2'
-        self.assertEqual(q.resolve(), p)
+        self.assertEqual(q.resolve(strict=True), p)
+        r = q / '3' / '4'
+        self.assertRaises(FileNotFoundError, r.resolve, strict=True)
+        # Non-strict
+        self.assertEqual(r.resolve(strict=False), p / '3')
 
     def test_with(self):
         p = self.cls(BASE)
@@ -1972,10 +2009,10 @@
 class PosixPathTest(_BasePathTest, unittest.TestCase):
     cls = pathlib.PosixPath
 
-    def _check_symlink_loop(self, *args):
+    def _check_symlink_loop(self, *args, strict=True):
         path = self.cls(*args)
         with self.assertRaises(RuntimeError):
-            print(path.resolve())
+            print(path.resolve(strict))
 
     def test_open_mode(self):
         old_mask = os.umask(0)
@@ -2008,7 +2045,6 @@
 
     @with_symlinks
     def test_resolve_loop(self):
-        # Loop detection for broken symlinks under POSIX
         # Loops with relative symlinks
         os.symlink('linkX/inside', join('linkX'))
         self._check_symlink_loop(BASE, 'linkX')
@@ -2016,6 +2052,8 @@
         self._check_symlink_loop(BASE, 'linkY')
         os.symlink('linkZ/../linkZ', join('linkZ'))
         self._check_symlink_loop(BASE, 'linkZ')
+        # Non-strict
+        self._check_symlink_loop(BASE, 'linkZ', 'foo', strict=False)
         # Loops with absolute symlinks
         os.symlink(join('linkU/inside'), join('linkU'))
         self._check_symlink_loop(BASE, 'linkU')
@@ -2023,6 +2061,8 @@
         self._check_symlink_loop(BASE, 'linkV')
         os.symlink(join('linkW/../linkW'), join('linkW'))
         self._check_symlink_loop(BASE, 'linkW')
+        # Non-strict
+        self._check_symlink_loop(BASE, 'linkW', 'foo', strict=False)
 
     def test_glob(self):
         P = self.cls
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -23,6 +23,9 @@
 Library
 -------
 
+- Issue #19717: Makes Path.resolve() succeed on paths that do not exist.
+  Patch by Vajrasky Kok
+
 - Issue #28563: Fixed possible DoS and arbitrary code execution when handle
   plural form selections in the gettext module.  The expression parser now
   supports exact syntax supported by GNU gettext.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 15:59:34 2016
From: python-checkins at python.org (steve.dower)
Date: Wed, 09 Nov 2016 20:59:34 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2319717=3A_Makes_Path=2Eresolve=28=29_succeed_on_?=
 =?utf-8?q?paths_that_do_not_exist_=28patch_by?=
Message-ID: <20161109205934.85695.21003.F596C459@psf.io>

https://hg.python.org/cpython/rev/445415e402be
changeset:   105017:445415e402be
parent:      105015:719da54652c5
parent:      105016:03bbee2b0d28
user:        Steve Dower <steve.dower at microsoft.com>
date:        Wed Nov 09 12:58:31 2016 -0800
summary:
  Issue #19717: Makes Path.resolve() succeed on paths that do not exist (patch by Vajrasky Kok)

files:
  Doc/library/pathlib.rst  |  12 ++++--
  Lib/pathlib.py           |  29 +++++++++++---
  Lib/test/test_pathlib.py |  54 ++++++++++++++++++++++++---
  Misc/NEWS                |   3 +
  4 files changed, 81 insertions(+), 17 deletions(-)


diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst
--- a/Doc/library/pathlib.rst
+++ b/Doc/library/pathlib.rst
@@ -919,7 +919,7 @@
    to an existing file or directory, it will be unconditionally replaced.
 
 
-.. method:: Path.resolve()
+.. method:: Path.resolve(strict=False)
 
    Make the path absolute, resolving any symlinks.  A new path object is
    returned::
@@ -936,10 +936,14 @@
       >>> p.resolve()
       PosixPath('/home/antoine/pathlib/setup.py')
 
-   If the path doesn't exist, :exc:`FileNotFoundError` is raised.  If an
-   infinite loop is encountered along the resolution path,
-   :exc:`RuntimeError` is raised.
+   If the path doesn't exist and *strict* is ``True``, :exc:`FileNotFoundError`
+   is raised.  If *strict* is ``False``, the path is resolved as far as possible
+   and any remainder is appended without checking whether it exists.  If an
+   infinite loop is encountered along the resolution path, :exc:`RuntimeError`
+   is raised.
 
+   .. versionadded:: 3.6
+      The *strict* argument.
 
 .. method:: Path.rglob(pattern)
 
diff --git a/Lib/pathlib.py b/Lib/pathlib.py
--- a/Lib/pathlib.py
+++ b/Lib/pathlib.py
@@ -178,12 +178,26 @@
     def casefold_parts(self, parts):
         return [p.lower() for p in parts]
 
-    def resolve(self, path):
+    def resolve(self, path, strict=False):
         s = str(path)
         if not s:
             return os.getcwd()
+        previous_s = None
         if _getfinalpathname is not None:
-            return self._ext_to_normal(_getfinalpathname(s))
+            if strict:
+                return self._ext_to_normal(_getfinalpathname(s))
+            else:
+                while True:
+                    try:
+                        s = self._ext_to_normal(_getfinalpathname(s))
+                    except FileNotFoundError:
+                        previous_s = s
+                        s = os.path.abspath(os.path.join(s, os.pardir))
+                    else:
+                        if previous_s is None:
+                            return s
+                        else:
+                            return s + os.path.sep + os.path.basename(previous_s)
         # Means fallback on absolute
         return None
 
@@ -285,7 +299,7 @@
     def casefold_parts(self, parts):
         return parts
 
-    def resolve(self, path):
+    def resolve(self, path, strict=False):
         sep = self.sep
         accessor = path._accessor
         seen = {}
@@ -315,7 +329,10 @@
                     target = accessor.readlink(newpath)
                 except OSError as e:
                     if e.errno != EINVAL:
-                        raise
+                        if strict:
+                            raise
+                        else:
+                            return newpath
                     # Not a symlink
                     path = newpath
                 else:
@@ -1092,7 +1109,7 @@
         obj._init(template=self)
         return obj
 
-    def resolve(self):
+    def resolve(self, strict=False):
         """
         Make the path absolute, resolving all symlinks on the way and also
         normalizing it (for example turning slashes into backslashes under
@@ -1100,7 +1117,7 @@
         """
         if self._closed:
             self._raise_closed()
-        s = self._flavour.resolve(self)
+        s = self._flavour.resolve(self, strict=strict)
         if s is None:
             # No symlink resolution => for consistency, raise an error if
             # the path doesn't exist or is forbidden
diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py
--- a/Lib/test/test_pathlib.py
+++ b/Lib/test/test_pathlib.py
@@ -1486,8 +1486,8 @@
         self.assertEqual(set(p.glob("../xyzzy")), set())
 
 
-    def _check_resolve(self, p, expected):
-        q = p.resolve()
+    def _check_resolve(self, p, expected, strict=True):
+        q = p.resolve(strict)
         self.assertEqual(q, expected)
 
     # this can be used to check both relative and absolute resolutions
@@ -1498,8 +1498,17 @@
         P = self.cls
         p = P(BASE, 'foo')
         with self.assertRaises(OSError) as cm:
-            p.resolve()
+            p.resolve(strict=True)
         self.assertEqual(cm.exception.errno, errno.ENOENT)
+        # Non-strict
+        self.assertEqual(str(p.resolve(strict=False)),
+                         os.path.join(BASE, 'foo'))
+        p = P(BASE, 'foo', 'in', 'spam')
+        self.assertEqual(str(p.resolve(strict=False)),
+                         os.path.join(BASE, 'foo'))
+        p = P(BASE, '..', 'foo', 'in', 'spam')
+        self.assertEqual(str(p.resolve(strict=False)),
+                         os.path.abspath(os.path.join('foo')))
         # These are all relative symlinks
         p = P(BASE, 'dirB', 'fileB')
         self._check_resolve_relative(p, p)
@@ -1509,6 +1518,18 @@
         self._check_resolve_relative(p, P(BASE, 'dirB', 'fileB'))
         p = P(BASE, 'dirB', 'linkD', 'fileB')
         self._check_resolve_relative(p, P(BASE, 'dirB', 'fileB'))
+        # Non-strict
+        p = P(BASE, 'dirA', 'linkC', 'fileB', 'foo', 'in', 'spam')
+        self._check_resolve_relative(p, P(BASE, 'dirB', 'fileB', 'foo'), False)
+        p = P(BASE, 'dirA', 'linkC', '..', 'foo', 'in', 'spam')
+        if os.name == 'nt':
+            # In Windows, if linkY points to dirB, 'dirA\linkY\..'
+            # resolves to 'dirA' without resolving linkY first.
+            self._check_resolve_relative(p, P(BASE, 'dirA', 'foo'), False)
+        else:
+            # In Posix, if linkY points to dirB, 'dirA/linkY/..'
+            # resolves to 'dirB/..' first before resolving to parent of dirB.
+            self._check_resolve_relative(p, P(BASE, 'foo'), False)
         # Now create absolute symlinks
         d = tempfile.mkdtemp(suffix='-dirD')
         self.addCleanup(support.rmtree, d)
@@ -1516,6 +1537,18 @@
         os.symlink(join('dirB'), os.path.join(d, 'linkY'))
         p = P(BASE, 'dirA', 'linkX', 'linkY', 'fileB')
         self._check_resolve_absolute(p, P(BASE, 'dirB', 'fileB'))
+        # Non-strict
+        p = P(BASE, 'dirA', 'linkX', 'linkY', 'foo', 'in', 'spam')
+        self._check_resolve_relative(p, P(BASE, 'dirB', 'foo'), False)
+        p = P(BASE, 'dirA', 'linkX', 'linkY', '..', 'foo', 'in', 'spam')
+        if os.name == 'nt':
+            # In Windows, if linkY points to dirB, 'dirA\linkY\..'
+            # resolves to 'dirA' without resolving linkY first.
+            self._check_resolve_relative(p, P(d, 'foo'), False)
+        else:
+            # In Posix, if linkY points to dirB, 'dirA/linkY/..'
+            # resolves to 'dirB/..' first before resolving to parent of dirB.
+            self._check_resolve_relative(p, P(BASE, 'foo'), False)
 
     @with_symlinks
     def test_resolve_dot(self):
@@ -1525,7 +1558,11 @@
         self.dirlink(os.path.join('0', '0'), join('1'))
         self.dirlink(os.path.join('1', '1'), join('2'))
         q = p / '2'
-        self.assertEqual(q.resolve(), p)
+        self.assertEqual(q.resolve(strict=True), p)
+        r = q / '3' / '4'
+        self.assertRaises(FileNotFoundError, r.resolve, strict=True)
+        # Non-strict
+        self.assertEqual(r.resolve(strict=False), p / '3')
 
     def test_with(self):
         p = self.cls(BASE)
@@ -1972,10 +2009,10 @@
 class PosixPathTest(_BasePathTest, unittest.TestCase):
     cls = pathlib.PosixPath
 
-    def _check_symlink_loop(self, *args):
+    def _check_symlink_loop(self, *args, strict=True):
         path = self.cls(*args)
         with self.assertRaises(RuntimeError):
-            print(path.resolve())
+            print(path.resolve(strict))
 
     def test_open_mode(self):
         old_mask = os.umask(0)
@@ -2008,7 +2045,6 @@
 
     @with_symlinks
     def test_resolve_loop(self):
-        # Loop detection for broken symlinks under POSIX
         # Loops with relative symlinks
         os.symlink('linkX/inside', join('linkX'))
         self._check_symlink_loop(BASE, 'linkX')
@@ -2016,6 +2052,8 @@
         self._check_symlink_loop(BASE, 'linkY')
         os.symlink('linkZ/../linkZ', join('linkZ'))
         self._check_symlink_loop(BASE, 'linkZ')
+        # Non-strict
+        self._check_symlink_loop(BASE, 'linkZ', 'foo', strict=False)
         # Loops with absolute symlinks
         os.symlink(join('linkU/inside'), join('linkU'))
         self._check_symlink_loop(BASE, 'linkU')
@@ -2023,6 +2061,8 @@
         self._check_symlink_loop(BASE, 'linkV')
         os.symlink(join('linkW/../linkW'), join('linkW'))
         self._check_symlink_loop(BASE, 'linkW')
+        # Non-strict
+        self._check_symlink_loop(BASE, 'linkW', 'foo', strict=False)
 
     def test_glob(self):
         P = self.cls
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -114,6 +114,9 @@
 Library
 -------
 
+- Issue #19717: Makes Path.resolve() succeed on paths that do not exist.
+  Patch by Vajrasky Kok
+
 - Issue #28563: Fixed possible DoS and arbitrary code execution when handle
   plural form selections in the gettext module.  The expression parser now
   supports exact syntax supported by GNU gettext.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 16:13:52 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Wed, 09 Nov 2016 21:13:52 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NTU2?=
 =?utf-8?q?=3A_More_typing=2Epy_updates_from_upstream=2E?=
Message-ID: <20161109211349.20451.13011.AC196057@psf.io>

https://hg.python.org/cpython/rev/9c0df5f51baa
changeset:   105018:9c0df5f51baa
branch:      3.5
parent:      105013:85022978d900
user:        Guido van Rossum <guido at python.org>
date:        Wed Nov 09 13:12:51 2016 -0800
summary:
  Issue #28556: More typing.py updates from upstream.

files:
  Lib/test/test_typing.py |   63 +++++--
  Lib/typing.py           |  222 ++++++++++-----------------
  2 files changed, 127 insertions(+), 158 deletions(-)


diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -378,6 +378,16 @@
         with self.assertRaises(TypeError):
             type(c)()
 
+    def test_callable_wrong_forms(self):
+        with self.assertRaises(TypeError):
+            Callable[[...], int]
+        with self.assertRaises(TypeError):
+            Callable[(), int]
+        with self.assertRaises(TypeError):
+            Callable[[()], int]
+        with self.assertRaises(TypeError):
+            Callable[[int, 1], 2]
+
     def test_callable_instance_works(self):
         def f():
             pass
@@ -1296,9 +1306,10 @@
 
 PY36_TESTS = """
 from test import ann_module, ann_module2, ann_module3
-from collections import ChainMap
 
-class B:
+class A:
+    y: float
+class B(A):
     x: ClassVar[Optional['B']] = None
     y: int
 class CSub(B):
@@ -1317,6 +1328,15 @@
 gth = get_type_hints
 
 class GetTypeHintTests(BaseTestCase):
+    def test_get_type_hints_from_various_objects(self):
+        # For invalid objects should fail with TypeError (not AttributeError etc).
+        with self.assertRaises(TypeError):
+            gth(123)
+        with self.assertRaises(TypeError):
+            gth('abc')
+        with self.assertRaises(TypeError):
+            gth(None)
+
     @skipUnless(PY36, 'Python 3.6 required')
     def test_get_type_hints_modules(self):
         self.assertEqual(gth(ann_module), {1: 2, 'f': Tuple[int, int], 'x': int, 'y': str})
@@ -1326,18 +1346,15 @@
     @skipUnless(PY36, 'Python 3.6 required')
     def test_get_type_hints_classes(self):
         self.assertEqual(gth(ann_module.C, ann_module.__dict__),
-                         ChainMap({'y': Optional[ann_module.C]}, {}))
-        self.assertEqual(repr(gth(ann_module.j_class)), 'ChainMap({}, {})')
-        self.assertEqual(gth(ann_module.M), ChainMap({'123': 123, 'o': type},
-                                                     {}, {}))
+                         {'y': Optional[ann_module.C]})
+        self.assertIsInstance(gth(ann_module.j_class), dict)
+        self.assertEqual(gth(ann_module.M), {'123': 123, 'o': type})
         self.assertEqual(gth(ann_module.D),
-                         ChainMap({'j': str, 'k': str,
-                                   'y': Optional[ann_module.C]}, {}))
-        self.assertEqual(gth(ann_module.Y), ChainMap({'z': int}, {}))
+                         {'j': str, 'k': str, 'y': Optional[ann_module.C]})
+        self.assertEqual(gth(ann_module.Y), {'z': int})
         self.assertEqual(gth(ann_module.h_class),
-                         ChainMap({}, {'y': Optional[ann_module.C]}, {}))
-        self.assertEqual(gth(ann_module.S), ChainMap({'x': str, 'y': str},
-                                                     {}))
+                         {'y': Optional[ann_module.C]})
+        self.assertEqual(gth(ann_module.S), {'x': str, 'y': str})
         self.assertEqual(gth(ann_module.foo), {'x': int})
 
     @skipUnless(PY36, 'Python 3.6 required')
@@ -1355,20 +1372,34 @@
         class Der(ABase): ...
         self.assertEqual(gth(ABase.meth), {'x': int})
 
+    def test_get_type_hints_for_builins(self):
+        # Should not fail for built-in classes and functions.
+        self.assertEqual(gth(int), {})
+        self.assertEqual(gth(type), {})
+        self.assertEqual(gth(dir), {})
+        self.assertEqual(gth(len), {})
 
     def test_previous_behavior(self):
         def testf(x, y): ...
         testf.__annotations__['x'] = 'int'
         self.assertEqual(gth(testf), {'x': int})
 
+    def test_get_type_hints_for_object_with_annotations(self):
+        class A: ...
+        class B: ...
+        b = B()
+        b.__annotations__ = {'x': 'A'}
+        self.assertEqual(gth(b, locals()), {'x': A})
+
     @skipUnless(PY36, 'Python 3.6 required')
     def test_get_type_hints_ClassVar(self):
+        self.assertEqual(gth(ann_module2.CV, ann_module2.__dict__),
+                         {'var': typing.ClassVar[ann_module2.CV]})
         self.assertEqual(gth(B, globals()),
-                         ChainMap({'y': int, 'x': ClassVar[Optional[B]]}, {}))
+                         {'y': int, 'x': ClassVar[Optional[B]]})
         self.assertEqual(gth(CSub, globals()),
-                         ChainMap({'z': ClassVar[CSub]},
-                                  {'y': int, 'x': ClassVar[Optional[B]]}, {}))
-        self.assertEqual(gth(G), ChainMap({'lst': ClassVar[List[T]]},{},{}))
+                         {'z': ClassVar[CSub], 'y': int, 'x': ClassVar[Optional[B]]})
+        self.assertEqual(gth(G), {'lst': ClassVar[List[T]]})
 
 
 class CollectionsAbcTests(BaseTestCase):
diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -10,8 +10,6 @@
     import collections.abc as collections_abc
 except ImportError:
     import collections as collections_abc  # Fallback for PY3.2.
-if sys.version_info[:2] >= (3, 3):
-    from collections import ChainMap
 
 
 # Please keep __all__ alphabetized within each category.
@@ -1194,14 +1192,12 @@
         # super()._tree_repr() for nice formatting.
         arg_list = []
         for arg in tree[1:]:
-            if arg == ():
-                arg_list.append('[]')
-            elif not isinstance(arg, tuple):
+            if not isinstance(arg, tuple):
                 arg_list.append(_type_repr(arg))
             else:
                 arg_list.append(arg[0]._tree_repr(arg))
-        if len(arg_list) == 2:
-            return repr(tree[0]) + '[%s]' % ', '.join(arg_list)
+        if arg_list[0] == '...':
+            return repr(tree[0]) + '[..., %s]' % arg_list[1]
         return (repr(tree[0]) +
                 '[[%s], %s]' % (', '.join(arg_list[:-1]), arg_list[-1]))
 
@@ -1216,26 +1212,22 @@
             raise TypeError("Callable must be used as "
                             "Callable[[arg, ...], result].")
         args, result = parameters
-        if args is ...:
-            parameters = (..., result)
-        elif args == []:
-            parameters = ((), result)
+        if args is Ellipsis:
+            parameters = (Ellipsis, result)
         else:
             if not isinstance(args, list):
                 raise TypeError("Callable[args, result]: args must be a list."
                                 " Got %.100r." % (args,))
-            parameters = tuple(args) + (result,)
+            parameters = (tuple(args), result)
         return self.__getitem_inner__(parameters)
 
     @_tp_cache
     def __getitem_inner__(self, parameters):
-        *args, result = parameters
+        args, result = parameters
         msg = "Callable[args, result]: result must be a type."
         result = _type_check(result, msg)
-        if args == [...,]:
+        if args is Ellipsis:
             return super().__getitem__((_TypingEllipsis, result))
-        if args == [(),]:
-            return super().__getitem__((_TypingEmpty, result))
         msg = "Callable[[arg, ...], result]: each arg must be a type."
         args = tuple(_type_check(arg, msg) for arg in args)
         parameters = args + (result,)
@@ -1332,7 +1324,11 @@
 
 def _get_defaults(func):
     """Internal helper to extract the default arguments, by name."""
-    code = func.__code__
+    try:
+        code = func.__code__
+    except AttributeError:
+        # Some built-in functions don't have __code__, __defaults__, etc.
+        return {}
     pos_count = code.co_argcount
     arg_names = code.co_varnames
     arg_names = arg_names[:pos_count]
@@ -1346,138 +1342,80 @@
     return res
 
 
-if sys.version_info[:2] >= (3, 3):
-    def get_type_hints(obj, globalns=None, localns=None):
-        """Return type hints for an object.
-
-        This is often the same as obj.__annotations__, but it handles
-        forward references encoded as string literals, and if necessary
-        adds Optional[t] if a default value equal to None is set.
-
-        The argument may be a module, class, method, or function. The annotations
-        are returned as a dictionary, or in the case of a class, a ChainMap of
-        dictionaries.
-
-        TypeError is raised if the argument is not of a type that can contain
-        annotations, and an empty dictionary is returned if no annotations are
-        present.
-
-        BEWARE -- the behavior of globalns and localns is counterintuitive
-        (unless you are familiar with how eval() and exec() work).  The
-        search order is locals first, then globals.
-
-        - If no dict arguments are passed, an attempt is made to use the
-          globals from obj, and these are also used as the locals.  If the
-          object does not appear to have globals, an exception is raised.
-
-        - If one dict argument is passed, it is used for both globals and
-          locals.
-
-        - If two dict arguments are passed, they specify globals and
-          locals, respectively.
-        """
-
-        if getattr(obj, '__no_type_check__', None):
-            return {}
-        if globalns is None:
-            globalns = getattr(obj, '__globals__', {})
-            if localns is None:
-                localns = globalns
-        elif localns is None:
+def get_type_hints(obj, globalns=None, localns=None):
+    """Return type hints for an object.
+
+    This is often the same as obj.__annotations__, but it handles
+    forward references encoded as string literals, and if necessary
+    adds Optional[t] if a default value equal to None is set.
+
+    The argument may be a module, class, method, or function. The annotations
+    are returned as a dictionary. For classes, annotations include also
+    inherited members.
+
+    TypeError is raised if the argument is not of a type that can contain
+    annotations, and an empty dictionary is returned if no annotations are
+    present.
+
+    BEWARE -- the behavior of globalns and localns is counterintuitive
+    (unless you are familiar with how eval() and exec() work).  The
+    search order is locals first, then globals.
+
+    - If no dict arguments are passed, an attempt is made to use the
+      globals from obj, and these are also used as the locals.  If the
+      object does not appear to have globals, an exception is raised.
+
+    - If one dict argument is passed, it is used for both globals and
+      locals.
+
+    - If two dict arguments are passed, they specify globals and
+      locals, respectively.
+    """
+
+    if getattr(obj, '__no_type_check__', None):
+        return {}
+    if globalns is None:
+        globalns = getattr(obj, '__globals__', {})
+        if localns is None:
             localns = globalns
-
-        if (isinstance(obj, types.FunctionType) or
-            isinstance(obj, types.BuiltinFunctionType) or
-            isinstance(obj, types.MethodType)):
-            defaults = _get_defaults(obj)
-            hints = obj.__annotations__
-            for name, value in hints.items():
-                if value is None:
-                    value = type(None)
-                if isinstance(value, str):
-                    value = _ForwardRef(value)
-                value = _eval_type(value, globalns, localns)
-                if name in defaults and defaults[name] is None:
-                    value = Optional[value]
-                hints[name] = value
-            return hints
-
-        if isinstance(obj, types.ModuleType):
-            try:
-                hints = obj.__annotations__
-            except AttributeError:
-                return {}
-            for name, value in hints.items():
+    elif localns is None:
+        localns = globalns
+    # Classes require a special treatment.
+    if isinstance(obj, type):
+        hints = {}
+        for base in reversed(obj.__mro__):
+            ann = base.__dict__.get('__annotations__', {})
+            for name, value in ann.items():
                 if value is None:
                     value = type(None)
                 if isinstance(value, str):
                     value = _ForwardRef(value)
                 value = _eval_type(value, globalns, localns)
                 hints[name] = value
-            return hints
-
-        if isinstance(object, type):
-            cmap = None
-            for base in reversed(obj.__mro__):
-                new_map = collections.ChainMap if cmap is None else cmap.new_child
-                try:
-                    hints = base.__dict__['__annotations__']
-                except KeyError:
-                    cmap = new_map()
-                else:
-                    for name, value in hints.items():
-                        if value is None:
-                            value = type(None)
-                        if isinstance(value, str):
-                            value = _ForwardRef(value)
-                        value = _eval_type(value, globalns, localns)
-                        hints[name] = value
-                    cmap = new_map(hints)
-            return cmap
-
-        raise TypeError('{!r} is not a module, class, method, '
-                        'or function.'.format(obj))
-
-else:
-    def get_type_hints(obj, globalns=None, localns=None):
-        """Return type hints for a function or method object.
-
-        This is often the same as obj.__annotations__, but it handles
-        forward references encoded as string literals, and if necessary
-        adds Optional[t] if a default value equal to None is set.
-
-        BEWARE -- the behavior of globalns and localns is counterintuitive
-        (unless you are familiar with how eval() and exec() work).  The
-        search order is locals first, then globals.
-
-        - If no dict arguments are passed, an attempt is made to use the
-          globals from obj, and these are also used as the locals.  If the
-          object does not appear to have globals, an exception is raised.
-
-        - If one dict argument is passed, it is used for both globals and
-          locals.
-
-        - If two dict arguments are passed, they specify globals and
-          locals, respectively.
-        """
-        if getattr(obj, '__no_type_check__', None):
+        return hints
+    hints = getattr(obj, '__annotations__', None)
+    if hints is None:
+        # Return empty annotations for something that _could_ have them.
+        if (isinstance(obj, types.FunctionType) or
+            isinstance(obj, types.BuiltinFunctionType) or
+            isinstance(obj, types.MethodType) or
+            isinstance(obj, types.ModuleType)):
             return {}
-        if globalns is None:
-            globalns = getattr(obj, '__globals__', {})
-            if localns is None:
-                localns = globalns
-        elif localns is None:
-            localns = globalns
-        defaults = _get_defaults(obj)
-        hints = dict(obj.__annotations__)
-        for name, value in hints.items():
-            if isinstance(value, str):
-                value = _ForwardRef(value)
-            value = _eval_type(value, globalns, localns)
-            if name in defaults and defaults[name] is None:
-                value = Optional[value]
-            hints[name] = value
-        return hints
+        else:
+            raise TypeError('{!r} is not a module, class, method, '
+                            'or function.'.format(obj))
+    defaults = _get_defaults(obj)
+    hints = dict(hints)
+    for name, value in hints.items():
+        if value is None:
+            value = type(None)
+        if isinstance(value, str):
+            value = _ForwardRef(value)
+        value = _eval_type(value, globalns, localns)
+        if name in defaults and defaults[name] is None:
+            value = Optional[value]
+        hints[name] = value
+    return hints
 
 
 def no_type_check(arg):
@@ -2160,7 +2098,7 @@
         pass
 
     @abstractproperty
-    def errors(self) -> str:
+    def errors(self) -> Optional[str]:
         pass
 
     @abstractproperty

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 16:13:52 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Wed, 09 Nov 2016 21:13:52 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328556=3A_More_typing=2Epy_updates_from_upstream=2E_?=
 =?utf-8?b?KDMuNS0+My42KQ==?=
Message-ID: <20161109211349.28438.10338.9E457EA6@psf.io>

https://hg.python.org/cpython/rev/9e65bc305a24
changeset:   105019:9e65bc305a24
branch:      3.6
parent:      105016:03bbee2b0d28
parent:      105018:9c0df5f51baa
user:        Guido van Rossum <guido at python.org>
date:        Wed Nov 09 13:12:53 2016 -0800
summary:
  Issue #28556: More typing.py updates from upstream. (3.5->3.6)

files:
  Lib/test/test_typing.py |   63 +++++--
  Lib/typing.py           |  222 ++++++++++-----------------
  2 files changed, 127 insertions(+), 158 deletions(-)


diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -378,6 +378,16 @@
         with self.assertRaises(TypeError):
             type(c)()
 
+    def test_callable_wrong_forms(self):
+        with self.assertRaises(TypeError):
+            Callable[[...], int]
+        with self.assertRaises(TypeError):
+            Callable[(), int]
+        with self.assertRaises(TypeError):
+            Callable[[()], int]
+        with self.assertRaises(TypeError):
+            Callable[[int, 1], 2]
+
     def test_callable_instance_works(self):
         def f():
             pass
@@ -1296,9 +1306,10 @@
 
 PY36_TESTS = """
 from test import ann_module, ann_module2, ann_module3
-from collections import ChainMap
 
-class B:
+class A:
+    y: float
+class B(A):
     x: ClassVar[Optional['B']] = None
     y: int
 class CSub(B):
@@ -1317,6 +1328,15 @@
 gth = get_type_hints
 
 class GetTypeHintTests(BaseTestCase):
+    def test_get_type_hints_from_various_objects(self):
+        # For invalid objects should fail with TypeError (not AttributeError etc).
+        with self.assertRaises(TypeError):
+            gth(123)
+        with self.assertRaises(TypeError):
+            gth('abc')
+        with self.assertRaises(TypeError):
+            gth(None)
+
     @skipUnless(PY36, 'Python 3.6 required')
     def test_get_type_hints_modules(self):
         self.assertEqual(gth(ann_module), {1: 2, 'f': Tuple[int, int], 'x': int, 'y': str})
@@ -1326,18 +1346,15 @@
     @skipUnless(PY36, 'Python 3.6 required')
     def test_get_type_hints_classes(self):
         self.assertEqual(gth(ann_module.C, ann_module.__dict__),
-                         ChainMap({'y': Optional[ann_module.C]}, {}))
-        self.assertEqual(repr(gth(ann_module.j_class)), 'ChainMap({}, {})')
-        self.assertEqual(gth(ann_module.M), ChainMap({'123': 123, 'o': type},
-                                                     {}, {}))
+                         {'y': Optional[ann_module.C]})
+        self.assertIsInstance(gth(ann_module.j_class), dict)
+        self.assertEqual(gth(ann_module.M), {'123': 123, 'o': type})
         self.assertEqual(gth(ann_module.D),
-                         ChainMap({'j': str, 'k': str,
-                                   'y': Optional[ann_module.C]}, {}))
-        self.assertEqual(gth(ann_module.Y), ChainMap({'z': int}, {}))
+                         {'j': str, 'k': str, 'y': Optional[ann_module.C]})
+        self.assertEqual(gth(ann_module.Y), {'z': int})
         self.assertEqual(gth(ann_module.h_class),
-                         ChainMap({}, {'y': Optional[ann_module.C]}, {}))
-        self.assertEqual(gth(ann_module.S), ChainMap({'x': str, 'y': str},
-                                                     {}))
+                         {'y': Optional[ann_module.C]})
+        self.assertEqual(gth(ann_module.S), {'x': str, 'y': str})
         self.assertEqual(gth(ann_module.foo), {'x': int})
 
     @skipUnless(PY36, 'Python 3.6 required')
@@ -1355,20 +1372,34 @@
         class Der(ABase): ...
         self.assertEqual(gth(ABase.meth), {'x': int})
 
+    def test_get_type_hints_for_builins(self):
+        # Should not fail for built-in classes and functions.
+        self.assertEqual(gth(int), {})
+        self.assertEqual(gth(type), {})
+        self.assertEqual(gth(dir), {})
+        self.assertEqual(gth(len), {})
 
     def test_previous_behavior(self):
         def testf(x, y): ...
         testf.__annotations__['x'] = 'int'
         self.assertEqual(gth(testf), {'x': int})
 
+    def test_get_type_hints_for_object_with_annotations(self):
+        class A: ...
+        class B: ...
+        b = B()
+        b.__annotations__ = {'x': 'A'}
+        self.assertEqual(gth(b, locals()), {'x': A})
+
     @skipUnless(PY36, 'Python 3.6 required')
     def test_get_type_hints_ClassVar(self):
+        self.assertEqual(gth(ann_module2.CV, ann_module2.__dict__),
+                         {'var': typing.ClassVar[ann_module2.CV]})
         self.assertEqual(gth(B, globals()),
-                         ChainMap({'y': int, 'x': ClassVar[Optional[B]]}, {}))
+                         {'y': int, 'x': ClassVar[Optional[B]]})
         self.assertEqual(gth(CSub, globals()),
-                         ChainMap({'z': ClassVar[CSub]},
-                                  {'y': int, 'x': ClassVar[Optional[B]]}, {}))
-        self.assertEqual(gth(G), ChainMap({'lst': ClassVar[List[T]]},{},{}))
+                         {'z': ClassVar[CSub], 'y': int, 'x': ClassVar[Optional[B]]})
+        self.assertEqual(gth(G), {'lst': ClassVar[List[T]]})
 
 
 class CollectionsAbcTests(BaseTestCase):
diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -10,8 +10,6 @@
     import collections.abc as collections_abc
 except ImportError:
     import collections as collections_abc  # Fallback for PY3.2.
-if sys.version_info[:2] >= (3, 3):
-    from collections import ChainMap
 
 
 # Please keep __all__ alphabetized within each category.
@@ -1194,14 +1192,12 @@
         # super()._tree_repr() for nice formatting.
         arg_list = []
         for arg in tree[1:]:
-            if arg == ():
-                arg_list.append('[]')
-            elif not isinstance(arg, tuple):
+            if not isinstance(arg, tuple):
                 arg_list.append(_type_repr(arg))
             else:
                 arg_list.append(arg[0]._tree_repr(arg))
-        if len(arg_list) == 2:
-            return repr(tree[0]) + '[%s]' % ', '.join(arg_list)
+        if arg_list[0] == '...':
+            return repr(tree[0]) + '[..., %s]' % arg_list[1]
         return (repr(tree[0]) +
                 '[[%s], %s]' % (', '.join(arg_list[:-1]), arg_list[-1]))
 
@@ -1216,26 +1212,22 @@
             raise TypeError("Callable must be used as "
                             "Callable[[arg, ...], result].")
         args, result = parameters
-        if args is ...:
-            parameters = (..., result)
-        elif args == []:
-            parameters = ((), result)
+        if args is Ellipsis:
+            parameters = (Ellipsis, result)
         else:
             if not isinstance(args, list):
                 raise TypeError("Callable[args, result]: args must be a list."
                                 " Got %.100r." % (args,))
-            parameters = tuple(args) + (result,)
+            parameters = (tuple(args), result)
         return self.__getitem_inner__(parameters)
 
     @_tp_cache
     def __getitem_inner__(self, parameters):
-        *args, result = parameters
+        args, result = parameters
         msg = "Callable[args, result]: result must be a type."
         result = _type_check(result, msg)
-        if args == [...,]:
+        if args is Ellipsis:
             return super().__getitem__((_TypingEllipsis, result))
-        if args == [(),]:
-            return super().__getitem__((_TypingEmpty, result))
         msg = "Callable[[arg, ...], result]: each arg must be a type."
         args = tuple(_type_check(arg, msg) for arg in args)
         parameters = args + (result,)
@@ -1332,7 +1324,11 @@
 
 def _get_defaults(func):
     """Internal helper to extract the default arguments, by name."""
-    code = func.__code__
+    try:
+        code = func.__code__
+    except AttributeError:
+        # Some built-in functions don't have __code__, __defaults__, etc.
+        return {}
     pos_count = code.co_argcount
     arg_names = code.co_varnames
     arg_names = arg_names[:pos_count]
@@ -1346,138 +1342,80 @@
     return res
 
 
-if sys.version_info[:2] >= (3, 3):
-    def get_type_hints(obj, globalns=None, localns=None):
-        """Return type hints for an object.
-
-        This is often the same as obj.__annotations__, but it handles
-        forward references encoded as string literals, and if necessary
-        adds Optional[t] if a default value equal to None is set.
-
-        The argument may be a module, class, method, or function. The annotations
-        are returned as a dictionary, or in the case of a class, a ChainMap of
-        dictionaries.
-
-        TypeError is raised if the argument is not of a type that can contain
-        annotations, and an empty dictionary is returned if no annotations are
-        present.
-
-        BEWARE -- the behavior of globalns and localns is counterintuitive
-        (unless you are familiar with how eval() and exec() work).  The
-        search order is locals first, then globals.
-
-        - If no dict arguments are passed, an attempt is made to use the
-          globals from obj, and these are also used as the locals.  If the
-          object does not appear to have globals, an exception is raised.
-
-        - If one dict argument is passed, it is used for both globals and
-          locals.
-
-        - If two dict arguments are passed, they specify globals and
-          locals, respectively.
-        """
-
-        if getattr(obj, '__no_type_check__', None):
-            return {}
-        if globalns is None:
-            globalns = getattr(obj, '__globals__', {})
-            if localns is None:
-                localns = globalns
-        elif localns is None:
+def get_type_hints(obj, globalns=None, localns=None):
+    """Return type hints for an object.
+
+    This is often the same as obj.__annotations__, but it handles
+    forward references encoded as string literals, and if necessary
+    adds Optional[t] if a default value equal to None is set.
+
+    The argument may be a module, class, method, or function. The annotations
+    are returned as a dictionary. For classes, annotations include also
+    inherited members.
+
+    TypeError is raised if the argument is not of a type that can contain
+    annotations, and an empty dictionary is returned if no annotations are
+    present.
+
+    BEWARE -- the behavior of globalns and localns is counterintuitive
+    (unless you are familiar with how eval() and exec() work).  The
+    search order is locals first, then globals.
+
+    - If no dict arguments are passed, an attempt is made to use the
+      globals from obj, and these are also used as the locals.  If the
+      object does not appear to have globals, an exception is raised.
+
+    - If one dict argument is passed, it is used for both globals and
+      locals.
+
+    - If two dict arguments are passed, they specify globals and
+      locals, respectively.
+    """
+
+    if getattr(obj, '__no_type_check__', None):
+        return {}
+    if globalns is None:
+        globalns = getattr(obj, '__globals__', {})
+        if localns is None:
             localns = globalns
-
-        if (isinstance(obj, types.FunctionType) or
-            isinstance(obj, types.BuiltinFunctionType) or
-            isinstance(obj, types.MethodType)):
-            defaults = _get_defaults(obj)
-            hints = obj.__annotations__
-            for name, value in hints.items():
-                if value is None:
-                    value = type(None)
-                if isinstance(value, str):
-                    value = _ForwardRef(value)
-                value = _eval_type(value, globalns, localns)
-                if name in defaults and defaults[name] is None:
-                    value = Optional[value]
-                hints[name] = value
-            return hints
-
-        if isinstance(obj, types.ModuleType):
-            try:
-                hints = obj.__annotations__
-            except AttributeError:
-                return {}
-            for name, value in hints.items():
+    elif localns is None:
+        localns = globalns
+    # Classes require a special treatment.
+    if isinstance(obj, type):
+        hints = {}
+        for base in reversed(obj.__mro__):
+            ann = base.__dict__.get('__annotations__', {})
+            for name, value in ann.items():
                 if value is None:
                     value = type(None)
                 if isinstance(value, str):
                     value = _ForwardRef(value)
                 value = _eval_type(value, globalns, localns)
                 hints[name] = value
-            return hints
-
-        if isinstance(object, type):
-            cmap = None
-            for base in reversed(obj.__mro__):
-                new_map = collections.ChainMap if cmap is None else cmap.new_child
-                try:
-                    hints = base.__dict__['__annotations__']
-                except KeyError:
-                    cmap = new_map()
-                else:
-                    for name, value in hints.items():
-                        if value is None:
-                            value = type(None)
-                        if isinstance(value, str):
-                            value = _ForwardRef(value)
-                        value = _eval_type(value, globalns, localns)
-                        hints[name] = value
-                    cmap = new_map(hints)
-            return cmap
-
-        raise TypeError('{!r} is not a module, class, method, '
-                        'or function.'.format(obj))
-
-else:
-    def get_type_hints(obj, globalns=None, localns=None):
-        """Return type hints for a function or method object.
-
-        This is often the same as obj.__annotations__, but it handles
-        forward references encoded as string literals, and if necessary
-        adds Optional[t] if a default value equal to None is set.
-
-        BEWARE -- the behavior of globalns and localns is counterintuitive
-        (unless you are familiar with how eval() and exec() work).  The
-        search order is locals first, then globals.
-
-        - If no dict arguments are passed, an attempt is made to use the
-          globals from obj, and these are also used as the locals.  If the
-          object does not appear to have globals, an exception is raised.
-
-        - If one dict argument is passed, it is used for both globals and
-          locals.
-
-        - If two dict arguments are passed, they specify globals and
-          locals, respectively.
-        """
-        if getattr(obj, '__no_type_check__', None):
+        return hints
+    hints = getattr(obj, '__annotations__', None)
+    if hints is None:
+        # Return empty annotations for something that _could_ have them.
+        if (isinstance(obj, types.FunctionType) or
+            isinstance(obj, types.BuiltinFunctionType) or
+            isinstance(obj, types.MethodType) or
+            isinstance(obj, types.ModuleType)):
             return {}
-        if globalns is None:
-            globalns = getattr(obj, '__globals__', {})
-            if localns is None:
-                localns = globalns
-        elif localns is None:
-            localns = globalns
-        defaults = _get_defaults(obj)
-        hints = dict(obj.__annotations__)
-        for name, value in hints.items():
-            if isinstance(value, str):
-                value = _ForwardRef(value)
-            value = _eval_type(value, globalns, localns)
-            if name in defaults and defaults[name] is None:
-                value = Optional[value]
-            hints[name] = value
-        return hints
+        else:
+            raise TypeError('{!r} is not a module, class, method, '
+                            'or function.'.format(obj))
+    defaults = _get_defaults(obj)
+    hints = dict(hints)
+    for name, value in hints.items():
+        if value is None:
+            value = type(None)
+        if isinstance(value, str):
+            value = _ForwardRef(value)
+        value = _eval_type(value, globalns, localns)
+        if name in defaults and defaults[name] is None:
+            value = Optional[value]
+        hints[name] = value
+    return hints
 
 
 def no_type_check(arg):
@@ -2160,7 +2098,7 @@
         pass
 
     @abstractproperty
-    def errors(self) -> str:
+    def errors(self) -> Optional[str]:
         pass
 
     @abstractproperty

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 16:13:52 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Wed, 09 Nov 2016 21:13:52 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328556=3A_More_typing=2Epy_updates_from_upstream?=
 =?utf-8?b?LiAoMy42LT4zLjcp?=
Message-ID: <20161109211349.50020.74708.A67AB203@psf.io>

https://hg.python.org/cpython/rev/00e386ac7b95
changeset:   105020:00e386ac7b95
parent:      105017:445415e402be
parent:      105019:9e65bc305a24
user:        Guido van Rossum <guido at python.org>
date:        Wed Nov 09 13:12:56 2016 -0800
summary:
  Issue #28556: More typing.py updates from upstream. (3.6->3.7)

files:
  Lib/test/test_typing.py |   63 +++++--
  Lib/typing.py           |  222 ++++++++++-----------------
  2 files changed, 127 insertions(+), 158 deletions(-)


diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -378,6 +378,16 @@
         with self.assertRaises(TypeError):
             type(c)()
 
+    def test_callable_wrong_forms(self):
+        with self.assertRaises(TypeError):
+            Callable[[...], int]
+        with self.assertRaises(TypeError):
+            Callable[(), int]
+        with self.assertRaises(TypeError):
+            Callable[[()], int]
+        with self.assertRaises(TypeError):
+            Callable[[int, 1], 2]
+
     def test_callable_instance_works(self):
         def f():
             pass
@@ -1296,9 +1306,10 @@
 
 PY36_TESTS = """
 from test import ann_module, ann_module2, ann_module3
-from collections import ChainMap
 
-class B:
+class A:
+    y: float
+class B(A):
     x: ClassVar[Optional['B']] = None
     y: int
 class CSub(B):
@@ -1317,6 +1328,15 @@
 gth = get_type_hints
 
 class GetTypeHintTests(BaseTestCase):
+    def test_get_type_hints_from_various_objects(self):
+        # For invalid objects should fail with TypeError (not AttributeError etc).
+        with self.assertRaises(TypeError):
+            gth(123)
+        with self.assertRaises(TypeError):
+            gth('abc')
+        with self.assertRaises(TypeError):
+            gth(None)
+
     @skipUnless(PY36, 'Python 3.6 required')
     def test_get_type_hints_modules(self):
         self.assertEqual(gth(ann_module), {1: 2, 'f': Tuple[int, int], 'x': int, 'y': str})
@@ -1326,18 +1346,15 @@
     @skipUnless(PY36, 'Python 3.6 required')
     def test_get_type_hints_classes(self):
         self.assertEqual(gth(ann_module.C, ann_module.__dict__),
-                         ChainMap({'y': Optional[ann_module.C]}, {}))
-        self.assertEqual(repr(gth(ann_module.j_class)), 'ChainMap({}, {})')
-        self.assertEqual(gth(ann_module.M), ChainMap({'123': 123, 'o': type},
-                                                     {}, {}))
+                         {'y': Optional[ann_module.C]})
+        self.assertIsInstance(gth(ann_module.j_class), dict)
+        self.assertEqual(gth(ann_module.M), {'123': 123, 'o': type})
         self.assertEqual(gth(ann_module.D),
-                         ChainMap({'j': str, 'k': str,
-                                   'y': Optional[ann_module.C]}, {}))
-        self.assertEqual(gth(ann_module.Y), ChainMap({'z': int}, {}))
+                         {'j': str, 'k': str, 'y': Optional[ann_module.C]})
+        self.assertEqual(gth(ann_module.Y), {'z': int})
         self.assertEqual(gth(ann_module.h_class),
-                         ChainMap({}, {'y': Optional[ann_module.C]}, {}))
-        self.assertEqual(gth(ann_module.S), ChainMap({'x': str, 'y': str},
-                                                     {}))
+                         {'y': Optional[ann_module.C]})
+        self.assertEqual(gth(ann_module.S), {'x': str, 'y': str})
         self.assertEqual(gth(ann_module.foo), {'x': int})
 
     @skipUnless(PY36, 'Python 3.6 required')
@@ -1355,20 +1372,34 @@
         class Der(ABase): ...
         self.assertEqual(gth(ABase.meth), {'x': int})
 
+    def test_get_type_hints_for_builins(self):
+        # Should not fail for built-in classes and functions.
+        self.assertEqual(gth(int), {})
+        self.assertEqual(gth(type), {})
+        self.assertEqual(gth(dir), {})
+        self.assertEqual(gth(len), {})
 
     def test_previous_behavior(self):
         def testf(x, y): ...
         testf.__annotations__['x'] = 'int'
         self.assertEqual(gth(testf), {'x': int})
 
+    def test_get_type_hints_for_object_with_annotations(self):
+        class A: ...
+        class B: ...
+        b = B()
+        b.__annotations__ = {'x': 'A'}
+        self.assertEqual(gth(b, locals()), {'x': A})
+
     @skipUnless(PY36, 'Python 3.6 required')
     def test_get_type_hints_ClassVar(self):
+        self.assertEqual(gth(ann_module2.CV, ann_module2.__dict__),
+                         {'var': typing.ClassVar[ann_module2.CV]})
         self.assertEqual(gth(B, globals()),
-                         ChainMap({'y': int, 'x': ClassVar[Optional[B]]}, {}))
+                         {'y': int, 'x': ClassVar[Optional[B]]})
         self.assertEqual(gth(CSub, globals()),
-                         ChainMap({'z': ClassVar[CSub]},
-                                  {'y': int, 'x': ClassVar[Optional[B]]}, {}))
-        self.assertEqual(gth(G), ChainMap({'lst': ClassVar[List[T]]},{},{}))
+                         {'z': ClassVar[CSub], 'y': int, 'x': ClassVar[Optional[B]]})
+        self.assertEqual(gth(G), {'lst': ClassVar[List[T]]})
 
 
 class CollectionsAbcTests(BaseTestCase):
diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -10,8 +10,6 @@
     import collections.abc as collections_abc
 except ImportError:
     import collections as collections_abc  # Fallback for PY3.2.
-if sys.version_info[:2] >= (3, 3):
-    from collections import ChainMap
 
 
 # Please keep __all__ alphabetized within each category.
@@ -1194,14 +1192,12 @@
         # super()._tree_repr() for nice formatting.
         arg_list = []
         for arg in tree[1:]:
-            if arg == ():
-                arg_list.append('[]')
-            elif not isinstance(arg, tuple):
+            if not isinstance(arg, tuple):
                 arg_list.append(_type_repr(arg))
             else:
                 arg_list.append(arg[0]._tree_repr(arg))
-        if len(arg_list) == 2:
-            return repr(tree[0]) + '[%s]' % ', '.join(arg_list)
+        if arg_list[0] == '...':
+            return repr(tree[0]) + '[..., %s]' % arg_list[1]
         return (repr(tree[0]) +
                 '[[%s], %s]' % (', '.join(arg_list[:-1]), arg_list[-1]))
 
@@ -1216,26 +1212,22 @@
             raise TypeError("Callable must be used as "
                             "Callable[[arg, ...], result].")
         args, result = parameters
-        if args is ...:
-            parameters = (..., result)
-        elif args == []:
-            parameters = ((), result)
+        if args is Ellipsis:
+            parameters = (Ellipsis, result)
         else:
             if not isinstance(args, list):
                 raise TypeError("Callable[args, result]: args must be a list."
                                 " Got %.100r." % (args,))
-            parameters = tuple(args) + (result,)
+            parameters = (tuple(args), result)
         return self.__getitem_inner__(parameters)
 
     @_tp_cache
     def __getitem_inner__(self, parameters):
-        *args, result = parameters
+        args, result = parameters
         msg = "Callable[args, result]: result must be a type."
         result = _type_check(result, msg)
-        if args == [...,]:
+        if args is Ellipsis:
             return super().__getitem__((_TypingEllipsis, result))
-        if args == [(),]:
-            return super().__getitem__((_TypingEmpty, result))
         msg = "Callable[[arg, ...], result]: each arg must be a type."
         args = tuple(_type_check(arg, msg) for arg in args)
         parameters = args + (result,)
@@ -1332,7 +1324,11 @@
 
 def _get_defaults(func):
     """Internal helper to extract the default arguments, by name."""
-    code = func.__code__
+    try:
+        code = func.__code__
+    except AttributeError:
+        # Some built-in functions don't have __code__, __defaults__, etc.
+        return {}
     pos_count = code.co_argcount
     arg_names = code.co_varnames
     arg_names = arg_names[:pos_count]
@@ -1346,138 +1342,80 @@
     return res
 
 
-if sys.version_info[:2] >= (3, 3):
-    def get_type_hints(obj, globalns=None, localns=None):
-        """Return type hints for an object.
-
-        This is often the same as obj.__annotations__, but it handles
-        forward references encoded as string literals, and if necessary
-        adds Optional[t] if a default value equal to None is set.
-
-        The argument may be a module, class, method, or function. The annotations
-        are returned as a dictionary, or in the case of a class, a ChainMap of
-        dictionaries.
-
-        TypeError is raised if the argument is not of a type that can contain
-        annotations, and an empty dictionary is returned if no annotations are
-        present.
-
-        BEWARE -- the behavior of globalns and localns is counterintuitive
-        (unless you are familiar with how eval() and exec() work).  The
-        search order is locals first, then globals.
-
-        - If no dict arguments are passed, an attempt is made to use the
-          globals from obj, and these are also used as the locals.  If the
-          object does not appear to have globals, an exception is raised.
-
-        - If one dict argument is passed, it is used for both globals and
-          locals.
-
-        - If two dict arguments are passed, they specify globals and
-          locals, respectively.
-        """
-
-        if getattr(obj, '__no_type_check__', None):
-            return {}
-        if globalns is None:
-            globalns = getattr(obj, '__globals__', {})
-            if localns is None:
-                localns = globalns
-        elif localns is None:
+def get_type_hints(obj, globalns=None, localns=None):
+    """Return type hints for an object.
+
+    This is often the same as obj.__annotations__, but it handles
+    forward references encoded as string literals, and if necessary
+    adds Optional[t] if a default value equal to None is set.
+
+    The argument may be a module, class, method, or function. The annotations
+    are returned as a dictionary. For classes, annotations include also
+    inherited members.
+
+    TypeError is raised if the argument is not of a type that can contain
+    annotations, and an empty dictionary is returned if no annotations are
+    present.
+
+    BEWARE -- the behavior of globalns and localns is counterintuitive
+    (unless you are familiar with how eval() and exec() work).  The
+    search order is locals first, then globals.
+
+    - If no dict arguments are passed, an attempt is made to use the
+      globals from obj, and these are also used as the locals.  If the
+      object does not appear to have globals, an exception is raised.
+
+    - If one dict argument is passed, it is used for both globals and
+      locals.
+
+    - If two dict arguments are passed, they specify globals and
+      locals, respectively.
+    """
+
+    if getattr(obj, '__no_type_check__', None):
+        return {}
+    if globalns is None:
+        globalns = getattr(obj, '__globals__', {})
+        if localns is None:
             localns = globalns
-
-        if (isinstance(obj, types.FunctionType) or
-            isinstance(obj, types.BuiltinFunctionType) or
-            isinstance(obj, types.MethodType)):
-            defaults = _get_defaults(obj)
-            hints = obj.__annotations__
-            for name, value in hints.items():
-                if value is None:
-                    value = type(None)
-                if isinstance(value, str):
-                    value = _ForwardRef(value)
-                value = _eval_type(value, globalns, localns)
-                if name in defaults and defaults[name] is None:
-                    value = Optional[value]
-                hints[name] = value
-            return hints
-
-        if isinstance(obj, types.ModuleType):
-            try:
-                hints = obj.__annotations__
-            except AttributeError:
-                return {}
-            for name, value in hints.items():
+    elif localns is None:
+        localns = globalns
+    # Classes require a special treatment.
+    if isinstance(obj, type):
+        hints = {}
+        for base in reversed(obj.__mro__):
+            ann = base.__dict__.get('__annotations__', {})
+            for name, value in ann.items():
                 if value is None:
                     value = type(None)
                 if isinstance(value, str):
                     value = _ForwardRef(value)
                 value = _eval_type(value, globalns, localns)
                 hints[name] = value
-            return hints
-
-        if isinstance(object, type):
-            cmap = None
-            for base in reversed(obj.__mro__):
-                new_map = collections.ChainMap if cmap is None else cmap.new_child
-                try:
-                    hints = base.__dict__['__annotations__']
-                except KeyError:
-                    cmap = new_map()
-                else:
-                    for name, value in hints.items():
-                        if value is None:
-                            value = type(None)
-                        if isinstance(value, str):
-                            value = _ForwardRef(value)
-                        value = _eval_type(value, globalns, localns)
-                        hints[name] = value
-                    cmap = new_map(hints)
-            return cmap
-
-        raise TypeError('{!r} is not a module, class, method, '
-                        'or function.'.format(obj))
-
-else:
-    def get_type_hints(obj, globalns=None, localns=None):
-        """Return type hints for a function or method object.
-
-        This is often the same as obj.__annotations__, but it handles
-        forward references encoded as string literals, and if necessary
-        adds Optional[t] if a default value equal to None is set.
-
-        BEWARE -- the behavior of globalns and localns is counterintuitive
-        (unless you are familiar with how eval() and exec() work).  The
-        search order is locals first, then globals.
-
-        - If no dict arguments are passed, an attempt is made to use the
-          globals from obj, and these are also used as the locals.  If the
-          object does not appear to have globals, an exception is raised.
-
-        - If one dict argument is passed, it is used for both globals and
-          locals.
-
-        - If two dict arguments are passed, they specify globals and
-          locals, respectively.
-        """
-        if getattr(obj, '__no_type_check__', None):
+        return hints
+    hints = getattr(obj, '__annotations__', None)
+    if hints is None:
+        # Return empty annotations for something that _could_ have them.
+        if (isinstance(obj, types.FunctionType) or
+            isinstance(obj, types.BuiltinFunctionType) or
+            isinstance(obj, types.MethodType) or
+            isinstance(obj, types.ModuleType)):
             return {}
-        if globalns is None:
-            globalns = getattr(obj, '__globals__', {})
-            if localns is None:
-                localns = globalns
-        elif localns is None:
-            localns = globalns
-        defaults = _get_defaults(obj)
-        hints = dict(obj.__annotations__)
-        for name, value in hints.items():
-            if isinstance(value, str):
-                value = _ForwardRef(value)
-            value = _eval_type(value, globalns, localns)
-            if name in defaults and defaults[name] is None:
-                value = Optional[value]
-            hints[name] = value
-        return hints
+        else:
+            raise TypeError('{!r} is not a module, class, method, '
+                            'or function.'.format(obj))
+    defaults = _get_defaults(obj)
+    hints = dict(hints)
+    for name, value in hints.items():
+        if value is None:
+            value = type(None)
+        if isinstance(value, str):
+            value = _ForwardRef(value)
+        value = _eval_type(value, globalns, localns)
+        if name in defaults and defaults[name] is None:
+            value = Optional[value]
+        hints[name] = value
+    return hints
 
 
 def no_type_check(arg):
@@ -2160,7 +2098,7 @@
         pass
 
     @abstractproperty
-    def errors(self) -> str:
+    def errors(self) -> Optional[str]:
         pass
 
     @abstractproperty

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 16:19:57 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Wed, 09 Nov 2016 21:19:57 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328649=3A_fix-typing-test-v2=2Ediff_=283=2E5-=3E3=2E6?=
 =?utf-8?q?=29?=
Message-ID: <20161109211957.122359.57391.C469DDBB@psf.io>

https://hg.python.org/cpython/rev/43be7891b1f5
changeset:   105022:43be7891b1f5
branch:      3.6
parent:      105019:9e65bc305a24
parent:      105021:d790078797bd
user:        Guido van Rossum <guido at python.org>
date:        Wed Nov 09 13:19:00 2016 -0800
summary:
  Issue #28649: fix-typing-test-v2.diff (3.5->3.6)

files:
  Lib/test/test_typing.py |  7 ++++---
  1 files changed, 4 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -743,10 +743,11 @@
         self.assertEqual(repr(Callable[[], List[T]][int]).replace('typing.', ''),
                          'Callable[[], List[int]]')
 
-    def test_generic_forvard_ref(self):
-        def foobar(x: List[List['T']]): ...
+    def test_generic_forward_ref(self):
+        def foobar(x: List[List['CC']]): ...
+        class CC: ...
+        self.assertEqual(get_type_hints(foobar, globals(), locals()), {'x': List[List[CC]]})
         T = TypeVar('T')
-        self.assertEqual(get_type_hints(foobar, globals(), locals()), {'x': List[List[T]]})
         def barfoo(x: Tuple[T, ...]): ...
         self.assertIs(get_type_hints(barfoo, globals(), locals())['x'], Tuple[T, ...])
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 16:19:57 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Wed, 09 Nov 2016 21:19:57 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjQ5?=
 =?utf-8?q?=3A_fix-typing-test-v2=2Ediff?=
Message-ID: <20161109211956.122404.9282.2485E7E9@psf.io>

https://hg.python.org/cpython/rev/d790078797bd
changeset:   105021:d790078797bd
branch:      3.5
parent:      105018:9c0df5f51baa
user:        Guido van Rossum <guido at python.org>
date:        Wed Nov 09 13:18:59 2016 -0800
summary:
  Issue #28649: fix-typing-test-v2.diff

files:
  Lib/test/test_typing.py |  7 ++++---
  1 files changed, 4 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -743,10 +743,11 @@
         self.assertEqual(repr(Callable[[], List[T]][int]).replace('typing.', ''),
                          'Callable[[], List[int]]')
 
-    def test_generic_forvard_ref(self):
-        def foobar(x: List[List['T']]): ...
+    def test_generic_forward_ref(self):
+        def foobar(x: List[List['CC']]): ...
+        class CC: ...
+        self.assertEqual(get_type_hints(foobar, globals(), locals()), {'x': List[List[CC]]})
         T = TypeVar('T')
-        self.assertEqual(get_type_hints(foobar, globals(), locals()), {'x': List[List[T]]})
         def barfoo(x: Tuple[T, ...]): ...
         self.assertIs(get_type_hints(barfoo, globals(), locals())['x'], Tuple[T, ...])
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 16:19:57 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Wed, 09 Nov 2016 21:19:57 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328649=3A_fix-typing-test-v2=2Ediff_=283=2E6-=3E?=
 =?utf-8?q?3=2E7=29?=
Message-ID: <20161109211957.122404.54519.B5D8402B@psf.io>

https://hg.python.org/cpython/rev/fd47a9d791b9
changeset:   105023:fd47a9d791b9
parent:      105020:00e386ac7b95
parent:      105022:43be7891b1f5
user:        Guido van Rossum <guido at python.org>
date:        Wed Nov 09 13:19:01 2016 -0800
summary:
  Issue #28649: fix-typing-test-v2.diff (3.6->3.7)

files:
  Lib/test/test_typing.py |  7 ++++---
  1 files changed, 4 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -743,10 +743,11 @@
         self.assertEqual(repr(Callable[[], List[T]][int]).replace('typing.', ''),
                          'Callable[[], List[int]]')
 
-    def test_generic_forvard_ref(self):
-        def foobar(x: List[List['T']]): ...
+    def test_generic_forward_ref(self):
+        def foobar(x: List[List['CC']]): ...
+        class CC: ...
+        self.assertEqual(get_type_hints(foobar, globals(), locals()), {'x': List[List[CC]]})
         T = TypeVar('T')
-        self.assertEqual(get_type_hints(foobar, globals(), locals()), {'x': List[List[T]]})
         def barfoo(x: Tuple[T, ...]): ...
         self.assertIs(get_type_hints(barfoo, globals(), locals())['x'], Tuple[T, ...])
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 16:23:55 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Wed, 09 Nov 2016 21:23:55 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328649=3A_typing-clear-caches=2Epatch_=283=2E5-=3E3=2E?=
 =?utf-8?q?6=29?=
Message-ID: <20161109212355.26426.99394.95310C8F@psf.io>

https://hg.python.org/cpython/rev/bd2ec9965f47
changeset:   105025:bd2ec9965f47
branch:      3.6
parent:      105022:43be7891b1f5
parent:      105024:d920bfa5a71a
user:        Guido van Rossum <guido at python.org>
date:        Wed Nov 09 13:23:05 2016 -0800
summary:
  Issue #28649: typing-clear-caches.patch (3.5->3.6)

files:
  Lib/typing.py |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -611,12 +611,16 @@
                         ("many" if alen > elen else "few", repr(cls), alen, elen))
 
 
+_cleanups = []
+
+
 def _tp_cache(func):
     """ Caching for __getitem__ of generic types with a fallback to
     original function for non-hashable arguments.
     """
 
     cached = functools.lru_cache()(func)
+    _cleanups.append(cached.cache_clear)
     @functools.wraps(func)
     def inner(*args, **kwds):
         try:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 16:23:59 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Wed, 09 Nov 2016 21:23:59 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328649=3A_typing-clear-caches=2Epatch_=283=2E6-?=
 =?utf-8?b?PjMuNyk=?=
Message-ID: <20161109212355.85457.90954.6A08F72E@psf.io>

https://hg.python.org/cpython/rev/08f76f89d199
changeset:   105026:08f76f89d199
parent:      105023:fd47a9d791b9
parent:      105025:bd2ec9965f47
user:        Guido van Rossum <guido at python.org>
date:        Wed Nov 09 13:23:06 2016 -0800
summary:
  Issue #28649: typing-clear-caches.patch (3.6->3.7)

files:
  Lib/typing.py |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -611,12 +611,16 @@
                         ("many" if alen > elen else "few", repr(cls), alen, elen))
 
 
+_cleanups = []
+
+
 def _tp_cache(func):
     """ Caching for __getitem__ of generic types with a fallback to
     original function for non-hashable arguments.
     """
 
     cached = functools.lru_cache()(func)
+    _cleanups.append(cached.cache_clear)
     @functools.wraps(func)
     def inner(*args, **kwds):
         try:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 16:24:00 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Wed, 09 Nov 2016 21:24:00 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjQ5?=
 =?utf-8?q?=3A_typing-clear-caches=2Epatch?=
Message-ID: <20161109212355.111216.66507.ED8AF291@psf.io>

https://hg.python.org/cpython/rev/d920bfa5a71a
changeset:   105024:d920bfa5a71a
branch:      3.5
parent:      105021:d790078797bd
user:        Guido van Rossum <guido at python.org>
date:        Wed Nov 09 13:23:04 2016 -0800
summary:
  Issue #28649: typing-clear-caches.patch

files:
  Lib/typing.py |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -611,12 +611,16 @@
                         ("many" if alen > elen else "few", repr(cls), alen, elen))
 
 
+_cleanups = []
+
+
 def _tp_cache(func):
     """ Caching for __getitem__ of generic types with a fallback to
     original function for non-hashable arguments.
     """
 
     cached = functools.lru_cache()(func)
+    _cleanups.append(cached.cache_clear)
     @functools.wraps(func)
     def inner(*args, **kwds):
         try:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 16:53:22 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 09 Nov 2016 21:53:22 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjQ5?=
 =?utf-8?q?=3A_Clear_the_typing_module_caches_when_search_for_reference_le?=
 =?utf-8?q?aks=2E?=
Message-ID: <20161109215322.111390.63143.AE6AA7D4@psf.io>

https://hg.python.org/cpython/rev/d926b484d33a
changeset:   105027:d926b484d33a
branch:      3.5
parent:      105024:d920bfa5a71a
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 09 23:49:02 2016 +0200
summary:
  Issue #28649: Clear the typing module caches when search for reference leaks.

files:
  Lib/test/regrtest.py |  8 ++++++++
  1 files changed, 8 insertions(+), 0 deletions(-)


diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
--- a/Lib/test/regrtest.py
+++ b/Lib/test/regrtest.py
@@ -1526,6 +1526,14 @@
     else:
         ctypes._reset_cache()
 
+    try:
+        typing = sys.modules['typing']
+    except KeyError:
+        pass
+    else:
+        for f in typing._cleanups:
+            f()
+
     # Collect cyclic trash and read memory statistics immediately after.
     func1 = sys.getallocatedblocks
     func2 = sys.gettotalrefcount

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 16:53:22 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 09 Nov 2016 21:53:22 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328649=3A_Clear_the_typing_module_caches_when_search_f?=
 =?utf-8?q?or_reference_leaks=2E?=
Message-ID: <20161109215322.26627.41980.300268C6@psf.io>

https://hg.python.org/cpython/rev/caf3ceb93307
changeset:   105028:caf3ceb93307
branch:      3.6
parent:      105025:bd2ec9965f47
parent:      105027:d926b484d33a
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 09 23:51:54 2016 +0200
summary:
  Issue #28649: Clear the typing module caches when search for reference leaks.

files:
  Lib/test/libregrtest/refleak.py |  8 ++++++++
  1 files changed, 8 insertions(+), 0 deletions(-)


diff --git a/Lib/test/libregrtest/refleak.py b/Lib/test/libregrtest/refleak.py
--- a/Lib/test/libregrtest/refleak.py
+++ b/Lib/test/libregrtest/refleak.py
@@ -184,6 +184,14 @@
     else:
         ctypes._reset_cache()
 
+    try:
+        typing = sys.modules['typing']
+    except KeyError:
+        pass
+    else:
+        for f in typing._cleanups:
+            f()
+
     # Collect cyclic trash and read memory statistics immediately after.
     func1 = sys.getallocatedblocks
     func2 = sys.gettotalrefcount

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 16:53:22 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 09 Nov 2016 21:53:22 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328649=3A_Clear_the_typing_module_caches_when_se?=
 =?utf-8?q?arch_for_reference_leaks=2E?=
Message-ID: <20161109215322.6181.34983.C887BA0C@psf.io>

https://hg.python.org/cpython/rev/437564294e6c
changeset:   105029:437564294e6c
parent:      105026:08f76f89d199
parent:      105028:caf3ceb93307
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 09 23:52:40 2016 +0200
summary:
  Issue #28649: Clear the typing module caches when search for reference leaks.

files:
  Lib/test/libregrtest/refleak.py |  8 ++++++++
  1 files changed, 8 insertions(+), 0 deletions(-)


diff --git a/Lib/test/libregrtest/refleak.py b/Lib/test/libregrtest/refleak.py
--- a/Lib/test/libregrtest/refleak.py
+++ b/Lib/test/libregrtest/refleak.py
@@ -184,6 +184,14 @@
     else:
         ctypes._reset_cache()
 
+    try:
+        typing = sys.modules['typing']
+    except KeyError:
+        pass
+    else:
+        for f in typing._cleanups:
+            f()
+
     # Collect cyclic trash and read memory statistics immediately after.
     func1 = sys.getallocatedblocks
     func2 = sys.gettotalrefcount

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 18:57:15 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 23:57:15 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjUz?=
 =?utf-8?q?=3A_Fix_a_refleak_in_functools=2Elru=5Fcache=2E?=
Message-ID: <20161109235715.111340.57203.0B5DBA4C@psf.io>

https://hg.python.org/cpython/rev/ba59f3328032
changeset:   105030:ba59f3328032
branch:      3.5
parent:      105027:d926b484d33a
user:        Yury Selivanov <yury at magic.io>
date:        Wed Nov 09 18:55:45 2016 -0500
summary:
  Issue #28653: Fix a refleak in functools.lru_cache.

files:
  Lib/test/test_functools.py |  19 +++++++++++++++++++
  Misc/NEWS                  |   2 ++
  Modules/_functoolsmodule.c |   8 ++++++--
  3 files changed, 27 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py
--- a/Lib/test/test_functools.py
+++ b/Lib/test/test_functools.py
@@ -1162,6 +1162,25 @@
         self.assertEqual(misses, 4)
         self.assertEqual(currsize, 2)
 
+    def test_lru_type_error(self):
+        # Regression test for issue #28653.
+        # lru_cache was leaking when one of the arguments
+        # wasn't cacheable.
+
+        @functools.lru_cache(maxsize=None)
+        def infinite_cache(o):
+            pass
+
+        @functools.lru_cache(maxsize=10)
+        def limited_cache(o):
+            pass
+
+        with self.assertRaises(TypeError):
+            infinite_cache([])
+
+        with self.assertRaises(TypeError):
+            limited_cache([])
+
     def test_lru_with_maxsize_none(self):
         @self.module.lru_cache(maxsize=None)
         def fib(n):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -457,6 +457,8 @@
 
 - Issue #28652: Make loop methods reject socket kinds they do not support.
 
+- Issue #28653: Fix a refleak in functools.lru_cache.
+
 IDLE
 ----
 
diff --git a/Modules/_functoolsmodule.c b/Modules/_functoolsmodule.c
--- a/Modules/_functoolsmodule.c
+++ b/Modules/_functoolsmodule.c
@@ -781,8 +781,10 @@
     if (!key)
         return NULL;
     hash = PyObject_Hash(key);
-    if (hash == -1)
+    if (hash == -1) {
+        Py_DECREF(key);
         return NULL;
+    }
     result = _PyDict_GetItem_KnownHash(self->cache, key, hash);
     if (result) {
         Py_INCREF(result);
@@ -837,8 +839,10 @@
     if (!key)
         return NULL;
     hash = PyObject_Hash(key);
-    if (hash == -1)
+    if (hash == -1) {
+        Py_DECREF(key);
         return NULL;
+    }
     link  = (lru_list_elem *)_PyDict_GetItem_KnownHash(self->cache, key, hash);
     if (link) {
         lru_cache_extricate_link(link);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 18:57:15 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 23:57:15 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E6_=28issue_=2328653=29?=
Message-ID: <20161109235715.2099.98407.A024D6AB@psf.io>

https://hg.python.org/cpython/rev/5b253d641826
changeset:   105031:5b253d641826
branch:      3.6
parent:      105028:caf3ceb93307
parent:      105030:ba59f3328032
user:        Yury Selivanov <yury at magic.io>
date:        Wed Nov 09 18:56:26 2016 -0500
summary:
  Merge 3.6 (issue #28653)

files:
  Lib/test/test_functools.py |  19 +++++++++++++++++++
  Misc/NEWS                  |   2 ++
  Modules/_functoolsmodule.c |   8 ++++++--
  3 files changed, 27 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py
--- a/Lib/test/test_functools.py
+++ b/Lib/test/test_functools.py
@@ -1189,6 +1189,25 @@
         self.assertEqual(misses, 4)
         self.assertEqual(currsize, 2)
 
+    def test_lru_type_error(self):
+        # Regression test for issue #28653.
+        # lru_cache was leaking when one of the arguments
+        # wasn't cacheable.
+
+        @functools.lru_cache(maxsize=None)
+        def infinite_cache(o):
+            pass
+
+        @functools.lru_cache(maxsize=10)
+        def limited_cache(o):
+            pass
+
+        with self.assertRaises(TypeError):
+            infinite_cache([])
+
+        with self.assertRaises(TypeError):
+            limited_cache([])
+
     def test_lru_with_maxsize_none(self):
         @self.module.lru_cache(maxsize=None)
         def fib(n):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -48,6 +48,8 @@
 
 - Issue #28652: Make loop methods reject socket kinds they do not support.
 
+- Issue #28653: Fix a refleak in functools.lru_cache.
+
 Documentation
 -------------
 
diff --git a/Modules/_functoolsmodule.c b/Modules/_functoolsmodule.c
--- a/Modules/_functoolsmodule.c
+++ b/Modules/_functoolsmodule.c
@@ -793,8 +793,10 @@
     if (!key)
         return NULL;
     hash = PyObject_Hash(key);
-    if (hash == -1)
+    if (hash == -1) {
+        Py_DECREF(key);
         return NULL;
+    }
     result = _PyDict_GetItem_KnownHash(self->cache, key, hash);
     if (result) {
         Py_INCREF(result);
@@ -849,8 +851,10 @@
     if (!key)
         return NULL;
     hash = PyObject_Hash(key);
-    if (hash == -1)
+    if (hash == -1) {
+        Py_DECREF(key);
         return NULL;
+    }
     link  = (lru_list_elem *)_PyDict_GetItem_KnownHash(self->cache, key, hash);
     if (link) {
         lru_cache_extricate_link(link);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov  9 18:57:15 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 09 Nov 2016 23:57:15 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg2NTMp?=
Message-ID: <20161109235715.50020.61907.00764454@psf.io>

https://hg.python.org/cpython/rev/784fea019cab
changeset:   105032:784fea019cab
parent:      105029:437564294e6c
parent:      105031:5b253d641826
user:        Yury Selivanov <yury at magic.io>
date:        Wed Nov 09 18:57:00 2016 -0500
summary:
  Merge 3.6 (issue #28653)

files:
  Lib/test/test_functools.py |  19 +++++++++++++++++++
  Modules/_functoolsmodule.c |   8 ++++++--
  2 files changed, 25 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py
--- a/Lib/test/test_functools.py
+++ b/Lib/test/test_functools.py
@@ -1189,6 +1189,25 @@
         self.assertEqual(misses, 4)
         self.assertEqual(currsize, 2)
 
+    def test_lru_type_error(self):
+        # Regression test for issue #28653.
+        # lru_cache was leaking when one of the arguments
+        # wasn't cacheable.
+
+        @functools.lru_cache(maxsize=None)
+        def infinite_cache(o):
+            pass
+
+        @functools.lru_cache(maxsize=10)
+        def limited_cache(o):
+            pass
+
+        with self.assertRaises(TypeError):
+            infinite_cache([])
+
+        with self.assertRaises(TypeError):
+            limited_cache([])
+
     def test_lru_with_maxsize_none(self):
         @self.module.lru_cache(maxsize=None)
         def fib(n):
diff --git a/Modules/_functoolsmodule.c b/Modules/_functoolsmodule.c
--- a/Modules/_functoolsmodule.c
+++ b/Modules/_functoolsmodule.c
@@ -793,8 +793,10 @@
     if (!key)
         return NULL;
     hash = PyObject_Hash(key);
-    if (hash == -1)
+    if (hash == -1) {
+        Py_DECREF(key);
         return NULL;
+    }
     result = _PyDict_GetItem_KnownHash(self->cache, key, hash);
     if (result) {
         Py_INCREF(result);
@@ -849,8 +851,10 @@
     if (!key)
         return NULL;
     hash = PyObject_Hash(key);
-    if (hash == -1)
+    if (hash == -1) {
+        Py_DECREF(key);
         return NULL;
+    }
     link  = (lru_list_elem *)_PyDict_GetItem_KnownHash(self->cache, key, hash);
     if (link) {
         lru_cache_extricate_link(link);

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Thu Nov 10 04:03:03 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Thu, 10 Nov 2016 09:03:03 +0000
Subject: [Python-checkins] Daily reference leaks (784fea019cab): sum=0
Message-ID: <20161110090303.96200.76071.099078B5@psf.io>

results for 784fea019cab on branch "default"
--------------------------------------------

test_collections leaked [-7, 1, 0] memory blocks, sum=-6
test_functools leaked [0, 3, 1] memory blocks, sum=4
test_multiprocessing_forkserver leaked [2, 0, 0] memory blocks, sum=2


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogirGTTI', '--timeout', '7200']

From python-checkins at python.org  Thu Nov 10 11:28:21 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Thu, 10 Nov 2016 16:28:21 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328649=3A_fix_first_issue_with_=5FForwardRef_=28=23327?=
 =?utf-8?b?KSAoMy41LT4zLjYp?=
Message-ID: <20161110162813.6258.64595.2C3B0A36@psf.io>

https://hg.python.org/cpython/rev/249a1f0b2857
changeset:   105034:249a1f0b2857
branch:      3.6
parent:      105031:5b253d641826
parent:      105033:0da2e381ad71
user:        Guido van Rossum <guido at python.org>
date:        Thu Nov 10 08:26:19 2016 -0800
summary:
  Issue #28649: fix first issue with _ForwardRef (#327) (3.5->3.6)

files:
  Lib/typing.py |  13 +++----------
  1 files changed, 3 insertions(+), 10 deletions(-)


diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -201,8 +201,7 @@
     """Wrapper to hold a forward reference."""
 
     __slots__ = ('__forward_arg__', '__forward_code__',
-                 '__forward_evaluated__', '__forward_value__',
-                 '__forward_frame__')
+                 '__forward_evaluated__', '__forward_value__')
 
     def __init__(self, arg):
         super().__init__(arg)
@@ -217,12 +216,6 @@
         self.__forward_code__ = code
         self.__forward_evaluated__ = False
         self.__forward_value__ = None
-        typing_globals = globals()
-        frame = sys._getframe(1)
-        while frame is not None and frame.f_globals is typing_globals:
-            frame = frame.f_back
-        assert frame is not None
-        self.__forward_frame__ = frame
 
     def _eval_type(self, globalns, localns):
         if not self.__forward_evaluated__:
@@ -242,10 +235,10 @@
         if not isinstance(other, _ForwardRef):
             return NotImplemented
         return (self.__forward_arg__ == other.__forward_arg__ and
-                self.__forward_frame__ == other.__forward_frame__)
+                self.__forward_value__ == other.__forward_value__)
 
     def __hash__(self):
-        return hash((self.__forward_arg__, self.__forward_frame__))
+        return hash((self.__forward_arg__, self.__forward_value__))
 
     def __instancecheck__(self, obj):
         raise TypeError("Forward references cannot be used with isinstance().")

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 10 11:28:21 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Thu, 10 Nov 2016 16:28:21 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328649=3A_fix_first_issue_with_=5FForwardRef_=28?=
 =?utf-8?b?IzMyNykgKDMuNi0+My43KQ==?=
Message-ID: <20161110162813.28652.97986.009C6EF6@psf.io>

https://hg.python.org/cpython/rev/304f017462e4
changeset:   105035:304f017462e4
parent:      105032:784fea019cab
parent:      105034:249a1f0b2857
user:        Guido van Rossum <guido at python.org>
date:        Thu Nov 10 08:27:50 2016 -0800
summary:
  Issue #28649: fix first issue with _ForwardRef (#327) (3.6->3.7)

files:
  Lib/typing.py |  13 +++----------
  1 files changed, 3 insertions(+), 10 deletions(-)


diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -201,8 +201,7 @@
     """Wrapper to hold a forward reference."""
 
     __slots__ = ('__forward_arg__', '__forward_code__',
-                 '__forward_evaluated__', '__forward_value__',
-                 '__forward_frame__')
+                 '__forward_evaluated__', '__forward_value__')
 
     def __init__(self, arg):
         super().__init__(arg)
@@ -217,12 +216,6 @@
         self.__forward_code__ = code
         self.__forward_evaluated__ = False
         self.__forward_value__ = None
-        typing_globals = globals()
-        frame = sys._getframe(1)
-        while frame is not None and frame.f_globals is typing_globals:
-            frame = frame.f_back
-        assert frame is not None
-        self.__forward_frame__ = frame
 
     def _eval_type(self, globalns, localns):
         if not self.__forward_evaluated__:
@@ -242,10 +235,10 @@
         if not isinstance(other, _ForwardRef):
             return NotImplemented
         return (self.__forward_arg__ == other.__forward_arg__ and
-                self.__forward_frame__ == other.__forward_frame__)
+                self.__forward_value__ == other.__forward_value__)
 
     def __hash__(self):
-        return hash((self.__forward_arg__, self.__forward_frame__))
+        return hash((self.__forward_arg__, self.__forward_value__))
 
     def __instancecheck__(self, obj):
         raise TypeError("Forward references cannot be used with isinstance().")

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 10 11:28:21 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Thu, 10 Nov 2016 16:28:21 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjQ5?=
 =?utf-8?q?=3A_fix_first_issue_with_=5FForwardRef_=28=23327=29?=
Message-ID: <20161110162813.20291.69829.A22D26AE@psf.io>

https://hg.python.org/cpython/rev/0da2e381ad71
changeset:   105033:0da2e381ad71
branch:      3.5
parent:      105030:ba59f3328032
user:        Guido van Rossum <guido at python.org>
date:        Thu Nov 10 08:24:06 2016 -0800
summary:
  Issue #28649: fix first issue with _ForwardRef (#327)

files:
  Lib/typing.py |  13 +++----------
  1 files changed, 3 insertions(+), 10 deletions(-)


diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -201,8 +201,7 @@
     """Wrapper to hold a forward reference."""
 
     __slots__ = ('__forward_arg__', '__forward_code__',
-                 '__forward_evaluated__', '__forward_value__',
-                 '__forward_frame__')
+                 '__forward_evaluated__', '__forward_value__')
 
     def __init__(self, arg):
         super().__init__(arg)
@@ -217,12 +216,6 @@
         self.__forward_code__ = code
         self.__forward_evaluated__ = False
         self.__forward_value__ = None
-        typing_globals = globals()
-        frame = sys._getframe(1)
-        while frame is not None and frame.f_globals is typing_globals:
-            frame = frame.f_back
-        assert frame is not None
-        self.__forward_frame__ = frame
 
     def _eval_type(self, globalns, localns):
         if not self.__forward_evaluated__:
@@ -242,10 +235,10 @@
         if not isinstance(other, _ForwardRef):
             return NotImplemented
         return (self.__forward_arg__ == other.__forward_arg__ and
-                self.__forward_frame__ == other.__forward_frame__)
+                self.__forward_value__ == other.__forward_value__)
 
     def __hash__(self):
-        return hash((self.__forward_arg__, self.__forward_frame__))
+        return hash((self.__forward_arg__, self.__forward_value__))
 
     def __instancecheck__(self, obj):
         raise TypeError("Forward references cannot be used with isinstance().")

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 10 11:29:45 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Thu, 10 Nov 2016 16:29:45 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328649=3A_fix_second_issue_with_=5FForwardRef_=28=2332?=
 =?utf-8?b?OCkgKDMuNS0+My42KQ==?=
Message-ID: <20161110162944.96124.37676.E263BA80@psf.io>

https://hg.python.org/cpython/rev/0f863906cf2e
changeset:   105037:0f863906cf2e
branch:      3.6
parent:      105034:249a1f0b2857
parent:      105036:555f0ca31587
user:        Guido van Rossum <guido at python.org>
date:        Thu Nov 10 08:29:19 2016 -0800
summary:
  Issue #28649: fix second issue with _ForwardRef (#328) (3.5->3.6)

files:
  Lib/typing.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -218,7 +218,7 @@
         self.__forward_value__ = None
 
     def _eval_type(self, globalns, localns):
-        if not self.__forward_evaluated__:
+        if not self.__forward_evaluated__ or localns is not globalns:
             if globalns is None and localns is None:
                 globalns = localns = {}
             elif globalns is None:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 10 11:29:45 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Thu, 10 Nov 2016 16:29:45 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjQ5?=
 =?utf-8?q?=3A_fix_second_issue_with_=5FForwardRef_=28=23328=29?=
Message-ID: <20161110162944.96175.56110.328BA843@psf.io>

https://hg.python.org/cpython/rev/555f0ca31587
changeset:   105036:555f0ca31587
branch:      3.5
parent:      105033:0da2e381ad71
user:        Guido van Rossum <guido at python.org>
date:        Thu Nov 10 08:29:18 2016 -0800
summary:
  Issue #28649: fix second issue with _ForwardRef (#328)

files:
  Lib/typing.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -218,7 +218,7 @@
         self.__forward_value__ = None
 
     def _eval_type(self, globalns, localns):
-        if not self.__forward_evaluated__:
+        if not self.__forward_evaluated__ or localns is not globalns:
             if globalns is None and localns is None:
                 globalns = localns = {}
             elif globalns is None:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 10 11:29:51 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Thu, 10 Nov 2016 16:29:51 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328649=3A_fix_second_issue_with_=5FForwardRef_?=
 =?utf-8?b?KCMzMjgpICgzLjYtPjMuNyk=?=
Message-ID: <20161110162944.20549.29340.F84DC628@psf.io>

https://hg.python.org/cpython/rev/84b4ac243508
changeset:   105038:84b4ac243508
parent:      105035:304f017462e4
parent:      105037:0f863906cf2e
user:        Guido van Rossum <guido at python.org>
date:        Thu Nov 10 08:29:20 2016 -0800
summary:
  Issue #28649: fix second issue with _ForwardRef (#328) (3.6->3.7)

files:
  Lib/typing.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -218,7 +218,7 @@
         self.__forward_value__ = None
 
     def _eval_type(self, globalns, localns):
-        if not self.__forward_evaluated__:
+        if not self.__forward_evaluated__ or localns is not globalns:
             if globalns is None and localns is None:
                 globalns = localns = {}
             elif globalns is None:

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Thu Nov 10 11:40:28 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Thu, 10 Nov 2016 16:40:28 +0000
Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7
 2016-11-10
Message-ID: <da31c391-1c85-441b-9299-6739fdbd92aa@irsmsx151.ger.corp.intel.com>

Results for project Python 2.7, build date 2016-11-10 03:48:06 +0000
commit:		41613bb27f80
previous commit:	e0cc3fadd7b3
revision date:	2016-11-09 14:41:15 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.18%          0.97%          4.56%             5.33%
:-)             pybench      0.13%          0.13%          5.94%             2.49%
:-(            regex_v8      0.65%          0.50%         -2.04%            11.08%
:-)               nbody      0.06%          0.22%          8.27%             1.86%
:-)        json_dump_v2      0.30%          0.55%          2.80%             9.27%
:-|      normal_startup      1.20%         -0.59%         -0.20%             2.26%
:-)             ssbench      0.15%          0.82%          2.37%             1.40%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-11-10/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Thu Nov 10 11:41:44 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Thu, 10 Nov 2016 16:41:44 +0000
Subject: [Python-checkins] BAD Benchmark Results for Python Default
 2016-11-10
Message-ID: <18c39edb-5577-4597-a000-8b2ef5e70ffe@irsmsx151.ger.corp.intel.com>

Results for project Python default, build date 2016-11-10 03:01:40 +0000
commit:		784fea019cab
previous commit:	8e3d359cc73b
revision date:	2016-11-09 23:57:00 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.12%          0.58%          6.89%            14.36%
:-)             pybench      0.26%          0.62%          5.79%             4.31%
:-|            regex_v8      2.78%          0.52%         -1.31%             3.08%
:-|               nbody      0.13%         -1.73%          0.85%             5.64%
:-(        json_dump_v2      0.37%          0.67%         -9.00%            12.53%
:-|      normal_startup      0.84%          0.17%         -0.16%             6.44%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-default-2016-11-10/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Thu Nov 10 13:26:11 2016
From: python-checkins at python.org (yury.selivanov)
Date: Thu, 10 Nov 2016 18:26:11 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NjM1?=
 =?utf-8?q?=3A_Fix_a_couple_of_missing/incorrect_versionchanged_tags?=
Message-ID: <20161110182611.124126.10275.3406B9F9@psf.io>

https://hg.python.org/cpython/rev/bcd4ab982429
changeset:   105039:bcd4ab982429
branch:      3.6
parent:      105037:0f863906cf2e
user:        Yury Selivanov <yury at magic.io>
date:        Thu Nov 10 13:25:26 2016 -0500
summary:
  Issue #28635: Fix a couple of missing/incorrect versionchanged tags

Patch by Elvis Pranskevichus.

files:
  Doc/c-api/exceptions.rst |  2 +-
  Doc/library/struct.rst   |  4 ++++
  2 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst
--- a/Doc/c-api/exceptions.rst
+++ b/Doc/c-api/exceptions.rst
@@ -311,7 +311,7 @@
    Much like :c:func:`PyErr_SetImportError` but this function allows for
    specifying a subclass of :exc:`ImportError` to raise.
 
-   .. versionadded:: 3.4
+   .. versionadded:: 3.6
 
 
 .. c:function:: int PyErr_WarnExplicitObject(PyObject *category, PyObject *message, PyObject *filename, int lineno, PyObject *module, PyObject *registry)
diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst
--- a/Doc/library/struct.rst
+++ b/Doc/library/struct.rst
@@ -232,6 +232,10 @@
 .. versionchanged:: 3.3
    Added support for the ``'n'`` and ``'N'`` formats.
 
+.. versionchanged:: 3.6
+   Added support for the ``'e'`` format.
+
+
 Notes:
 
 (1)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 10 13:26:11 2016
From: python-checkins at python.org (yury.selivanov)
Date: Thu, 10 Nov 2016 18:26:11 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg2MzUp?=
Message-ID: <20161110182611.123817.61093.879D9BA5@psf.io>

https://hg.python.org/cpython/rev/5c4ce500dd35
changeset:   105040:5c4ce500dd35
parent:      105038:84b4ac243508
parent:      105039:bcd4ab982429
user:        Yury Selivanov <yury at magic.io>
date:        Thu Nov 10 13:26:06 2016 -0500
summary:
  Merge 3.6 (issue #28635)

files:
  Doc/c-api/exceptions.rst |  2 +-
  Doc/library/struct.rst   |  4 ++++
  2 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst
--- a/Doc/c-api/exceptions.rst
+++ b/Doc/c-api/exceptions.rst
@@ -311,7 +311,7 @@
    Much like :c:func:`PyErr_SetImportError` but this function allows for
    specifying a subclass of :exc:`ImportError` to raise.
 
-   .. versionadded:: 3.4
+   .. versionadded:: 3.6
 
 
 .. c:function:: int PyErr_WarnExplicitObject(PyObject *category, PyObject *message, PyObject *filename, int lineno, PyObject *module, PyObject *registry)
diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst
--- a/Doc/library/struct.rst
+++ b/Doc/library/struct.rst
@@ -232,6 +232,10 @@
 .. versionchanged:: 3.3
    Added support for the ``'n'`` and ``'N'`` formats.
 
+.. versionchanged:: 3.6
+   Added support for the ``'e'`` format.
+
+
 Notes:
 
 (1)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 10 13:28:07 2016
From: python-checkins at python.org (yury.selivanov)
Date: Thu, 10 Nov 2016 18:28:07 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NjM1?=
 =?utf-8?q?=3A_What=27s_New_in_Python_3=2E6_updates?=
Message-ID: <20161110182807.96922.86257.211C8184@psf.io>

https://hg.python.org/cpython/rev/c0060567f35d
changeset:   105041:c0060567f35d
branch:      3.6
parent:      105039:bcd4ab982429
user:        Yury Selivanov <yury at magic.io>
date:        Thu Nov 10 13:27:22 2016 -0500
summary:
  Issue #28635: What's New in Python 3.6 updates

Patch by Elvis Pranskevichus.

files:
  Doc/whatsnew/3.6.rst |  1125 +++++++++++++++++++++++------
  1 files changed, 878 insertions(+), 247 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -87,7 +87,7 @@
 
 * The :ref:`dict <typesmapping>` type has been reimplemented to use
   a :ref:`faster, more compact representation <whatsnew36-compactdict>`
-  similar to the `PyPy dict implementation`_. This resulted in dictionaries
+  similar to the `PyPy dict implementation`_.  This resulted in dictionaries
   using 20% to 25% less memory when compared to Python 3.5.
 
 * Customization of class creation has been simplified with the
@@ -96,27 +96,49 @@
 * The class attibute definition order is
   :ref:`now preserved  <whatsnew36-pep520>`.
 
-* The order of elements in ``**kwargs`` now corresponds to the order in
-  the function signature:
-  :ref:`Preserving Keyword Argument Order <whatsnew36-pep468>`.
-
-* DTrace and SystemTap :ref:`probing support <whatsnew36-tracing>`.
+* The order of elements in ``**kwargs`` now
+  :ref:`corresponds to the order <whatsnew36-pep468>` in which keyword
+  arguments were passed to the function.
+
+* DTrace and SystemTap :ref:`probing support <whatsnew36-tracing>` has
+  been added.
+
+* The new :ref:`PYTHONMALLOC <whatsnew36-pythonmalloc>` environment variable
+  can now be used to debug the interpreter memory allocation and access
+  errors.
 
 
 Significant improvements in the standard library:
 
+* The :mod:`asyncio` module has received new features, significant
+  usability and performance improvements, and a fair amount of bug fixes.
+  Starting with Python 3.6 the ``asyncio`` module is no longer provisional
+  and its API is considered stable.
+
 * A new :ref:`file system path protocol <whatsnew36-pep519>` has been
   implemented to support :term:`path-like objects <path-like object>`.
   All standard library functions operating on paths have been updated to
   work with the new protocol.
 
-* The overhead of :mod:`asyncio` implementation has been reduced by
-  up to 50% thanks to the new C implementation of the :class:`asyncio.Future`
-  and :class:`asyncio.Task` classes and other optimizations.
+* The :mod:`datetime` module has gained support for
+  :ref:`Local Time Disambiguation <whatsnew36-pep495>`.
+
+* The :mod:`typing` module received a number of
+  :ref:`improvements <whatsnew36-typing>` and is no longer provisional.
+
+* The :mod:`tracemalloc` module has been significantly reworked
+  and is now used to provide better output for :exc:`ResourceWarning`s
+  as well as provide better diagnostics for memory allocation errors.
+  See the :ref:`PYTHONMALLOC section <whatsnew36-pythonmalloc>` for more
+  information.
 
 
 Security improvements:
 
+* The new :mod:`secrets` module has been added to simplify the generation of
+  cryptographically strong pseudo-random numbers suitable for
+  managing secrets such as account authentication, tokens, and similar.
+
 * On Linux, :func:`os.urandom` now blocks until the system urandom entropy
   pool is initialized to increase the security. See the :pep:`524` for the
   rationale.
@@ -153,26 +175,6 @@
   more information.
 
 
-A complete list of PEP's implemented in Python 3.6:
-
-* :pep:`468`, :ref:`Preserving Keyword Argument Order <whatsnew36-pep468>`
-* :pep:`487`, :ref:`Simpler customization of class creation <whatsnew36-pep487>`
-* :pep:`495`, Local Time Disambiguation
-* :pep:`498`, :ref:`Formatted string literals <whatsnew36-pep498>`
-* :pep:`506`, Adding A Secrets Module To The Standard Library
-* :pep:`509`, :ref:`Add a private version to dict <whatsnew36-pep509>`
-* :pep:`515`, :ref:`Underscores in Numeric Literals <whatsnew36-pep515>`
-* :pep:`519`, :ref:`Adding a file system path protocol <whatsnew36-pep519>`
-* :pep:`520`, :ref:`Preserving Class Attribute Definition Order <whatsnew36-pep520>`
-* :pep:`523`, :ref:`Adding a frame evaluation API to CPython <whatsnew36-pep523>`
-* :pep:`524`, Make os.urandom() blocking on Linux (during system startup)
-* :pep:`525`, Asynchronous Generators (provisional)
-* :pep:`526`, :ref:`Syntax for Variable Annotations (provisional) <whatsnew36-pep526>`
-* :pep:`528`, :ref:`Change Windows console encoding to UTF-8 (provisional) <whatsnew36-pep528>`
-* :pep:`529`, :ref:`Change Windows filesystem encoding to UTF-8 (provisional) <whatsnew36-pep529>`
-* :pep:`530`, Asynchronous Comprehensions
-
-
 .. _PyPy dict implementation: https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html
 
 
@@ -185,7 +187,7 @@
 ----------------------------------
 
 :pep:`498` introduces a new kind of string literals: *f-strings*, or
-*formatted string literals*.
+:ref:`formatted string literals <f-strings>`.
 
 Formatted string literals are prefixed with ``'f'`` and are similar to
 the format strings accepted by :meth:`str.format`.  They contain replacement
@@ -196,6 +198,11 @@
     >>> name = "Fred"
     >>> f"He said his name is {name}."
     'He said his name is Fred.'
+    >>> width = 10
+    >>> precision = 4
+    >>> value = decimal.Decimal("12.34567")
+    >>> f"result: {value:{width}.{precision}}"  # nested fields
+    'result:      12.35'
 
 .. seealso::
 
@@ -258,6 +265,18 @@
 specifier.  Leading, trailing, or multiple underscores in a row are not
 allowed.
 
+The :ref:`string formatting <formatspec>` language also now has support
+for the ``'_'`` option to signal the use of an underscore for a thousands
+separator for floating point presentation types and for integer
+presentation type ``'d'``.  For integer presentation types ``'b'``,
+``'o'``, ``'x'``, and ``'X'``, underscores will be inserted every 4
+digits::
+
+    >>> '{:_}'.format(1000000)
+    '1_000_000'
+    >>> '{:_x}'.format(0xFFFFFFFF)
+    'ffff_ffff'
+
 .. seealso::
 
    :pep:`515` -- Underscores in Numeric Literals
@@ -302,7 +321,7 @@
 Additionally, ``await`` expressions are supported in all kinds
 of comprehensions::
 
-    result = [await fun() for fun in funcs]
+    result = [await fun() for fun in funcs if await condition()]
 
 .. seealso::
 
@@ -319,17 +338,18 @@
 The new ``__init_subclass__`` classmethod will be called on the base class
 whenever a new subclass is created::
 
-    >>> class QuestBase:
-    ...    # this is implicitly a @classmethod
-    ...    def __init_subclass__(cls, swallow, **kwargs):
-    ...        cls.swallow = swallow
-    ...        super().__init_subclass__(**kwargs)
-
-    >>> class Quest(QuestBase, swallow="african"):
-    ...    pass
-
-    >>> Quest.swallow
-    'african'
+    class PluginBase:
+        subclasses = []
+
+        def __init_subclass__(cls, **kwargs):
+            super().__init_subclass__(**kwargs)
+            cls.subclasses.append(cls)
+
+    class Plugin1(PluginBase):
+        pass
+
+    class Plugin2(PluginBase):
+        pass
 
 .. seealso::
 
@@ -345,10 +365,29 @@
 -----------------------------------------
 
 :pep:`487` extends the descriptor protocol has to include the new optional
-``__set_name__`` method.  Whenever a new class is defined, the new method
-will be called on all descriptors included in the definition, providing
+:meth:`~object.__set_name__` method.  Whenever a new class is defined, the new
+method will be called on all descriptors included in the definition, providing
 them with a reference to the class being defined and the name given to the
-descriptor within the class namespace.
+descriptor within the class namespace.  In other words, instances of
+descriptors can now know the attribute name of the descriptor in the
+owner class::
+
+    class IntField:
+        def __get__(self, instance, owner):
+            return instance.__dict__[self.name]
+
+        def __set__(self, instance, value):
+            if not isinstance(value, int):
+                raise ValueError(f'expecting integer in {self.name}')
+            instance.__dict__[self.name] = value
+
+        # this is the new initializer:
+        def __set_name__(self, owner, name):
+            self.name = name
+
+    class Model:
+        int_field = IntField()
+
 
 .. seealso::
 
@@ -358,20 +397,6 @@
     :ref:`Feature documentation <descriptors>`
 
 
-.. _whatsnew36-pep506:
-
-PEP 506: Adding A Secrets Module To The Standard Library
---------------------------------------------------------
-
-
-
-.. seealso::
-
-    :pep:`506` -- Adding A Secrets Module To The Standard Library
-      PEP written and implemented by Steven D'Aprano.
-
-
-
 .. _whatsnew36-pep519:
 
 PEP 519: Adding a file system path protocol
@@ -401,9 +426,8 @@
 
 The built-in :func:`open` function has been updated to accept
 :class:`os.PathLike` objects as have all relevant functions in the
-:mod:`os` and :mod:`os.path` modules. :c:func:`PyUnicode_FSConverter`
-and :c:func:`PyUnicode_FSConverter` have been changed to accept
-path-like objects. The :class:`os.DirEntry` class
+:mod:`os` and :mod:`os.path` modules, as well as most functions and
+classes in the standard library.  The :class:`os.DirEntry` class
 and relevant classes in :mod:`pathlib` have also been updated to
 implement :class:`os.PathLike`.
 
@@ -439,6 +463,43 @@
        PEP written by Brett Cannon and Koos Zevenhoven.
 
 
+.. _whatsnew36-pep495:
+
+PEP 495: Local Time Disambiguation
+----------------------------------
+
+In most world locations, there have been and will be times when local clocks
+are moved back.  In those times, intervals are introduced in which local
+clocks show the same time twice in the same day. In these situations, the
+information displayed on a local clock (or stored in a Python datetime
+instance) is insufficient to identify a particular moment in time.
+
+:pep:`495` adds the new *fold* attribute to instances of
+:class:`datetime.datetime` and :class:`datetime.time` classes to differentiate
+between two moments in time for which local times are the same::
+
+    >>> u0 = datetime(2016, 11, 6, 4, tzinfo=timezone.utc)
+    >>> for i in range(4):
+    ...     u = u0 + i*HOUR
+    ...     t = u.astimezone(Eastern)
+    ...     print(u.time(), 'UTC =', t.time(), t.tzname(), t.fold)
+    ...
+    04:00:00 UTC = 00:00:00 EDT 0
+    05:00:00 UTC = 01:00:00 EDT 0
+    06:00:00 UTC = 01:00:00 EST 1
+    07:00:00 UTC = 02:00:00 EST 0
+
+The values of the :attr:`fold <datetime.datetime.fold>` attribute have the
+value `0`  all instances except those that represent the second
+(chronologically) moment in time in an ambiguous case.
+
+.. seealso::
+
+  :pep:`495` -- Local Time Disambiguation
+     PEP written by Alexander Belopolsky and Tim Peters, implementation
+     by Alexander Belopolsky.
+
+
 .. _whatsnew36-pep529:
 
 PEP 529: Change Windows filesystem encoding to UTF-8
@@ -489,10 +550,11 @@
 
 Attributes in a class definition body have a natural ordering: the same
 order in which the names appear in the source.  This order is now
-preserved in the new class's ``__dict__`` attribute.
+preserved in the new class's :attr:`~object.__dict__` attribute.
 
 Also, the effective default class *execution* namespace (returned from
-``type.__prepare__()``) is now an insertion-order-preserving mapping.
+:ref:`type.__prepare__() <prepare>`) is now an insertion-order-preserving
+mapping.
 
 .. seealso::
 
@@ -523,16 +585,16 @@
 `pioneered by PyPy <https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html>`_.
 The memory usage of the new :func:`dict` is between 20% and 25% smaller
 compared to Python 3.5.
-:pep:`468` (Preserving the order of ``**kwargs`` in a function.) is
-implemented by this. The order-preserving aspect of this new
-implementation is considered an implementation detail and should
-not be relied upon (this may change in the future, but it is desired
-to have this new dict implementation in the language for a few
-releases before changing the language spec to mandate
+
+The order-preserving aspect of this new implementation is considered an
+implementation detail and should not be relied upon (this may change in
+the future, but it is desired to have this new dict implementation in
+the language for a few releases before changing the language spec to mandate
 order-preserving semantics for all current and future Python
 implementations; this also helps preserve backwards-compatibility
 with older versions of the language where random iteration order is
 still in effect, e.g. Python 3.5).
+
 (Contributed by INADA Naoki in :issue:`27350`. Idea
 `originally suggested by Raymond Hettinger
 <https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_.)
@@ -545,7 +607,7 @@
 
 While Python provides extensive support to customize how code
 executes, one place it has not done so is in the evaluation of frame
-objects. If you wanted some way to intercept frame evaluation in
+objects.  If you wanted some way to intercept frame evaluation in
 Python there really wasn't any way without directly manipulating
 function pointers for defined functions.
 
@@ -567,17 +629,7 @@
      PEP written by Brett Cannon and Dino Viehland.
 
 
-.. _whatsnew36-pep509:
-
-PEP 509: Add a private version to dict
---------------------------------------
-
-Add a new private version to the builtin ``dict`` type, incremented at
-each dictionary creation and at each dictionary change, to implement
-fast guards on namespaces.
-
-(Contributed by Victor Stinner in :issue:`26058`.)
-
+.. _whatsnew36-pythonmalloc:
 
 PYTHONMALLOC environment variable
 ---------------------------------
@@ -691,34 +743,77 @@
 
 * Long sequences of repeated traceback lines are now abbreviated as
   ``"[Previous line repeated {count} more times]"`` (see
-  :ref:`py36-traceback` for an example).
+  :ref:`whatsnew36-traceback` for an example).
   (Contributed by Emanuel Barry in :issue:`26823`.)
 
 * Import now raises the new exception :exc:`ModuleNotFoundError`
   (subclass of :exc:`ImportError`) when it cannot find a module.  Code
   that current checks for ImportError (in try-except) will still work.
+  (Contributed by Eric Snow in :issue:`15767`.)
+
+* Class methods relying on zero-argument ``super()`` will now work correctly
+  when called from metaclass methods during class creation.
+  (Contributed by Martin Teichmann in :issue:`23722`.)
 
 
 New Modules
 ===========
 
+.. _whatsnew36-pep506:
+
 secrets
 -------
 
-The new :mod:`secrets` module.
+The main purpose of the new :mod:`secrets` module is to provide an obvious way
+to reliably generate cryptographically strong pseudo-random values suitable
+for managing secrets, such as account authentication, tokens, and similar.
+
+.. warning::
+
+  Note that the pseudo-random generators in the :mod:`random` module
+  should *NOT* be used for security purposes.  Use :mod:`secrets`
+  on Python 3.6+ and :func:`os.urandom()` on Python 3.5 and earlier.
+
+.. seealso::
+
+    :pep:`506` -- Adding A Secrets Module To The Standard Library
+      PEP written and implemented by Steven D'Aprano.
 
 
 Improved Modules
 ================
 
+array
+-----
+
+Exhausted iterators of :class:`array.array` will now stay exhausted even
+if the iterated array is extended.  This is consistent with the behavior
+of other mutable sequences.
+
+Contributed by Serhiy Storchaka in :issue:`26492`.
+
+ast
+---
+
+The new :class:`ast.Constant` AST node has been added.  It can be used
+by external AST optimizers for the purposes of constant folding.
+
+Contributed by Victor Stinner in :issue:`26146`.
+
+
 asyncio
 -------
 
-Since the :mod:`asyncio` module is :term:`provisional <provisional api>`,
-all changes introduced in Python 3.6 have also been backported to Python
-3.5.x.
-
-Notable changes in the :mod:`asyncio` module since Python 3.5.0:
+Starting with Python 3.6 the ``asyncio`` is no longer provisional and its
+API is considered stable.
+
+Notable changes in the :mod:`asyncio` module since Python 3.5.0
+(all backported to 3.5.x due to the provisional status):
+
+* The :func:`~asyncio.get_event_loop` function has been changed to
+  always return the currently running loop when called from couroutines
+  and callbacks.
+  (Contributed by Yury Selivanov in :issue:`28613`.)
 
 * The :func:`~asyncio.ensure_future` function and all functions that
   use it, such as :meth:`loop.run_until_complete() <asyncio.BaseEventLoop.run_until_complete>`,
@@ -742,54 +837,165 @@
   loop implementations, such as
   `uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster
   :class:`asyncio.Future` implementation.
-  (Contributed by Yury Selivanov.)
+  (Contributed by Yury Selivanov in :issue:`27041`.)
 
 * New :meth:`loop.get_exception_handler() <asyncio.BaseEventLoop.get_exception_handler>`
   method to get the current exception handler.
-  (Contributed by Yury Selivanov.)
+  (Contributed by Yury Selivanov in :issue:`27040`.)
 
 * New :meth:`StreamReader.readuntil() <asyncio.StreamReader.readuntil>`
   method to read data from the stream until a separator bytes
   sequence appears.
   (Contributed by Mark Korenberg.)
 
+* The performance of :meth:`StreamReader.readexactly() <asyncio.StreamReader.readexactly>`
+  has been improved.
+  (Contributed by Mark Korenberg in :issue:`28370`.)
+
 * The :meth:`loop.getaddrinfo() <asyncio.BaseEventLoop.getaddrinfo>`
   method is optimized to avoid calling the system ``getaddrinfo``
   function if the address is already resolved.
   (Contributed by A. Jesse Jiryu Davis.)
 
+* The :meth:`BaseEventLoop.stop() <asyncio.BaseEventLoop.stop>`
+  method has been changed to stop the loop immediately after
+  the current iteration.  Any new callbacks scheduled as a result
+  of the last iteration will be discarded.
+  (Contributed by Guido van Rossum in :issue:`25593`.)
+
+* :meth:`Future.set_exception <asyncio.futures.Future.set_exception>`
+  will now raise :exc:`TypeError` when passed an instance of
+  :exc:`StopIteration` exception.
+  (Contributed by Chris Angelico in :issue:`26221`.)
+
+* New :meth:`Loop.connect_accepted_socket() <asyncio.BaseEventLoop.connect_accepted_socket>`
+  method to be used by servers that accept connections outside of asyncio,
+  but that use asyncio to handle them.
+  (Contributed by Jim Fulton in :issue:`27392`.)
+
+
+base64
+------
+
+The :func:`~base64.a85decode` function no longer requires the leading
+``'<~'`` characters in input when the *adobe* argument is set.
+(Contributed by Swati Jaiswal in :issue:`25913`.)
+
+
+binascii
+--------
+
+The :func:`~binascii.b2a_base64` function now accepts an optional *newline*
+keyword argument to control whether the newline character is appended to the
+return value.
+(Contributed by Victor Stinner in :issue:`25357`.)
+
+
+cmath
+-----
+
+The new :const:`cmath.tau` (?) constant has been added.
+(Contributed by Lisa Roach in :issue:`12345`, see :pep:`628` for details.)
+
+New constants: :const:`cmath.inf` and :const:`cmath.nan` to
+match :const:`math.inf` and :const:`math.nan`, and also :const:`cmath.infj`
+and :const:`cmath.nanj` to match the format used by complex repr.
+(Contributed by Mark Dickinson in :issue:`23229`.)
+
+
+collections
+-----------
+
+The new :class:`~collections.Collection` abstract base class has been
+added to represent sized iterable container classes.
+
+The :func:`~collections.namedtuple` function now accepts an optional
+keyword argument *module*, which, when specified, is used for
+the ``__module__`` attribute of the returned named tuple class.
+(Contributed by Raymond Hettinger in :issue:`17941`.)
+
+The *verbose* and *rename* arguments for
+:func:`~collections.namedtuple` are now keyword-only.
+(Contributed by Raymond Hettinger in :issue:`25628`.)
+
+Recursive :class:`collections.deque` instances can now be pickled.
+(Contributed by Serhiy Storchaka in :issue:`26482`.)
+
+
+concurrent.futures
+------------------
+
+The :class:`ThreadPoolExecutor <concurrent.futures.thread.ThreadPoolExecutor`
+class constructor now accepts an optional *thread_name_prefix* argument
+to make it possible to customize the names of the threads created by the
+pool.
+(Contributed by Gregory P. Smith in :issue:`27664`.)
+
 
 contextlib
 ----------
 
 The :class:`contextlib.AbstractContextManager` class has been added to
-provide an abstract base class for context managers. It provides a
+provide an abstract base class for context managers.  It provides a
 sensible default implementation for `__enter__()` which returns
-``self`` and leaves `__exit__()` an abstract method. A matching
+``self`` and leaves `__exit__()` an abstract method.  A matching
 class has been added to the :mod:`typing` module as
 :class:`typing.ContextManager`.
 (Contributed by Brett Cannon in :issue:`25609`.)
 
 
-venv
-----
-
-:mod:`venv` accepts a new parameter ``--prompt``. This parameter provides an
-alternative prefix for the virtual environment. (Proposed by ?ukasz.Balcerzak
-and ported to 3.6 by St?phane Wirtel in :issue:`22829`.)
-
-
 datetime
 --------
 
+The :class:`~datetime.datetime` and :class:`~datetime.time` classes have
+the new :attr:`~time.fold` attribute used to disambiguate local time
+when necessary.  Many functions in the :mod:`datetime` have been
+updated to support local time disambiguation.
+See :ref:`Local Time Disambiguation <whatsnew36-pep495>` section for more
+information.
+(Contributed by Alexander Belopolsky in :issue:`24773`.)
+
 The :meth:`datetime.strftime() <datetime.datetime.strftime>` and
-:meth:`date.strftime() <datetime.date.strftime>` methods now support ISO 8601 date
-directives ``%G``, ``%u`` and ``%V``.
+:meth:`date.strftime() <datetime.date.strftime>` methods now support
+ISO 8601 date directives ``%G``, ``%u`` and ``%V``.
 (Contributed by Ashley Anderson in :issue:`12006`.)
 
-
-distutils.command.sdist
------------------------
+The :func:`datetime.isoformat() <datetime.datetime.isoformat>` function
+now accepts an optional *timespec* argument that specifies the number
+of additional components of the time value to include.
+(Contributed by Alessandro Cucci and Alexander Belopolsky in :issue:`19475`.)
+
+The :meth:`datetime.combine() <datetime.datetime.combine>` now
+accepts an optional *tzinfo* argument.
+(Contributed by Alexander Belopolsky in :issue:`27661`.)
+
+
+decimal
+-------
+
+New :meth:`Decimal.as_integer_ratio() <decimal.Decimal.as_integer_ratio>`
+method that returns a pair ``(n, d)`` of integers that represent the given
+:class:`~decimal.Decimal` instance as a fraction, in lowest terms and
+with a positive denominator::
+
+    >>> Decimal('-3.14').as_integer_ratio()
+    (-157, 50)
+
+(Contributed by Stefan Krah amd Mark Dickinson in :issue:`25928`.)
+
+
+dis
+---
+
+Disassembling a class now disassembles class and static
+methods.  (Contributed by Xiang Zhang in :issue:`26733`.)
+
+The disassembler now decodes ``FORMAT_VALUE`` argument.
+(Contributed by Serhiy Storchaka in :issue:`28317`.)
+
+
+distutils
+---------
 
 The ``default_format`` attribute has been removed from
 :class:`distutils.command.sdist.sdist` and the ``formats``
@@ -825,6 +1031,31 @@
 
 On Windows, added the ``'oem'`` encoding to use ``CP_OEMCP`` and the ``'ansi'``
 alias for the existing ``'mbcs'`` encoding, which uses the ``CP_ACP`` code page.
+(Contributed by Steve Dower in :issue:`27959`.)
+
+
+enum
+----
+
+Two new enumeration base classes have been added to the :mod:`enum` module:
+:class:`~enum.Flag` and :class:`~enum.IntFlags`.  Both are used to define
+constants that can be combined using the bitwise operators.
+(Contributed by Ethan Furman in :issue:`23591`.)
+
+Many standard library modules have been updated to use the
+:class:`~enum.IntFlags` class for their constants.
+
+The new :class:`enum.auto` value can be used to assign values to enum
+members automatically::
+
+    >>> from enum import Enum, auto
+    >>> class Color(Enum):
+    ...     red = auto()
+    ...     blue = auto()
+    ...     green = auto()
+    ...
+    >>> list(Color)
+    [<Color.red: 1>, <Color.blue: 2>, <Color.green: 3>]
 
 
 faulthandler
@@ -835,12 +1066,17 @@
 :issue:`23848`.)
 
 
+fileinput
+---------
+
+:func:`~fileinput.hook_encoded` now supports the *errors* argument.
+(Contributed by Joseph Hackman in :issue:`25788`.)
+
+
 hashlib
 -------
 
-:mod:`hashlib` supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2.
-It has been tested with 0.9.8zc, 0.9.8zh and 1.0.1t as well as LibreSSL 2.3
-and 2.4.
+:mod:`hashlib` supports OpenSSL 1.1.0.  The minimum recommend version is 1.0.2.
 (Contributed by Christian Heimes in :issue:`26470`.)
 
 BLAKE2 hash functions were added to the module. :func:`~hashlib.blake2b`
@@ -872,16 +1108,35 @@
 idlelib and IDLE
 ----------------
 
-The idlelib package is being modernized and refactored to make IDLE look and work better and to make the code easier to understand, test, and improve. Part of making IDLE look better, especially on Linux and Mac, is using ttk widgets, mostly in the dialogs.  As a result, IDLE no longer runs with tcl/tk 8.4.  It now requires tcl/tk 8.5 or 8.6.  We recommend running the latest release of either.
-
-'Modernizing' includes renaming and consolidation of idlelib modules. The renaming of files with partial uppercase names is similar to the renaming of, for instance, Tkinter and TkFont to tkinter and tkinter.font in 3.0.  As a result, imports of idlelib files that worked in 3.5 will usually not work in 3.6.  At least a module name change will be needed (see idlelib/README.txt), sometimes more.  (Name changes contributed by Al Swiegart and Terry Reedy in :issue:`24225`.  Most idlelib patches since have been and will be part of the process.)
-
-In compensation, the eventual result with be that some idlelib classes will be easier to use, with better APIs and docstrings explaining them.  Additional useful information will be added to idlelib when available.
+The idlelib package is being modernized and refactored to make IDLE look and
+work better and to make the code easier to understand, test, and improve. Part
+of making IDLE look better, especially on Linux and Mac, is using ttk widgets,
+mostly in the dialogs.  As a result, IDLE no longer runs with tcl/tk 8.4.  It
+now requires tcl/tk 8.5 or 8.6.  We recommend running the latest release of
+either.
+
+'Modernizing' includes renaming and consolidation of idlelib modules. The
+renaming of files with partial uppercase names is similar to the renaming of,
+for instance, Tkinter and TkFont to tkinter and tkinter.font in 3.0.  As a
+result, imports of idlelib files that worked in 3.5 will usually not work in
+3.6.  At least a module name change will be needed (see idlelib/README.txt),
+sometimes more.  (Name changes contributed by Al Swiegart and Terry Reedy in
+:issue:`24225`.  Most idlelib patches since have been and will be part of the
+process.)
+
+In compensation, the eventual result with be that some idlelib classes will be
+easier to use, with better APIs and docstrings explaining them.  Additional
+useful information will be added to idlelib when available.
 
 
 importlib
 ---------
 
+Import now raises the new exception :exc:`ModuleNotFoundError`
+(subclass of :exc:`ImportError`) when it cannot find a module.  Code
+that current checks for ``ImportError`` (in try-except) will still work.
+(Contributed by Eric Snow in :issue:`15767`.)
+
 :class:`importlib.util.LazyLoader` now calls
 :meth:`~importlib.abc.Loader.create_module` on the wrapped loader, removing the
 restriction that :class:`importlib.machinery.BuiltinImporter` and
@@ -894,6 +1149,15 @@
 :term:`path-like object`.
 
 
+inspect
+-------
+
+The :func:`inspect.signature() <inspect.signature>` function now reports the
+implicit ``.0`` parameters generated by the compiler for comprehension and
+generator expression scopes as if they were positional-only parameters called
+``implicit0``. (Contributed by Jelle Zijlstra in :issue:`19611`.)
+
+
 json
 ----
 
@@ -902,9 +1166,38 @@
 (Contributed by Serhiy Storchaka in :issue:`17909`.)
 
 
+logging
+-------
+
+The new :meth:`WatchedFileHandler.reopenIfNeeded() <logging.handlers.WatchedFileHandler.reopenIfNeeded>`
+method has been added to add the ability to check if the log file needs to
+be reopened.
+(Contributed by Marian Horban in :issue:`24884`.)
+
+
+math
+----
+
+The tau (?) constant has been added to the :mod:`math` and :mod:`cmath`
+modules.
+(Contributed by Lisa Roach in :issue:`12345`, see :pep:`628` for details.)
+
+
+multiprocessing
+---------------
+
+:ref:`Proxy Objects <multiprocessing-proxy_objects>` returned by
+:func:`multiprocessing.Manager` can now be nested.
+(Contributed by Davin Potts in :issue:`6766`.)
+
+
 os
 --
 
+See the summary for :ref:`PEP 519 <whatsnew36-pep519>` for details on how the
+:mod:`os` and :mod:`os.path` modules now support
+:term:`path-like objects <path-like object>`.
+
 A new :meth:`~os.scandir.close` method allows explicitly closing a
 :func:`~os.scandir` iterator.  The :func:`~os.scandir` iterator now
 supports the :term:`context manager` protocol.  If a :func:`scandir`
@@ -919,9 +1212,21 @@
 :func:`os.getrandom` function.
 (Contributed by Victor Stinner, part of the :pep:`524`)
 
-See the summary for :ref:`PEP 519 <whatsnew36-pep519>` for details on how the
-:mod:`os` and :mod:`os.path` modules now support
-:term:`path-like objects <path-like object>`.
+
+pathlib
+-------
+
+:mod:`pathlib` now supports :term:`path-like objects <path-like object>`.
+(Contributed by Brett Cannon in :issue:`27186`.)
+
+See the summary for :ref:`PEP 519 <whatsnew36-pep519>` for details.
+
+
+pdb
+---
+
+The :class:`~pdb.Pdb` class constructor has a new optional *readrc* argument
+to control whether ``.pdbrc`` files should be read.
 
 
 pickle
@@ -933,6 +1238,34 @@
 Storchaka in :issue:`24164`.)
 
 
+pickletools
+-----------
+
+:func:`pickletools.dis()` now outputs implicit memo index for the
+``MEMOIZE`` opcode.
+(Contributed by Serhiy Storchaka in :issue:`25382`.)
+
+
+pydoc
+-----
+
+The :mod:`pydoc` module has learned to respect the ``MANPAGER``
+environment variable.
+(Contributed by Matthias Klose in :issue:`8637`.)
+
+:func:`help` and :mod:`pydoc` can now list named tuple fields in the
+order they were defined rather than alphabetically.
+(Contributed by Raymond Hettinger in :issue:`24879`.)
+
+
+random
+-------
+
+The new :func:`~random.choices` function returns a list of elements of
+specified size from the given population with optional weights.
+(Contributed by Raymond Hettinger in :issue:`18844`.)
+
+
 re
 --
 
@@ -945,6 +1278,11 @@
 equivalent to ``group()``.  So ``mo['name']`` is now equivalent to
 ``mo.group('name')``.  (Contributed by Eric Smith in :issue:`24454`.)
 
+:class:`~re.Match` objects in the now support
+:meth:`index-like objects <object.__index__>` as group
+indices.
+(Contributed by Jeroen Demeyer and Xiang Zhang in :issue:`27177`.)
+
 
 readline
 --------
@@ -966,6 +1304,16 @@
 an instance were excluded.  (Contributed by Martin Panter in :issue:`25590`.)
 
 
+shlex
+-----
+
+The :class:`~shlex.shlex` has much
+:ref:`improved shell compatibility <improved-shell-compatibility>`
+through the new *punctuation_chars* argument to control which characters
+are treated as punctuation.
+(Contributed by Vinay Sajip in :issue:`1521950`.)
+
+
 site
 ----
 
@@ -984,20 +1332,25 @@
 socket
 ------
 
-The :func:`~socket.socket.ioctl` function now supports the :data:`~socket.SIO_LOOPBACK_FAST_PATH`
-control code.
+The :func:`~socket.socket.ioctl` function now supports the
+:data:`~socket.SIO_LOOPBACK_FAST_PATH` control code.
 (Contributed by Daniel Stokes in :issue:`26536`.)
 
 The :meth:`~socket.socket.getsockopt` constants ``SO_DOMAIN``,
 ``SO_PROTOCOL``, ``SO_PEERSEC``, and ``SO_PASSSEC`` are now supported.
 (Contributed by Christian Heimes in :issue:`26907`.)
 
+The :meth:`~socket.socket.setsockopt` now supports the
+``setsockopt(level, optname, None, optlen: int)`` form.
+(Contributed by Christian Heimes in issue:`27744`.)
+
 The socket module now supports the address family
 :data:`~socket.AF_ALG` to interface with Linux Kernel crypto API. ``ALG_*``,
 ``SOL_ALG`` and :meth:`~socket.socket.sendmsg_afalg` were added.
 (Contributed by Christian Heimes in :issue:`27744` with support from
 Victor Stinner.)
 
+
 socketserver
 ------------
 
@@ -1013,16 +1366,15 @@
 calling :meth:`~io.BufferedIOBase.write` is now guaranteed to send the
 data in full.  (Contributed by Martin Panter in :issue:`26721`.)
 
+
 ssl
 ---
 
-:mod:`ssl` supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2.
-It has been tested with 0.9.8zc, 0.9.8zh and 1.0.1t as well as LibreSSL 2.3
-and 2.4.
+:mod:`ssl` supports OpenSSL 1.1.0.  The minimum recommend version is 1.0.2.
 (Contributed by Christian Heimes in :issue:`26470`.)
 
 3DES has been removed from the default cipher suites and ChaCha20 Poly1305
-cipher suites are now in the right position.
+cipher suites have been added.
 (Contributed by Christian Heimes in :issue:`27850` and :issue:`27766`.)
 
 :class:`~ssl.SSLContext` has better default configuration for options
@@ -1030,11 +1382,14 @@
 (Contributed by Christian Heimes in :issue:`28043`.)
 
 SSL session can be copied from one client-side connection to another
-with :class:`~ssl.SSLSession`. TLS session resumption can speed up
-the initial handshake, reduce latency and improve performance
+with the new :class:`~ssl.SSLSession` class.  TLS session resumption can
+speed up the initial handshake, reduce latency and improve performance
 (Contributed by Christian Heimes in :issue:`19500` based on a draft by
 Alex Warhawk.)
 
+The new :meth:`~ssl.SSLContext.get_ciphers` method can be used to
+get a list of enabled ciphers in order of cipher priority.
+
 All constants and flags have been converted to :class:`~enum.IntEnum` and
 :class:`~enum.IntFlags`.
 (Contributed by Christian Heimes in :issue:`28025`.)
@@ -1047,6 +1402,22 @@
 no longer case a SystemError.
 (Contributed by Christian Heimes in :issue:`27691`.)
 
+
+statistics
+----------
+
+A new :func:`~statistics.harmonic_mean` function has been added.
+(Contributed by Steven D'Aprano in :issue:`27181`.)
+
+
+struct
+------
+
+:mod:`struct` now supports IEEE 754 half-precision floats via the ``'e'``
+format specifier.
+(Contributed by Eli Stevens, Mark Dickinson in :issue:`11734`.)
+
+
 subprocess
 ----------
 
@@ -1059,6 +1430,22 @@
 The :class:`subprocess.Popen` constructor and all functions that pass arguments
 through to it now accept *encoding* and *errors* arguments. Specifying either
 of these will enable text mode for the *stdin*, *stdout* and *stderr* streams.
+(Contributed by Steve Dower in :issue:`6135`.)
+
+
+sys
+---
+
+The new :func:`~sys.getfilesystemencodeerrors` function returns the name of
+the error mode used to convert between Unicode filenames and bytes filenames.
+(Contributed by Steve Dower in :issue:`27781`.)
+
+On Windows the return value of the :func:`~sys.getwindowsversion` function
+now includes the *platform_version* field which contains the accurate major
+version, minor version and build number of the current operating system,
+rather than the version that is being emulated for the process
+(Contributed by Steve Dower in :issue:`27932`.)
+
 
 telnetlib
 ---------
@@ -1067,6 +1454,26 @@
 St?phane Wirtel in :issue:`25485`).
 
 
+time
+----
+
+The :class:`~time.struct_time` attributes :attr:`tm_gmtoff` and
+:attr:`tm_zone` are now available on all platforms.
+
+
+timeit
+------
+
+The new :meth:`Timer.autorange() <timeit.Timer.autorange>` convenience
+method has been added to call :meth:`Timer.timeit() <timeit.Timer.timeit>`
+repeatedly so that the total run time is greater or equal to 200 milliseconds.
+(Contributed by Steven D'Aprano in :issue:`6422`.)
+
+:mod:`timeit` now warns when there is substantial (4x) variance
+between best and worst times.
+(Contributed by Serhiy Storchaka in :issue:`23552`.)
+
+
 tkinter
 -------
 
@@ -1080,7 +1487,7 @@
 (Contributed by Serhiy Storchaka in :issue:`22115`).
 
 
-.. _py36-traceback:
+.. _whatsnew36-traceback:
 
 traceback
 ---------
@@ -1103,19 +1510,69 @@
 (Contributed by Emanuel Barry in :issue:`26823`.)
 
 
+tracemalloc
+-----------
+
+The :mod:`tracemalloc` module now supports tracing memory allocations in
+multiple different address spaces.
+
+The new :class:`~tracemalloc.DomainFilter` filter class has been added
+to filter block traces by their address space (domain).
+
+(Contributed by Victor Stinner in :issue:`26588`.)
+
+
+.. _whatsnew36-typing:
+
 typing
 ------
 
+Starting with Python 3.6 the :mod:`typing` module is no longer provisional
+and its API is considered stable.
+
+Since the :mod:`typing` module was :term:`provisional <provisional api>`
+in Python 3.5, all changes introduced in Python 3.6 have also been
+backported to Python 3.5.x.
+
 The :class:`typing.ContextManager` class has been added for
 representing :class:`contextlib.AbstractContextManager`.
 (Contributed by Brett Cannon in :issue:`25609`.)
 
+The :class:`typing.Collection` class has been added for
+representing :class:`collections.abc.Collection`.
+(Contributed by Ivan Levkivskyi in :issue:`27598`.)
+
+The :const:`typing.ClassVar` type construct has been added to
+mark class variables.  As introduced in :pep:`526`, a variable annotation
+wrapped in ClassVar indicates that a given attribute is intended to be used as
+a class variable and should not be set on instances of that class.
+(Contributed by Ivan Levkivskyi in `Github #280
+<https://github.com/python/typing/issues/280>`_.)
+
+A new :const:`~typing.TYPE_CHECKING` constant that is assumed to be
+``True`` by the static type chekers, but is ``False`` at runtime.
+(Contributed by Guido van Rossum in `Github #230
+<https://github.com/python/typing/issues/230>`_.)
+
+A new :func:`~typing.NewType` helper function has been added to create
+lightweight distinct types for annotations::
+
+    from typing import NewType
+
+    UserId = NewType('UserId', int)
+    some_id = UserId(524313)
+
+The static type checker will treat the new type as if it were a subclass
+of the original type.  (Contributed by Ivan Levkivskyi in `Github #189
+<https://github.com/python/typing/issues/189>`_.)
+
 
 unicodedata
 -----------
 
-The internal database has been upgraded to use Unicode 9.0.0. (Contributed by
-Benjamin Peterson.)
+The :mod:`unicodedata` module now uses data from `Unicode 9.0.0
+<http://unicode.org/versions/Unicode9.0.0/>`_.
+(Contributed by Benjamin Peterson.)
 
 
 unittest.mock
@@ -1129,12 +1586,17 @@
   was called.
   (Contributed by Amit Saha in :issue:`26323`.)
 
+* The :meth:`Mock.reset_mock() <unittest.mock.Mock.reset_mock>` method
+  now has two optional keyword only arguments: *return_value* and
+  *side_effect*.
+  (Contributed by Kushal Das in :issue:`21271`.)
+
 
 urllib.request
 --------------
 
 If a HTTP request has a file or iterable body (other than a
-bytes object) but no Content-Length header, rather than
+bytes object) but no ``Content-Length`` header, rather than
 throwing an error, :class:`~urllib.request.AbstractHTTPHandler` now
 falls back to use chunked transfer encoding.
 (Contributed by Demian Brecht and Rolf Krahl in :issue:`12319`.)
@@ -1148,6 +1610,14 @@
 (Contributed by Nikolay Bogoychev in :issue:`16099`.)
 
 
+venv
+----
+
+:mod:`venv` accepts a new parameter ``--prompt``. This parameter provides an
+alternative prefix for the virtual environment. (Proposed by ?ukasz Balcerzak
+and ported to 3.6 by St?phane Wirtel in :issue:`22829`.)
+
+
 warnings
 --------
 
@@ -1203,8 +1673,9 @@
 xmlrpc.client
 -------------
 
-The module now supports unmarshalling additional data types used by
-Apache XML-RPC implementation for numerics and ``None``.
+The :mod:`xmlrpc.client` module now supports unmarshalling
+additional data types used by Apache XML-RPC implementation
+for numerics and ``None``.
 (Contributed by Serhiy Storchaka in :issue:`26885`.)
 
 
@@ -1225,20 +1696,27 @@
 zlib
 ----
 
-The :func:`~zlib.compress` function now accepts keyword arguments.
-(Contributed by Aviv Palivoda in :issue:`26243`.)
-
-
-fileinput
----------
-
-:func:`~fileinput.hook_encoded` now supports the *errors* argument.
-(Contributed by Joseph Hackman in :issue:`25788`.)
+The :func:`~zlib.compress` and :func:`~zlib.decompress` functions now accept
+keyword arguments.
+(Contributed by Aviv Palivoda in :issue:`26243` and
+Xiang Zhang in :issue:`16764` respectively.)
 
 
 Optimizations
 =============
 
+* Python interpreter now uses 16-bit wordcode instead of bytecode which
+  made a number of opcode optimizations possible.
+  (Contributed by Demur Rumed with input and reviews from
+  Serhiy Storchaka and Victor Stinner in :issue:`26647` and :issue:`28050`.)
+
+* The :class:`Future <asyncio.futures.Future>` now has an optimized
+  C implementation.
+  (Contributed by Yury Selivanov and INADA Naoki in :issue:`26801`.)
+
+* The :class:`Task <asyncio.tasks.Task>` now has an optimized
+  C implementation. (Contributed by Yury Selivanov in :issue:`28544`.)
+
 * The ASCII decoder is now up to 60 times as fast for error handlers
   ``surrogateescape``, ``ignore`` and ``replace`` (Contributed
   by Victor Stinner in :issue:`24870`).
@@ -1291,12 +1769,23 @@
   it is now about 1.5--4 times faster.
   (Contributed by Serhiy Storchaka in :issue:`26032`).
 
+* :class:`xml.etree.ElementTree` parsing, iteration and deepcopy performance
+  has been significantly improved.
+  (Contributed by Serhiy Storchaka in :issue:`25638`, :issue:`25873`,
+  and :issue:`25869`.)
+
+* Creation of :class:`fractions.Fraction` instances from floats and
+  decimals is now 2 to 3 times faster.
+  (Contributed by Serhiy Storchaka in :issue:`25971`.)
+
 
 Build and C API Changes
 =======================
 
-* Python now requires some C99 support in the toolchain to build. For more
-  information, see :pep:`7`.
+* Python now requires some C99 support in the toolchain to build.
+  Most notably, Python now uses standard integer types and macros in
+  place of custom macros like ``PY_LONG_LONG``.
+  For more information, see :pep:`7` and :issue:`17884`.
 
 * Cross-compiling CPython with the Android NDK and the Android API level set to
   21 (Android 5.0 Lollilop) or greater, runs successfully. While Android is not
@@ -1307,8 +1796,13 @@
   will activate expensive optimizations like PGO.
   (Original patch by Alecsandru Patrascu of Intel in :issue:`26539`.)
 
+* The :term:`GIL <global interpreter lock>` must now be held when allocator
+  functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and
+  :c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called.
+
 * New :c:func:`Py_FinalizeEx` API which indicates if flushing buffered data
-  failed (:issue:`5319`).
+  failed.
+  (Contributed by Martin Panter in :issue:`5319`.)
 
 * :c:func:`PyArg_ParseTupleAndKeywords` now supports :ref:`positional-only
   parameters <positional-only_parameter>`.  Positional-only parameters are
@@ -1319,112 +1813,178 @@
   as ``"[Previous line repeated {count} more times]"``.
   (Contributed by Emanuel Barry in :issue:`26823`.)
 
+* The new :c:func:`PyErr_SetImportErrorSubclass` function allows for
+  specifying a subclass of :exc:`ImportError` to raise.
+  (Contributed by Eric Snow in :issue:`15767`.)
+
+* The new :c:func:`PyErr_ResourceWarning` function can be used to generate
+  the :exc:`ResourceWarning` providing the source of the resource allocation.
+  (Contributed by Victor Stinner in :issue:`26567`.)
+
+* The new :c:func:`PyOS_FSPath` function returns the file system
+  representation of a :term:`path-like object`.
+  (Contributed by Brett Cannon in :issue:`27186`.)
+
+* The :c:func:`PyUnicode_FSConverter` and :c:func:`PyUnicode_FSDecoder`
+  functions will now accept :term:`path-like objects <path-like object>`.
+
+
 
 Deprecated
 ==========
 
+New Keywords
+------------
+
+``async`` and ``await`` are not recommended to be used as variable, class,
+function or module names.  Introduced by :pep:`492` in Python 3.5, they will
+become proper keywords in Python 3.7.  Starting in Python 3.6, the use of
+``async`` or ``await`` as names will generate a :exc:`DeprecationWarning`.
+
+
+Deprecated Python behavior
+--------------------------
+
+Raising the :exc:`StopIteration` exception inside a generator will now
+generate a :exc:`DeprecationWarning`, and will trigger a :exc:`RuntimeError`
+in Python 3.7.  See :ref:`whatsnew-pep-479` for details.
+
+The :meth:`__aiter__` method is now expected to return an asynchronous
+iterator directly instead of returning an awaitable as previously.
+Doing the former will trigger a :exc:`DeprecationWarning`.  Backward
+compatibility will be removed in Python 3.7.
+(Contributed by Yury Selivanov in :issue:`27243`.)
+
+A backslash-character pair that is not a valid escape sequence now generates
+a :exc:`DeprecationWarning`.  Although this will eventually become a
+:exc:`SyntaxError`, that will not be for several Python releases.
+(Contributed by Emanuel Barry in :issue:`27364`.)
+
+When performing a relative import, falling back on ``__name__`` and
+``__path__`` from the calling module when ``__spec__`` or
+``__package__`` are not defined now raises an :exc:`ImportWarning`.
+(Contributed by Rose Ames in :issue:`25791`.)
+
+
+Deprecated Python modules, functions and methods
+------------------------------------------------
+
+asynchat
+~~~~~~~~
+
+The :mod:`asynchat` has been deprecated in favor of :mod:`asyncio`.
+(Contributed by Mariatta in :issue:`25002`.)
+
+
+asyncore
+~~~~~~~~
+
+The :mod:`asyncore` has been deprecated in favor of :mod:`asyncio`.
+(Contributed by Mariatta in :issue:`25002`.)
+
+
+dbm
+~~~
+
+Unlike other :mod:`dbm` implementations, the :mod:`dbm.dumb` module
+creates databases with the ``'rw'`` mode and allows modifying the database
+opened with the ``'r'`` mode.  This behavior is now deprecated and will
+be removed in 3.8.
+(Contributed by Serhiy Storchaka in :issue:`21708`.)
+
+
+distutils
+~~~~~~~~~
+
+The undocumented ``extra_path`` argument to the
+:class:`~distutils.Distribution` constructor is now considered deprecated
+and will raise a warning if set.   Support for this parameter will be
+removed in a future Python release.  See :issue:`27919` for details.
+
+
+grp
+~~~
+
+The support of non-integer arguments in :func:`~grp.getgrgid` has been
+deprecated.
+(Contributed by Serhiy Storchaka in :issue:`26129`.)
+
+
+importlib
+~~~~~~~~~
+
+The :meth:`importlib.machinery.SourceFileLoader.load_module` and
+:meth:`importlib.machinery.SourcelessFileLoader.load_module` methods
+are now deprecated. They were the only remaining implementations of
+:meth:`importlib.abc.Loader.load_module` in :mod:`importlib` that had not
+been deprecated in previous versions of Python in favour of
+:meth:`importlib.abc.Loader.exec_module`.
+
+os
+~~
+
+Undocumented support of general :term:`bytes-like objects <bytes-like object>`
+as paths in :mod:`os` functions, :func:`compile` and similar functions is
+now deprecated.
+(Contributed by Serhiy Storchaka in :issue:`25791` and :issue:`26754`.)
+
+re
+~~
+
+Support for inline flags ``(?letters)`` in the middle of the regular
+expression has been deprecated and will be removed in a future Python
+version.  Flags at the start of a regular expression are still allowed.
+(Contributed by Serhiy Storchaka in :issue:`22493`.)
+
+ssl
+~~~
+
+OpenSSL 0.9.8, 1.0.0 and 1.0.1 are deprecated and no longer supported.
+In the future the :mod:`ssl` module will require at least OpenSSL 1.0.2 or
+1.1.0.
+
+SSL-related arguments like ``certfile``, ``keyfile`` and ``check_hostname``
+in :mod:`ftplib`, :mod:`http.client`, :mod:`imaplib`, :mod:`poplib`,
+and :mod:`smtplib` have been deprecated in favor of ``context``.
+(Contributed by Christian Heimes in :issue:`28022`.)
+
+A couple of protocols and functions of the :mod:`ssl` module are now
+deprecated. Some features will no longer be available in future versions
+of OpenSSL. Other features are deprecated in favor of a different API.
+(Contributed by Christian Heimes in :issue:`28022` and :issue:`26470`.)
+
+tkinter
+~~~~~~~
+
+The :mod:`tkinter.tix` module is now deprecated.  :mod:`tkinter` users
+should use :mod:`tkinter.ttk` instead.
+
+venv
+~~~~
+
+The ``pyvenv`` script has been deprecated in favour of ``python3 -m venv``.
+This prevents confusion as to what Python interpreter ``pyvenv`` is
+connected to and thus what Python interpreter will be used by the virtual
+environment.  (Contributed by Brett Cannon in :issue:`25154`.)
+
+
+Deprecated functions and types of the C API
+-------------------------------------------
+
+Undocumented functions :c:func:`PyUnicode_AsEncodedObject`,
+:c:func:`PyUnicode_AsDecodedObject`, :c:func:`PyUnicode_AsEncodedUnicode`
+and :c:func:`PyUnicode_AsDecodedUnicode` are deprecated now.
+Use :ref:`generic codec based API <codec-registry>` instead.
+
+
 Deprecated Build Options
 ------------------------
 
-The ``--with-system-ffi`` configure flag is now on by default on non-OSX UNIX
-platforms.  It may be disabled by using ``--without-system-ffi``, but using the
-flag is deprecated and will not be accepted in Python 3.7.  OSX is unaffected
-by this change.  Note that many OS distributors already use the
-``--with-system-ffi`` flag when building their system Python.
-
-
-New Keywords
-------------
-
-``async`` and ``await`` are not recommended to be used as variable, class,
-function or module names.  Introduced by :pep:`492` in Python 3.5, they will
-become proper keywords in Python 3.7.
-
-
-Deprecated Python modules, functions and methods
-------------------------------------------------
-
-* :meth:`importlib.machinery.SourceFileLoader.load_module` and
-  :meth:`importlib.machinery.SourcelessFileLoader.load_module` are now
-  deprecated. They were the only remaining implementations of
-  :meth:`importlib.abc.Loader.load_module` in :mod:`importlib` that had not
-  been deprecated in previous versions of Python in favour of
-  :meth:`importlib.abc.Loader.exec_module`.
-
-* The :mod:`tkinter.tix` module is now deprecated.  :mod:`tkinter` users should
-  use :mod:`tkinter.ttk` instead.
-
-
-Deprecated functions and types of the C API
--------------------------------------------
-
-* Undocumented functions :c:func:`PyUnicode_AsEncodedObject`,
-  :c:func:`PyUnicode_AsDecodedObject`, :c:func:`PyUnicode_AsEncodedUnicode`
-  and :c:func:`PyUnicode_AsDecodedUnicode` are deprecated now.
-  Use :ref:`generic codec based API <codec-registry>` instead.
-
-
-Deprecated features
--------------------
-
-* The ``pyvenv`` script has been deprecated in favour of ``python3 -m venv``.
-  This prevents confusion as to what Python interpreter ``pyvenv`` is
-  connected to and thus what Python interpreter will be used by the virtual
-  environment. (Contributed by Brett Cannon in :issue:`25154`.)
-
-* When performing a relative import, falling back on ``__name__`` and
-  ``__path__`` from the calling module when ``__spec__`` or
-  ``__package__`` are not defined now raises an :exc:`ImportWarning`.
-  (Contributed by Rose Ames in :issue:`25791`.)
-
-* Unlike to other :mod:`dbm` implementations, the :mod:`dbm.dumb` module
-  creates database in ``'r'`` and ``'w'`` modes if it doesn't exist and
-  allows modifying database in ``'r'`` mode.  This behavior is now deprecated
-  and will be removed in 3.8.
-  (Contributed by Serhiy Storchaka in :issue:`21708`.)
-
-* Undocumented support of general :term:`bytes-like objects <bytes-like object>`
-  as paths in :mod:`os` functions, :func:`compile` and similar functions is
-  now deprecated.
-  (Contributed by Serhiy Storchaka in :issue:`25791` and :issue:`26754`.)
-
-* The undocumented ``extra_path`` argument to a distutils Distribution
-  is now considered
-  deprecated, will raise a warning during install if set. Support for this
-  parameter will be dropped in a future Python release and likely earlier
-  through third party tools. See :issue:`27919` for details.
-
-* A backslash-character pair that is not a valid escape sequence now generates
-  a DeprecationWarning.  Although this will eventually become a SyntaxError,
-  that will not be for several Python releases.  (Contributed by Emanuel Barry
-  in :issue:`27364`.)
-
-* Inline flags ``(?letters)`` now should be used only at the start of the
-  regular expression.  Inline flags in the middle of the regular expression
-  affects global flags in Python :mod:`re` module.  This is an exception to
-  other regular expression engines that either apply flags to only part of
-  the regular expression or treat them as an error.  To avoid distinguishing
-  inline flags in the middle of the regular expression now emit a deprecation
-  warning.  It will be an error in future Python releases.
-  (Contributed by Serhiy Storchaka in :issue:`22493`.)
-
-* SSL-related arguments like ``certfile``, ``keyfile`` and ``check_hostname``
-  in :mod:`ftplib`, :mod:`http.client`, :mod:`imaplib`, :mod:`poplib`,
-  and :mod:`smtplib` have been deprecated in favor of ``context``.
-  (Contributed by Christian Heimes in :issue:`28022`.)
-
-* A couple of protocols and functions of the :mod:`ssl` module are now
-  deprecated. Some features will no longer be available in future versions
-  of OpenSSL. Other features are deprecated in favor of a different API.
-  (Contributed by Christian Heimes in :issue:`28022` and :issue:`26470`.)
-
-
-Deprecated Python behavior
---------------------------
-
-* Raising the :exc:`StopIteration` exception inside a generator will now generate a
-  :exc:`DeprecationWarning`, and will trigger a :exc:`RuntimeError` in Python 3.7.
-  See :ref:`whatsnew-pep-479` for details.
+The ``--with-system-ffi`` configure flag is now on by default on non-macOS
+UNIX platforms.  It may be disabled by using ``--without-system-ffi``, but
+using the flag is deprecated and will not be accepted in Python 3.7.
+macOS is unaffected by this change.  Note that many OS distributors already
+use the ``--with-system-ffi`` flag when building their system Python.
 
 
 Removed
@@ -1433,9 +1993,14 @@
 API and Feature Removals
 ------------------------
 
+* Unknown escapes consisting of ``'\'`` and an ASCII letter in
+  regular expressions will now cause an error.  The :const:`re.LOCALE`
+  flag can now only be used with binary patterns.
+
 * ``inspect.getmoduleinfo()`` was removed (was deprecated since CPython 3.3).
   :func:`inspect.getmodulename` should be used for obtaining the module
   name for a given path.
+  (Contributed by Yury Selivanov in :issue:`13248`.)
 
 * ``traceback.Ignore`` class and ``traceback.usage``, ``traceback.modname``,
   ``traceback.fullmodname``, ``traceback.find_lines_from_code``,
@@ -1460,6 +2025,8 @@
   script that created these modules is still available in the source
   distribution at :source:`Tools/scripts/h2py.py`.
 
+* The deprecated ``asynchat.fifo`` class has been removed.
+
 
 Porting to Python 3.6
 =====================
@@ -1480,6 +2047,10 @@
 Changes in the Python API
 -------------------------
 
+* :func:`open() <open>` will no longer allow combining the ``'U'`` mode flag
+  with ``'+'``.
+  (Contributed by Jeff Balogh and John O'Connor in :issue:`2091`.)
+
 * :mod:`sqlite3` no longer implicitly commit an open transaction before DDL
   statements.
 
@@ -1523,7 +2094,8 @@
   :mod:`mimetypes`, :mod:`optparse`, :mod:`plistlib`, :mod:`smtpd`,
   :mod:`subprocess`, :mod:`tarfile`, :mod:`threading` and
   :mod:`wave`.  This means they will export new symbols when ``import *``
-  is used.  See :issue:`23883`.
+  is used.
+  (Contributed by Joel Taddei and Jacek Ko?odziej in :issue:`23883`.)
 
 * When performing a relative import, if ``__package__`` does not compare equal
   to ``__spec__.parent`` then :exc:`ImportWarning` is raised.
@@ -1546,8 +2118,8 @@
   :exc:`KeyError` if the user doesn't have privileges.
 
 * The :meth:`socket.socket.close` method now raises an exception if
-  an error (e.g. EBADF) was reported by the underlying system call.
-  See :issue:`26685`.
+  an error (e.g. ``EBADF``) was reported by the underlying system call.
+  (Contributed by Martin Panter in :issue:`26685`.)
 
 * The *decode_data* argument for :class:`smtpd.SMTPChannel` and
   :class:`smtpd.SMTPServer` constructors is now ``False`` by default.
@@ -1557,13 +2129,16 @@
   Code that has already been updated in accordance with the deprecation
   warning generated by 3.5 will not be affected.
 
-* All optional parameters of the :func:`~json.dump`, :func:`~json.dumps`,
+* All optional arguments of the :func:`~json.dump`, :func:`~json.dumps`,
   :func:`~json.load` and :func:`~json.loads` functions and
   :class:`~json.JSONEncoder` and :class:`~json.JSONDecoder` class
   constructors in the :mod:`json` module are now :ref:`keyword-only
   <keyword-only_parameter>`.
   (Contributed by Serhiy Storchaka in :issue:`18726`.)
 
+* Subclasses of :class:`type` which don't override ``type.__new__`` may no
+  longer use the one-argument form to get the type of an object.
+
 * As part of :pep:`487`, the handling of keyword arguments passed to
   :class:`type` (other than the metaclass hint, ``metaclass``) is now
   consistently delegated to :meth:`object.__init_subclass__`. This means that
@@ -1592,7 +2167,63 @@
   header field has been specified and the request body is a file object,
   it is now sent with HTTP 1.1 chunked encoding. If a file object has to
   be sent to a HTTP 1.0 server, the Content-Length value now has to be
-  specified by the caller. See :issue:`12319`.
+  specified by the caller.
+  (Contributed by Demian Brecht and Rolf Krahl with tweaks from
+  Martin Panter in :issue:`12319`.)
+
+* The :class:`~csv.DictReader` now returns rows of type
+  :class:`~collections.OrderedDict`.
+  (Contributed by Steve Holden in :issue:`27842`.)
+
+* The :const:`crypt.METHOD_CRYPT` will no longer be added to ``crypt.methods``
+  if unsupported by the platform.
+  (Contributed by Victor Stinner in :issue:`25287`.)
+
+* The *verbose* and *rename* arguments for
+  :func:`~collections.namedtuple` are now keyword-only.
+  (Contributed by Raymond Hettinger in :issue:`25628`.)
+
+* The :meth:`~cgi.FieldStorage.read_multi` method now ignores the
+  ``Content-Length`` header in part headers.
+  (Contributed by Peter Landry in :issue:`24764`.)
+
+* On Linux, :func:`ctypes.util.find_library` now looks in
+  ``LD_LIBRARY_PATH`` for shared libraries.
+  (Contributed by Vinay Sajip in :issue:`9998`.)
+
+* The :meth:`datetime.fromtimestamp() <datetime.datetime.fromtimestamp>` and
+  :meth:`datetime.utcfromtimestamp() <datetime.datetime.utcfromtimestamp>`
+  methods now round microseconds to nearest with ties going to
+  nearest even integer (``ROUND_HALF_EVEN``), instead of
+  rounding towards negative infinity (``ROUND_FLOOR``).
+  (Contributed by Victor Stinner in :issue:`23517`.)
+
+* The :class:`imaplib.IMAP4` class now handles flags containing the
+  ``']'`` character in messages sent from the server to improve
+  real-world compatibility.
+  (Contributed by Lita Cho in :issue:`21815`.)
+
+* The :func:`mmap.write() <mmap.write>` function now returns the number
+  of bytes written like other write methods.
+  (Contributed by Jakub Stasiak in :issue:`26335`.)
+
+* The :func:`pkgutil.iter_modules` and :func:`pkgutil.walk_packages`
+  functions now return :class:`~pkgutil.ModuleInfo` named tuples.
+  (Contributed by Ramchandra Apte in :issue:`17211`.)
+
+* :func:`re.sub` now raises an error for invalid numerical group
+  reference in replacement template even if the pattern is not
+  found in the string.  Error message for invalid group reference
+  now includes the group index and the position of the reference.
+  (Contributed by SilentGhost, Serhiy Storchaka in :issue:`25953`.)
+
+* :class:`zipfile.ZipFile` will now raise :exc:`NotImplementedError` for
+  unrecognized compression values.  Previously a plain :exc:`RuntimeError`
+  was raised.  Additionally, calling :class:`~zipfile.ZipFile` methods or
+  on a closed ZipFile or calling :meth:`~zipfile.ZipFile.write` methods
+  on a ZipFile created with mode ``'r'`` will raise a :exc:`ValueError`.
+  Previously, a :exc:`RuntimeError` was raised in those scenarios.
+
 
 Changes in the C API
 --------------------

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 10 13:28:11 2016
From: python-checkins at python.org (yury.selivanov)
Date: Thu, 10 Nov 2016 18:28:11 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg2MzUp?=
Message-ID: <20161110182807.2235.38353.D3659E3D@psf.io>

https://hg.python.org/cpython/rev/69bdf8bdfd61
changeset:   105042:69bdf8bdfd61
parent:      105040:5c4ce500dd35
parent:      105041:c0060567f35d
user:        Yury Selivanov <yury at magic.io>
date:        Thu Nov 10 13:28:03 2016 -0500
summary:
  Merge 3.6 (issue #28635)

files:
  Doc/whatsnew/3.6.rst |  1125 +++++++++++++++++++++++------
  1 files changed, 878 insertions(+), 247 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -87,7 +87,7 @@
 
 * The :ref:`dict <typesmapping>` type has been reimplemented to use
   a :ref:`faster, more compact representation <whatsnew36-compactdict>`
-  similar to the `PyPy dict implementation`_. This resulted in dictionaries
+  similar to the `PyPy dict implementation`_.  This resulted in dictionaries
   using 20% to 25% less memory when compared to Python 3.5.
 
 * Customization of class creation has been simplified with the
@@ -96,27 +96,49 @@
 * The class attibute definition order is
   :ref:`now preserved  <whatsnew36-pep520>`.
 
-* The order of elements in ``**kwargs`` now corresponds to the order in
-  the function signature:
-  :ref:`Preserving Keyword Argument Order <whatsnew36-pep468>`.
-
-* DTrace and SystemTap :ref:`probing support <whatsnew36-tracing>`.
+* The order of elements in ``**kwargs`` now
+  :ref:`corresponds to the order <whatsnew36-pep468>` in which keyword
+  arguments were passed to the function.
+
+* DTrace and SystemTap :ref:`probing support <whatsnew36-tracing>` has
+  been added.
+
+* The new :ref:`PYTHONMALLOC <whatsnew36-pythonmalloc>` environment variable
+  can now be used to debug the interpreter memory allocation and access
+  errors.
 
 
 Significant improvements in the standard library:
 
+* The :mod:`asyncio` module has received new features, significant
+  usability and performance improvements, and a fair amount of bug fixes.
+  Starting with Python 3.6 the ``asyncio`` module is no longer provisional
+  and its API is considered stable.
+
 * A new :ref:`file system path protocol <whatsnew36-pep519>` has been
   implemented to support :term:`path-like objects <path-like object>`.
   All standard library functions operating on paths have been updated to
   work with the new protocol.
 
-* The overhead of :mod:`asyncio` implementation has been reduced by
-  up to 50% thanks to the new C implementation of the :class:`asyncio.Future`
-  and :class:`asyncio.Task` classes and other optimizations.
+* The :mod:`datetime` module has gained support for
+  :ref:`Local Time Disambiguation <whatsnew36-pep495>`.
+
+* The :mod:`typing` module received a number of
+  :ref:`improvements <whatsnew36-typing>` and is no longer provisional.
+
+* The :mod:`tracemalloc` module has been significantly reworked
+  and is now used to provide better output for :exc:`ResourceWarning`s
+  as well as provide better diagnostics for memory allocation errors.
+  See the :ref:`PYTHONMALLOC section <whatsnew36-pythonmalloc>` for more
+  information.
 
 
 Security improvements:
 
+* The new :mod:`secrets` module has been added to simplify the generation of
+  cryptographically strong pseudo-random numbers suitable for
+  managing secrets such as account authentication, tokens, and similar.
+
 * On Linux, :func:`os.urandom` now blocks until the system urandom entropy
   pool is initialized to increase the security. See the :pep:`524` for the
   rationale.
@@ -153,26 +175,6 @@
   more information.
 
 
-A complete list of PEP's implemented in Python 3.6:
-
-* :pep:`468`, :ref:`Preserving Keyword Argument Order <whatsnew36-pep468>`
-* :pep:`487`, :ref:`Simpler customization of class creation <whatsnew36-pep487>`
-* :pep:`495`, Local Time Disambiguation
-* :pep:`498`, :ref:`Formatted string literals <whatsnew36-pep498>`
-* :pep:`506`, Adding A Secrets Module To The Standard Library
-* :pep:`509`, :ref:`Add a private version to dict <whatsnew36-pep509>`
-* :pep:`515`, :ref:`Underscores in Numeric Literals <whatsnew36-pep515>`
-* :pep:`519`, :ref:`Adding a file system path protocol <whatsnew36-pep519>`
-* :pep:`520`, :ref:`Preserving Class Attribute Definition Order <whatsnew36-pep520>`
-* :pep:`523`, :ref:`Adding a frame evaluation API to CPython <whatsnew36-pep523>`
-* :pep:`524`, Make os.urandom() blocking on Linux (during system startup)
-* :pep:`525`, Asynchronous Generators (provisional)
-* :pep:`526`, :ref:`Syntax for Variable Annotations (provisional) <whatsnew36-pep526>`
-* :pep:`528`, :ref:`Change Windows console encoding to UTF-8 (provisional) <whatsnew36-pep528>`
-* :pep:`529`, :ref:`Change Windows filesystem encoding to UTF-8 (provisional) <whatsnew36-pep529>`
-* :pep:`530`, Asynchronous Comprehensions
-
-
 .. _PyPy dict implementation: https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html
 
 
@@ -185,7 +187,7 @@
 ----------------------------------
 
 :pep:`498` introduces a new kind of string literals: *f-strings*, or
-*formatted string literals*.
+:ref:`formatted string literals <f-strings>`.
 
 Formatted string literals are prefixed with ``'f'`` and are similar to
 the format strings accepted by :meth:`str.format`.  They contain replacement
@@ -196,6 +198,11 @@
     >>> name = "Fred"
     >>> f"He said his name is {name}."
     'He said his name is Fred.'
+    >>> width = 10
+    >>> precision = 4
+    >>> value = decimal.Decimal("12.34567")
+    >>> f"result: {value:{width}.{precision}}"  # nested fields
+    'result:      12.35'
 
 .. seealso::
 
@@ -258,6 +265,18 @@
 specifier.  Leading, trailing, or multiple underscores in a row are not
 allowed.
 
+The :ref:`string formatting <formatspec>` language also now has support
+for the ``'_'`` option to signal the use of an underscore for a thousands
+separator for floating point presentation types and for integer
+presentation type ``'d'``.  For integer presentation types ``'b'``,
+``'o'``, ``'x'``, and ``'X'``, underscores will be inserted every 4
+digits::
+
+    >>> '{:_}'.format(1000000)
+    '1_000_000'
+    >>> '{:_x}'.format(0xFFFFFFFF)
+    'ffff_ffff'
+
 .. seealso::
 
    :pep:`515` -- Underscores in Numeric Literals
@@ -302,7 +321,7 @@
 Additionally, ``await`` expressions are supported in all kinds
 of comprehensions::
 
-    result = [await fun() for fun in funcs]
+    result = [await fun() for fun in funcs if await condition()]
 
 .. seealso::
 
@@ -319,17 +338,18 @@
 The new ``__init_subclass__`` classmethod will be called on the base class
 whenever a new subclass is created::
 
-    >>> class QuestBase:
-    ...    # this is implicitly a @classmethod
-    ...    def __init_subclass__(cls, swallow, **kwargs):
-    ...        cls.swallow = swallow
-    ...        super().__init_subclass__(**kwargs)
-
-    >>> class Quest(QuestBase, swallow="african"):
-    ...    pass
-
-    >>> Quest.swallow
-    'african'
+    class PluginBase:
+        subclasses = []
+
+        def __init_subclass__(cls, **kwargs):
+            super().__init_subclass__(**kwargs)
+            cls.subclasses.append(cls)
+
+    class Plugin1(PluginBase):
+        pass
+
+    class Plugin2(PluginBase):
+        pass
 
 .. seealso::
 
@@ -345,10 +365,29 @@
 -----------------------------------------
 
 :pep:`487` extends the descriptor protocol has to include the new optional
-``__set_name__`` method.  Whenever a new class is defined, the new method
-will be called on all descriptors included in the definition, providing
+:meth:`~object.__set_name__` method.  Whenever a new class is defined, the new
+method will be called on all descriptors included in the definition, providing
 them with a reference to the class being defined and the name given to the
-descriptor within the class namespace.
+descriptor within the class namespace.  In other words, instances of
+descriptors can now know the attribute name of the descriptor in the
+owner class::
+
+    class IntField:
+        def __get__(self, instance, owner):
+            return instance.__dict__[self.name]
+
+        def __set__(self, instance, value):
+            if not isinstance(value, int):
+                raise ValueError(f'expecting integer in {self.name}')
+            instance.__dict__[self.name] = value
+
+        # this is the new initializer:
+        def __set_name__(self, owner, name):
+            self.name = name
+
+    class Model:
+        int_field = IntField()
+
 
 .. seealso::
 
@@ -358,20 +397,6 @@
     :ref:`Feature documentation <descriptors>`
 
 
-.. _whatsnew36-pep506:
-
-PEP 506: Adding A Secrets Module To The Standard Library
---------------------------------------------------------
-
-
-
-.. seealso::
-
-    :pep:`506` -- Adding A Secrets Module To The Standard Library
-      PEP written and implemented by Steven D'Aprano.
-
-
-
 .. _whatsnew36-pep519:
 
 PEP 519: Adding a file system path protocol
@@ -401,9 +426,8 @@
 
 The built-in :func:`open` function has been updated to accept
 :class:`os.PathLike` objects as have all relevant functions in the
-:mod:`os` and :mod:`os.path` modules. :c:func:`PyUnicode_FSConverter`
-and :c:func:`PyUnicode_FSConverter` have been changed to accept
-path-like objects. The :class:`os.DirEntry` class
+:mod:`os` and :mod:`os.path` modules, as well as most functions and
+classes in the standard library.  The :class:`os.DirEntry` class
 and relevant classes in :mod:`pathlib` have also been updated to
 implement :class:`os.PathLike`.
 
@@ -439,6 +463,43 @@
        PEP written by Brett Cannon and Koos Zevenhoven.
 
 
+.. _whatsnew36-pep495:
+
+PEP 495: Local Time Disambiguation
+----------------------------------
+
+In most world locations, there have been and will be times when local clocks
+are moved back.  In those times, intervals are introduced in which local
+clocks show the same time twice in the same day. In these situations, the
+information displayed on a local clock (or stored in a Python datetime
+instance) is insufficient to identify a particular moment in time.
+
+:pep:`495` adds the new *fold* attribute to instances of
+:class:`datetime.datetime` and :class:`datetime.time` classes to differentiate
+between two moments in time for which local times are the same::
+
+    >>> u0 = datetime(2016, 11, 6, 4, tzinfo=timezone.utc)
+    >>> for i in range(4):
+    ...     u = u0 + i*HOUR
+    ...     t = u.astimezone(Eastern)
+    ...     print(u.time(), 'UTC =', t.time(), t.tzname(), t.fold)
+    ...
+    04:00:00 UTC = 00:00:00 EDT 0
+    05:00:00 UTC = 01:00:00 EDT 0
+    06:00:00 UTC = 01:00:00 EST 1
+    07:00:00 UTC = 02:00:00 EST 0
+
+The values of the :attr:`fold <datetime.datetime.fold>` attribute have the
+value `0`  all instances except those that represent the second
+(chronologically) moment in time in an ambiguous case.
+
+.. seealso::
+
+  :pep:`495` -- Local Time Disambiguation
+     PEP written by Alexander Belopolsky and Tim Peters, implementation
+     by Alexander Belopolsky.
+
+
 .. _whatsnew36-pep529:
 
 PEP 529: Change Windows filesystem encoding to UTF-8
@@ -489,10 +550,11 @@
 
 Attributes in a class definition body have a natural ordering: the same
 order in which the names appear in the source.  This order is now
-preserved in the new class's ``__dict__`` attribute.
+preserved in the new class's :attr:`~object.__dict__` attribute.
 
 Also, the effective default class *execution* namespace (returned from
-``type.__prepare__()``) is now an insertion-order-preserving mapping.
+:ref:`type.__prepare__() <prepare>`) is now an insertion-order-preserving
+mapping.
 
 .. seealso::
 
@@ -523,16 +585,16 @@
 `pioneered by PyPy <https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html>`_.
 The memory usage of the new :func:`dict` is between 20% and 25% smaller
 compared to Python 3.5.
-:pep:`468` (Preserving the order of ``**kwargs`` in a function.) is
-implemented by this. The order-preserving aspect of this new
-implementation is considered an implementation detail and should
-not be relied upon (this may change in the future, but it is desired
-to have this new dict implementation in the language for a few
-releases before changing the language spec to mandate
+
+The order-preserving aspect of this new implementation is considered an
+implementation detail and should not be relied upon (this may change in
+the future, but it is desired to have this new dict implementation in
+the language for a few releases before changing the language spec to mandate
 order-preserving semantics for all current and future Python
 implementations; this also helps preserve backwards-compatibility
 with older versions of the language where random iteration order is
 still in effect, e.g. Python 3.5).
+
 (Contributed by INADA Naoki in :issue:`27350`. Idea
 `originally suggested by Raymond Hettinger
 <https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_.)
@@ -545,7 +607,7 @@
 
 While Python provides extensive support to customize how code
 executes, one place it has not done so is in the evaluation of frame
-objects. If you wanted some way to intercept frame evaluation in
+objects.  If you wanted some way to intercept frame evaluation in
 Python there really wasn't any way without directly manipulating
 function pointers for defined functions.
 
@@ -567,17 +629,7 @@
      PEP written by Brett Cannon and Dino Viehland.
 
 
-.. _whatsnew36-pep509:
-
-PEP 509: Add a private version to dict
---------------------------------------
-
-Add a new private version to the builtin ``dict`` type, incremented at
-each dictionary creation and at each dictionary change, to implement
-fast guards on namespaces.
-
-(Contributed by Victor Stinner in :issue:`26058`.)
-
+.. _whatsnew36-pythonmalloc:
 
 PYTHONMALLOC environment variable
 ---------------------------------
@@ -691,34 +743,77 @@
 
 * Long sequences of repeated traceback lines are now abbreviated as
   ``"[Previous line repeated {count} more times]"`` (see
-  :ref:`py36-traceback` for an example).
+  :ref:`whatsnew36-traceback` for an example).
   (Contributed by Emanuel Barry in :issue:`26823`.)
 
 * Import now raises the new exception :exc:`ModuleNotFoundError`
   (subclass of :exc:`ImportError`) when it cannot find a module.  Code
   that current checks for ImportError (in try-except) will still work.
+  (Contributed by Eric Snow in :issue:`15767`.)
+
+* Class methods relying on zero-argument ``super()`` will now work correctly
+  when called from metaclass methods during class creation.
+  (Contributed by Martin Teichmann in :issue:`23722`.)
 
 
 New Modules
 ===========
 
+.. _whatsnew36-pep506:
+
 secrets
 -------
 
-The new :mod:`secrets` module.
+The main purpose of the new :mod:`secrets` module is to provide an obvious way
+to reliably generate cryptographically strong pseudo-random values suitable
+for managing secrets, such as account authentication, tokens, and similar.
+
+.. warning::
+
+  Note that the pseudo-random generators in the :mod:`random` module
+  should *NOT* be used for security purposes.  Use :mod:`secrets`
+  on Python 3.6+ and :func:`os.urandom()` on Python 3.5 and earlier.
+
+.. seealso::
+
+    :pep:`506` -- Adding A Secrets Module To The Standard Library
+      PEP written and implemented by Steven D'Aprano.
 
 
 Improved Modules
 ================
 
+array
+-----
+
+Exhausted iterators of :class:`array.array` will now stay exhausted even
+if the iterated array is extended.  This is consistent with the behavior
+of other mutable sequences.
+
+Contributed by Serhiy Storchaka in :issue:`26492`.
+
+ast
+---
+
+The new :class:`ast.Constant` AST node has been added.  It can be used
+by external AST optimizers for the purposes of constant folding.
+
+Contributed by Victor Stinner in :issue:`26146`.
+
+
 asyncio
 -------
 
-Since the :mod:`asyncio` module is :term:`provisional <provisional api>`,
-all changes introduced in Python 3.6 have also been backported to Python
-3.5.x.
-
-Notable changes in the :mod:`asyncio` module since Python 3.5.0:
+Starting with Python 3.6 the ``asyncio`` is no longer provisional and its
+API is considered stable.
+
+Notable changes in the :mod:`asyncio` module since Python 3.5.0
+(all backported to 3.5.x due to the provisional status):
+
+* The :func:`~asyncio.get_event_loop` function has been changed to
+  always return the currently running loop when called from couroutines
+  and callbacks.
+  (Contributed by Yury Selivanov in :issue:`28613`.)
 
 * The :func:`~asyncio.ensure_future` function and all functions that
   use it, such as :meth:`loop.run_until_complete() <asyncio.BaseEventLoop.run_until_complete>`,
@@ -742,54 +837,165 @@
   loop implementations, such as
   `uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster
   :class:`asyncio.Future` implementation.
-  (Contributed by Yury Selivanov.)
+  (Contributed by Yury Selivanov in :issue:`27041`.)
 
 * New :meth:`loop.get_exception_handler() <asyncio.BaseEventLoop.get_exception_handler>`
   method to get the current exception handler.
-  (Contributed by Yury Selivanov.)
+  (Contributed by Yury Selivanov in :issue:`27040`.)
 
 * New :meth:`StreamReader.readuntil() <asyncio.StreamReader.readuntil>`
   method to read data from the stream until a separator bytes
   sequence appears.
   (Contributed by Mark Korenberg.)
 
+* The performance of :meth:`StreamReader.readexactly() <asyncio.StreamReader.readexactly>`
+  has been improved.
+  (Contributed by Mark Korenberg in :issue:`28370`.)
+
 * The :meth:`loop.getaddrinfo() <asyncio.BaseEventLoop.getaddrinfo>`
   method is optimized to avoid calling the system ``getaddrinfo``
   function if the address is already resolved.
   (Contributed by A. Jesse Jiryu Davis.)
 
+* The :meth:`BaseEventLoop.stop() <asyncio.BaseEventLoop.stop>`
+  method has been changed to stop the loop immediately after
+  the current iteration.  Any new callbacks scheduled as a result
+  of the last iteration will be discarded.
+  (Contributed by Guido van Rossum in :issue:`25593`.)
+
+* :meth:`Future.set_exception <asyncio.futures.Future.set_exception>`
+  will now raise :exc:`TypeError` when passed an instance of
+  :exc:`StopIteration` exception.
+  (Contributed by Chris Angelico in :issue:`26221`.)
+
+* New :meth:`Loop.connect_accepted_socket() <asyncio.BaseEventLoop.connect_accepted_socket>`
+  method to be used by servers that accept connections outside of asyncio,
+  but that use asyncio to handle them.
+  (Contributed by Jim Fulton in :issue:`27392`.)
+
+
+base64
+------
+
+The :func:`~base64.a85decode` function no longer requires the leading
+``'<~'`` characters in input when the *adobe* argument is set.
+(Contributed by Swati Jaiswal in :issue:`25913`.)
+
+
+binascii
+--------
+
+The :func:`~binascii.b2a_base64` function now accepts an optional *newline*
+keyword argument to control whether the newline character is appended to the
+return value.
+(Contributed by Victor Stinner in :issue:`25357`.)
+
+
+cmath
+-----
+
+The new :const:`cmath.tau` (?) constant has been added.
+(Contributed by Lisa Roach in :issue:`12345`, see :pep:`628` for details.)
+
+New constants: :const:`cmath.inf` and :const:`cmath.nan` to
+match :const:`math.inf` and :const:`math.nan`, and also :const:`cmath.infj`
+and :const:`cmath.nanj` to match the format used by complex repr.
+(Contributed by Mark Dickinson in :issue:`23229`.)
+
+
+collections
+-----------
+
+The new :class:`~collections.Collection` abstract base class has been
+added to represent sized iterable container classes.
+
+The :func:`~collections.namedtuple` function now accepts an optional
+keyword argument *module*, which, when specified, is used for
+the ``__module__`` attribute of the returned named tuple class.
+(Contributed by Raymond Hettinger in :issue:`17941`.)
+
+The *verbose* and *rename* arguments for
+:func:`~collections.namedtuple` are now keyword-only.
+(Contributed by Raymond Hettinger in :issue:`25628`.)
+
+Recursive :class:`collections.deque` instances can now be pickled.
+(Contributed by Serhiy Storchaka in :issue:`26482`.)
+
+
+concurrent.futures
+------------------
+
+The :class:`ThreadPoolExecutor <concurrent.futures.thread.ThreadPoolExecutor`
+class constructor now accepts an optional *thread_name_prefix* argument
+to make it possible to customize the names of the threads created by the
+pool.
+(Contributed by Gregory P. Smith in :issue:`27664`.)
+
 
 contextlib
 ----------
 
 The :class:`contextlib.AbstractContextManager` class has been added to
-provide an abstract base class for context managers. It provides a
+provide an abstract base class for context managers.  It provides a
 sensible default implementation for `__enter__()` which returns
-``self`` and leaves `__exit__()` an abstract method. A matching
+``self`` and leaves `__exit__()` an abstract method.  A matching
 class has been added to the :mod:`typing` module as
 :class:`typing.ContextManager`.
 (Contributed by Brett Cannon in :issue:`25609`.)
 
 
-venv
-----
-
-:mod:`venv` accepts a new parameter ``--prompt``. This parameter provides an
-alternative prefix for the virtual environment. (Proposed by ?ukasz.Balcerzak
-and ported to 3.6 by St?phane Wirtel in :issue:`22829`.)
-
-
 datetime
 --------
 
+The :class:`~datetime.datetime` and :class:`~datetime.time` classes have
+the new :attr:`~time.fold` attribute used to disambiguate local time
+when necessary.  Many functions in the :mod:`datetime` have been
+updated to support local time disambiguation.
+See :ref:`Local Time Disambiguation <whatsnew36-pep495>` section for more
+information.
+(Contributed by Alexander Belopolsky in :issue:`24773`.)
+
 The :meth:`datetime.strftime() <datetime.datetime.strftime>` and
-:meth:`date.strftime() <datetime.date.strftime>` methods now support ISO 8601 date
-directives ``%G``, ``%u`` and ``%V``.
+:meth:`date.strftime() <datetime.date.strftime>` methods now support
+ISO 8601 date directives ``%G``, ``%u`` and ``%V``.
 (Contributed by Ashley Anderson in :issue:`12006`.)
 
-
-distutils.command.sdist
------------------------
+The :func:`datetime.isoformat() <datetime.datetime.isoformat>` function
+now accepts an optional *timespec* argument that specifies the number
+of additional components of the time value to include.
+(Contributed by Alessandro Cucci and Alexander Belopolsky in :issue:`19475`.)
+
+The :meth:`datetime.combine() <datetime.datetime.combine>` now
+accepts an optional *tzinfo* argument.
+(Contributed by Alexander Belopolsky in :issue:`27661`.)
+
+
+decimal
+-------
+
+New :meth:`Decimal.as_integer_ratio() <decimal.Decimal.as_integer_ratio>`
+method that returns a pair ``(n, d)`` of integers that represent the given
+:class:`~decimal.Decimal` instance as a fraction, in lowest terms and
+with a positive denominator::
+
+    >>> Decimal('-3.14').as_integer_ratio()
+    (-157, 50)
+
+(Contributed by Stefan Krah amd Mark Dickinson in :issue:`25928`.)
+
+
+dis
+---
+
+Disassembling a class now disassembles class and static
+methods.  (Contributed by Xiang Zhang in :issue:`26733`.)
+
+The disassembler now decodes ``FORMAT_VALUE`` argument.
+(Contributed by Serhiy Storchaka in :issue:`28317`.)
+
+
+distutils
+---------
 
 The ``default_format`` attribute has been removed from
 :class:`distutils.command.sdist.sdist` and the ``formats``
@@ -825,6 +1031,31 @@
 
 On Windows, added the ``'oem'`` encoding to use ``CP_OEMCP`` and the ``'ansi'``
 alias for the existing ``'mbcs'`` encoding, which uses the ``CP_ACP`` code page.
+(Contributed by Steve Dower in :issue:`27959`.)
+
+
+enum
+----
+
+Two new enumeration base classes have been added to the :mod:`enum` module:
+:class:`~enum.Flag` and :class:`~enum.IntFlags`.  Both are used to define
+constants that can be combined using the bitwise operators.
+(Contributed by Ethan Furman in :issue:`23591`.)
+
+Many standard library modules have been updated to use the
+:class:`~enum.IntFlags` class for their constants.
+
+The new :class:`enum.auto` value can be used to assign values to enum
+members automatically::
+
+    >>> from enum import Enum, auto
+    >>> class Color(Enum):
+    ...     red = auto()
+    ...     blue = auto()
+    ...     green = auto()
+    ...
+    >>> list(Color)
+    [<Color.red: 1>, <Color.blue: 2>, <Color.green: 3>]
 
 
 faulthandler
@@ -835,12 +1066,17 @@
 :issue:`23848`.)
 
 
+fileinput
+---------
+
+:func:`~fileinput.hook_encoded` now supports the *errors* argument.
+(Contributed by Joseph Hackman in :issue:`25788`.)
+
+
 hashlib
 -------
 
-:mod:`hashlib` supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2.
-It has been tested with 0.9.8zc, 0.9.8zh and 1.0.1t as well as LibreSSL 2.3
-and 2.4.
+:mod:`hashlib` supports OpenSSL 1.1.0.  The minimum recommend version is 1.0.2.
 (Contributed by Christian Heimes in :issue:`26470`.)
 
 BLAKE2 hash functions were added to the module. :func:`~hashlib.blake2b`
@@ -872,16 +1108,35 @@
 idlelib and IDLE
 ----------------
 
-The idlelib package is being modernized and refactored to make IDLE look and work better and to make the code easier to understand, test, and improve. Part of making IDLE look better, especially on Linux and Mac, is using ttk widgets, mostly in the dialogs.  As a result, IDLE no longer runs with tcl/tk 8.4.  It now requires tcl/tk 8.5 or 8.6.  We recommend running the latest release of either.
-
-'Modernizing' includes renaming and consolidation of idlelib modules. The renaming of files with partial uppercase names is similar to the renaming of, for instance, Tkinter and TkFont to tkinter and tkinter.font in 3.0.  As a result, imports of idlelib files that worked in 3.5 will usually not work in 3.6.  At least a module name change will be needed (see idlelib/README.txt), sometimes more.  (Name changes contributed by Al Swiegart and Terry Reedy in :issue:`24225`.  Most idlelib patches since have been and will be part of the process.)
-
-In compensation, the eventual result with be that some idlelib classes will be easier to use, with better APIs and docstrings explaining them.  Additional useful information will be added to idlelib when available.
+The idlelib package is being modernized and refactored to make IDLE look and
+work better and to make the code easier to understand, test, and improve. Part
+of making IDLE look better, especially on Linux and Mac, is using ttk widgets,
+mostly in the dialogs.  As a result, IDLE no longer runs with tcl/tk 8.4.  It
+now requires tcl/tk 8.5 or 8.6.  We recommend running the latest release of
+either.
+
+'Modernizing' includes renaming and consolidation of idlelib modules. The
+renaming of files with partial uppercase names is similar to the renaming of,
+for instance, Tkinter and TkFont to tkinter and tkinter.font in 3.0.  As a
+result, imports of idlelib files that worked in 3.5 will usually not work in
+3.6.  At least a module name change will be needed (see idlelib/README.txt),
+sometimes more.  (Name changes contributed by Al Swiegart and Terry Reedy in
+:issue:`24225`.  Most idlelib patches since have been and will be part of the
+process.)
+
+In compensation, the eventual result with be that some idlelib classes will be
+easier to use, with better APIs and docstrings explaining them.  Additional
+useful information will be added to idlelib when available.
 
 
 importlib
 ---------
 
+Import now raises the new exception :exc:`ModuleNotFoundError`
+(subclass of :exc:`ImportError`) when it cannot find a module.  Code
+that current checks for ``ImportError`` (in try-except) will still work.
+(Contributed by Eric Snow in :issue:`15767`.)
+
 :class:`importlib.util.LazyLoader` now calls
 :meth:`~importlib.abc.Loader.create_module` on the wrapped loader, removing the
 restriction that :class:`importlib.machinery.BuiltinImporter` and
@@ -894,6 +1149,15 @@
 :term:`path-like object`.
 
 
+inspect
+-------
+
+The :func:`inspect.signature() <inspect.signature>` function now reports the
+implicit ``.0`` parameters generated by the compiler for comprehension and
+generator expression scopes as if they were positional-only parameters called
+``implicit0``. (Contributed by Jelle Zijlstra in :issue:`19611`.)
+
+
 json
 ----
 
@@ -902,9 +1166,38 @@
 (Contributed by Serhiy Storchaka in :issue:`17909`.)
 
 
+logging
+-------
+
+The new :meth:`WatchedFileHandler.reopenIfNeeded() <logging.handlers.WatchedFileHandler.reopenIfNeeded>`
+method has been added to add the ability to check if the log file needs to
+be reopened.
+(Contributed by Marian Horban in :issue:`24884`.)
+
+
+math
+----
+
+The tau (?) constant has been added to the :mod:`math` and :mod:`cmath`
+modules.
+(Contributed by Lisa Roach in :issue:`12345`, see :pep:`628` for details.)
+
+
+multiprocessing
+---------------
+
+:ref:`Proxy Objects <multiprocessing-proxy_objects>` returned by
+:func:`multiprocessing.Manager` can now be nested.
+(Contributed by Davin Potts in :issue:`6766`.)
+
+
 os
 --
 
+See the summary for :ref:`PEP 519 <whatsnew36-pep519>` for details on how the
+:mod:`os` and :mod:`os.path` modules now support
+:term:`path-like objects <path-like object>`.
+
 A new :meth:`~os.scandir.close` method allows explicitly closing a
 :func:`~os.scandir` iterator.  The :func:`~os.scandir` iterator now
 supports the :term:`context manager` protocol.  If a :func:`scandir`
@@ -919,9 +1212,21 @@
 :func:`os.getrandom` function.
 (Contributed by Victor Stinner, part of the :pep:`524`)
 
-See the summary for :ref:`PEP 519 <whatsnew36-pep519>` for details on how the
-:mod:`os` and :mod:`os.path` modules now support
-:term:`path-like objects <path-like object>`.
+
+pathlib
+-------
+
+:mod:`pathlib` now supports :term:`path-like objects <path-like object>`.
+(Contributed by Brett Cannon in :issue:`27186`.)
+
+See the summary for :ref:`PEP 519 <whatsnew36-pep519>` for details.
+
+
+pdb
+---
+
+The :class:`~pdb.Pdb` class constructor has a new optional *readrc* argument
+to control whether ``.pdbrc`` files should be read.
 
 
 pickle
@@ -933,6 +1238,34 @@
 Storchaka in :issue:`24164`.)
 
 
+pickletools
+-----------
+
+:func:`pickletools.dis()` now outputs implicit memo index for the
+``MEMOIZE`` opcode.
+(Contributed by Serhiy Storchaka in :issue:`25382`.)
+
+
+pydoc
+-----
+
+The :mod:`pydoc` module has learned to respect the ``MANPAGER``
+environment variable.
+(Contributed by Matthias Klose in :issue:`8637`.)
+
+:func:`help` and :mod:`pydoc` can now list named tuple fields in the
+order they were defined rather than alphabetically.
+(Contributed by Raymond Hettinger in :issue:`24879`.)
+
+
+random
+-------
+
+The new :func:`~random.choices` function returns a list of elements of
+specified size from the given population with optional weights.
+(Contributed by Raymond Hettinger in :issue:`18844`.)
+
+
 re
 --
 
@@ -945,6 +1278,11 @@
 equivalent to ``group()``.  So ``mo['name']`` is now equivalent to
 ``mo.group('name')``.  (Contributed by Eric Smith in :issue:`24454`.)
 
+:class:`~re.Match` objects in the now support
+:meth:`index-like objects <object.__index__>` as group
+indices.
+(Contributed by Jeroen Demeyer and Xiang Zhang in :issue:`27177`.)
+
 
 readline
 --------
@@ -966,6 +1304,16 @@
 an instance were excluded.  (Contributed by Martin Panter in :issue:`25590`.)
 
 
+shlex
+-----
+
+The :class:`~shlex.shlex` has much
+:ref:`improved shell compatibility <improved-shell-compatibility>`
+through the new *punctuation_chars* argument to control which characters
+are treated as punctuation.
+(Contributed by Vinay Sajip in :issue:`1521950`.)
+
+
 site
 ----
 
@@ -984,20 +1332,25 @@
 socket
 ------
 
-The :func:`~socket.socket.ioctl` function now supports the :data:`~socket.SIO_LOOPBACK_FAST_PATH`
-control code.
+The :func:`~socket.socket.ioctl` function now supports the
+:data:`~socket.SIO_LOOPBACK_FAST_PATH` control code.
 (Contributed by Daniel Stokes in :issue:`26536`.)
 
 The :meth:`~socket.socket.getsockopt` constants ``SO_DOMAIN``,
 ``SO_PROTOCOL``, ``SO_PEERSEC``, and ``SO_PASSSEC`` are now supported.
 (Contributed by Christian Heimes in :issue:`26907`.)
 
+The :meth:`~socket.socket.setsockopt` now supports the
+``setsockopt(level, optname, None, optlen: int)`` form.
+(Contributed by Christian Heimes in issue:`27744`.)
+
 The socket module now supports the address family
 :data:`~socket.AF_ALG` to interface with Linux Kernel crypto API. ``ALG_*``,
 ``SOL_ALG`` and :meth:`~socket.socket.sendmsg_afalg` were added.
 (Contributed by Christian Heimes in :issue:`27744` with support from
 Victor Stinner.)
 
+
 socketserver
 ------------
 
@@ -1013,16 +1366,15 @@
 calling :meth:`~io.BufferedIOBase.write` is now guaranteed to send the
 data in full.  (Contributed by Martin Panter in :issue:`26721`.)
 
+
 ssl
 ---
 
-:mod:`ssl` supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2.
-It has been tested with 0.9.8zc, 0.9.8zh and 1.0.1t as well as LibreSSL 2.3
-and 2.4.
+:mod:`ssl` supports OpenSSL 1.1.0.  The minimum recommend version is 1.0.2.
 (Contributed by Christian Heimes in :issue:`26470`.)
 
 3DES has been removed from the default cipher suites and ChaCha20 Poly1305
-cipher suites are now in the right position.
+cipher suites have been added.
 (Contributed by Christian Heimes in :issue:`27850` and :issue:`27766`.)
 
 :class:`~ssl.SSLContext` has better default configuration for options
@@ -1030,11 +1382,14 @@
 (Contributed by Christian Heimes in :issue:`28043`.)
 
 SSL session can be copied from one client-side connection to another
-with :class:`~ssl.SSLSession`. TLS session resumption can speed up
-the initial handshake, reduce latency and improve performance
+with the new :class:`~ssl.SSLSession` class.  TLS session resumption can
+speed up the initial handshake, reduce latency and improve performance
 (Contributed by Christian Heimes in :issue:`19500` based on a draft by
 Alex Warhawk.)
 
+The new :meth:`~ssl.SSLContext.get_ciphers` method can be used to
+get a list of enabled ciphers in order of cipher priority.
+
 All constants and flags have been converted to :class:`~enum.IntEnum` and
 :class:`~enum.IntFlags`.
 (Contributed by Christian Heimes in :issue:`28025`.)
@@ -1047,6 +1402,22 @@
 no longer case a SystemError.
 (Contributed by Christian Heimes in :issue:`27691`.)
 
+
+statistics
+----------
+
+A new :func:`~statistics.harmonic_mean` function has been added.
+(Contributed by Steven D'Aprano in :issue:`27181`.)
+
+
+struct
+------
+
+:mod:`struct` now supports IEEE 754 half-precision floats via the ``'e'``
+format specifier.
+(Contributed by Eli Stevens, Mark Dickinson in :issue:`11734`.)
+
+
 subprocess
 ----------
 
@@ -1059,6 +1430,22 @@
 The :class:`subprocess.Popen` constructor and all functions that pass arguments
 through to it now accept *encoding* and *errors* arguments. Specifying either
 of these will enable text mode for the *stdin*, *stdout* and *stderr* streams.
+(Contributed by Steve Dower in :issue:`6135`.)
+
+
+sys
+---
+
+The new :func:`~sys.getfilesystemencodeerrors` function returns the name of
+the error mode used to convert between Unicode filenames and bytes filenames.
+(Contributed by Steve Dower in :issue:`27781`.)
+
+On Windows the return value of the :func:`~sys.getwindowsversion` function
+now includes the *platform_version* field which contains the accurate major
+version, minor version and build number of the current operating system,
+rather than the version that is being emulated for the process
+(Contributed by Steve Dower in :issue:`27932`.)
+
 
 telnetlib
 ---------
@@ -1067,6 +1454,26 @@
 St?phane Wirtel in :issue:`25485`).
 
 
+time
+----
+
+The :class:`~time.struct_time` attributes :attr:`tm_gmtoff` and
+:attr:`tm_zone` are now available on all platforms.
+
+
+timeit
+------
+
+The new :meth:`Timer.autorange() <timeit.Timer.autorange>` convenience
+method has been added to call :meth:`Timer.timeit() <timeit.Timer.timeit>`
+repeatedly so that the total run time is greater or equal to 200 milliseconds.
+(Contributed by Steven D'Aprano in :issue:`6422`.)
+
+:mod:`timeit` now warns when there is substantial (4x) variance
+between best and worst times.
+(Contributed by Serhiy Storchaka in :issue:`23552`.)
+
+
 tkinter
 -------
 
@@ -1080,7 +1487,7 @@
 (Contributed by Serhiy Storchaka in :issue:`22115`).
 
 
-.. _py36-traceback:
+.. _whatsnew36-traceback:
 
 traceback
 ---------
@@ -1103,19 +1510,69 @@
 (Contributed by Emanuel Barry in :issue:`26823`.)
 
 
+tracemalloc
+-----------
+
+The :mod:`tracemalloc` module now supports tracing memory allocations in
+multiple different address spaces.
+
+The new :class:`~tracemalloc.DomainFilter` filter class has been added
+to filter block traces by their address space (domain).
+
+(Contributed by Victor Stinner in :issue:`26588`.)
+
+
+.. _whatsnew36-typing:
+
 typing
 ------
 
+Starting with Python 3.6 the :mod:`typing` module is no longer provisional
+and its API is considered stable.
+
+Since the :mod:`typing` module was :term:`provisional <provisional api>`
+in Python 3.5, all changes introduced in Python 3.6 have also been
+backported to Python 3.5.x.
+
 The :class:`typing.ContextManager` class has been added for
 representing :class:`contextlib.AbstractContextManager`.
 (Contributed by Brett Cannon in :issue:`25609`.)
 
+The :class:`typing.Collection` class has been added for
+representing :class:`collections.abc.Collection`.
+(Contributed by Ivan Levkivskyi in :issue:`27598`.)
+
+The :const:`typing.ClassVar` type construct has been added to
+mark class variables.  As introduced in :pep:`526`, a variable annotation
+wrapped in ClassVar indicates that a given attribute is intended to be used as
+a class variable and should not be set on instances of that class.
+(Contributed by Ivan Levkivskyi in `Github #280
+<https://github.com/python/typing/issues/280>`_.)
+
+A new :const:`~typing.TYPE_CHECKING` constant that is assumed to be
+``True`` by the static type chekers, but is ``False`` at runtime.
+(Contributed by Guido van Rossum in `Github #230
+<https://github.com/python/typing/issues/230>`_.)
+
+A new :func:`~typing.NewType` helper function has been added to create
+lightweight distinct types for annotations::
+
+    from typing import NewType
+
+    UserId = NewType('UserId', int)
+    some_id = UserId(524313)
+
+The static type checker will treat the new type as if it were a subclass
+of the original type.  (Contributed by Ivan Levkivskyi in `Github #189
+<https://github.com/python/typing/issues/189>`_.)
+
 
 unicodedata
 -----------
 
-The internal database has been upgraded to use Unicode 9.0.0. (Contributed by
-Benjamin Peterson.)
+The :mod:`unicodedata` module now uses data from `Unicode 9.0.0
+<http://unicode.org/versions/Unicode9.0.0/>`_.
+(Contributed by Benjamin Peterson.)
 
 
 unittest.mock
@@ -1129,12 +1586,17 @@
   was called.
   (Contributed by Amit Saha in :issue:`26323`.)
 
+* The :meth:`Mock.reset_mock() <unittest.mock.Mock.reset_mock>` method
+  now has two optional keyword only arguments: *return_value* and
+  *side_effect*.
+  (Contributed by Kushal Das in :issue:`21271`.)
+
 
 urllib.request
 --------------
 
 If a HTTP request has a file or iterable body (other than a
-bytes object) but no Content-Length header, rather than
+bytes object) but no ``Content-Length`` header, rather than
 throwing an error, :class:`~urllib.request.AbstractHTTPHandler` now
 falls back to use chunked transfer encoding.
 (Contributed by Demian Brecht and Rolf Krahl in :issue:`12319`.)
@@ -1148,6 +1610,14 @@
 (Contributed by Nikolay Bogoychev in :issue:`16099`.)
 
 
+venv
+----
+
+:mod:`venv` accepts a new parameter ``--prompt``. This parameter provides an
+alternative prefix for the virtual environment. (Proposed by ?ukasz Balcerzak
+and ported to 3.6 by St?phane Wirtel in :issue:`22829`.)
+
+
 warnings
 --------
 
@@ -1203,8 +1673,9 @@
 xmlrpc.client
 -------------
 
-The module now supports unmarshalling additional data types used by
-Apache XML-RPC implementation for numerics and ``None``.
+The :mod:`xmlrpc.client` module now supports unmarshalling
+additional data types used by Apache XML-RPC implementation
+for numerics and ``None``.
 (Contributed by Serhiy Storchaka in :issue:`26885`.)
 
 
@@ -1225,20 +1696,27 @@
 zlib
 ----
 
-The :func:`~zlib.compress` function now accepts keyword arguments.
-(Contributed by Aviv Palivoda in :issue:`26243`.)
-
-
-fileinput
----------
-
-:func:`~fileinput.hook_encoded` now supports the *errors* argument.
-(Contributed by Joseph Hackman in :issue:`25788`.)
+The :func:`~zlib.compress` and :func:`~zlib.decompress` functions now accept
+keyword arguments.
+(Contributed by Aviv Palivoda in :issue:`26243` and
+Xiang Zhang in :issue:`16764` respectively.)
 
 
 Optimizations
 =============
 
+* Python interpreter now uses 16-bit wordcode instead of bytecode which
+  made a number of opcode optimizations possible.
+  (Contributed by Demur Rumed with input and reviews from
+  Serhiy Storchaka and Victor Stinner in :issue:`26647` and :issue:`28050`.)
+
+* The :class:`Future <asyncio.futures.Future>` now has an optimized
+  C implementation.
+  (Contributed by Yury Selivanov and INADA Naoki in :issue:`26801`.)
+
+* The :class:`Task <asyncio.tasks.Task>` now has an optimized
+  C implementation. (Contributed by Yury Selivanov in :issue:`28544`.)
+
 * The ASCII decoder is now up to 60 times as fast for error handlers
   ``surrogateescape``, ``ignore`` and ``replace`` (Contributed
   by Victor Stinner in :issue:`24870`).
@@ -1291,12 +1769,23 @@
   it is now about 1.5--4 times faster.
   (Contributed by Serhiy Storchaka in :issue:`26032`).
 
+* :class:`xml.etree.ElementTree` parsing, iteration and deepcopy performance
+  has been significantly improved.
+  (Contributed by Serhiy Storchaka in :issue:`25638`, :issue:`25873`,
+  and :issue:`25869`.)
+
+* Creation of :class:`fractions.Fraction` instances from floats and
+  decimals is now 2 to 3 times faster.
+  (Contributed by Serhiy Storchaka in :issue:`25971`.)
+
 
 Build and C API Changes
 =======================
 
-* Python now requires some C99 support in the toolchain to build. For more
-  information, see :pep:`7`.
+* Python now requires some C99 support in the toolchain to build.
+  Most notably, Python now uses standard integer types and macros in
+  place of custom macros like ``PY_LONG_LONG``.
+  For more information, see :pep:`7` and :issue:`17884`.
 
 * Cross-compiling CPython with the Android NDK and the Android API level set to
   21 (Android 5.0 Lollilop) or greater, runs successfully. While Android is not
@@ -1307,8 +1796,13 @@
   will activate expensive optimizations like PGO.
   (Original patch by Alecsandru Patrascu of Intel in :issue:`26539`.)
 
+* The :term:`GIL <global interpreter lock>` must now be held when allocator
+  functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and
+  :c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called.
+
 * New :c:func:`Py_FinalizeEx` API which indicates if flushing buffered data
-  failed (:issue:`5319`).
+  failed.
+  (Contributed by Martin Panter in :issue:`5319`.)
 
 * :c:func:`PyArg_ParseTupleAndKeywords` now supports :ref:`positional-only
   parameters <positional-only_parameter>`.  Positional-only parameters are
@@ -1319,112 +1813,178 @@
   as ``"[Previous line repeated {count} more times]"``.
   (Contributed by Emanuel Barry in :issue:`26823`.)
 
+* The new :c:func:`PyErr_SetImportErrorSubclass` function allows for
+  specifying a subclass of :exc:`ImportError` to raise.
+  (Contributed by Eric Snow in :issue:`15767`.)
+
+* The new :c:func:`PyErr_ResourceWarning` function can be used to generate
+  the :exc:`ResourceWarning` providing the source of the resource allocation.
+  (Contributed by Victor Stinner in :issue:`26567`.)
+
+* The new :c:func:`PyOS_FSPath` function returns the file system
+  representation of a :term:`path-like object`.
+  (Contributed by Brett Cannon in :issue:`27186`.)
+
+* The :c:func:`PyUnicode_FSConverter` and :c:func:`PyUnicode_FSDecoder`
+  functions will now accept :term:`path-like objects <path-like object>`.
+
+
 
 Deprecated
 ==========
 
+New Keywords
+------------
+
+``async`` and ``await`` are not recommended to be used as variable, class,
+function or module names.  Introduced by :pep:`492` in Python 3.5, they will
+become proper keywords in Python 3.7.  Starting in Python 3.6, the use of
+``async`` or ``await`` as names will generate a :exc:`DeprecationWarning`.
+
+
+Deprecated Python behavior
+--------------------------
+
+Raising the :exc:`StopIteration` exception inside a generator will now
+generate a :exc:`DeprecationWarning`, and will trigger a :exc:`RuntimeError`
+in Python 3.7.  See :ref:`whatsnew-pep-479` for details.
+
+The :meth:`__aiter__` method is now expected to return an asynchronous
+iterator directly instead of returning an awaitable as previously.
+Doing the former will trigger a :exc:`DeprecationWarning`.  Backward
+compatibility will be removed in Python 3.7.
+(Contributed by Yury Selivanov in :issue:`27243`.)
+
+A backslash-character pair that is not a valid escape sequence now generates
+a :exc:`DeprecationWarning`.  Although this will eventually become a
+:exc:`SyntaxError`, that will not be for several Python releases.
+(Contributed by Emanuel Barry in :issue:`27364`.)
+
+When performing a relative import, falling back on ``__name__`` and
+``__path__`` from the calling module when ``__spec__`` or
+``__package__`` are not defined now raises an :exc:`ImportWarning`.
+(Contributed by Rose Ames in :issue:`25791`.)
+
+
+Deprecated Python modules, functions and methods
+------------------------------------------------
+
+asynchat
+~~~~~~~~
+
+The :mod:`asynchat` has been deprecated in favor of :mod:`asyncio`.
+(Contributed by Mariatta in :issue:`25002`.)
+
+
+asyncore
+~~~~~~~~
+
+The :mod:`asyncore` has been deprecated in favor of :mod:`asyncio`.
+(Contributed by Mariatta in :issue:`25002`.)
+
+
+dbm
+~~~
+
+Unlike other :mod:`dbm` implementations, the :mod:`dbm.dumb` module
+creates databases with the ``'rw'`` mode and allows modifying the database
+opened with the ``'r'`` mode.  This behavior is now deprecated and will
+be removed in 3.8.
+(Contributed by Serhiy Storchaka in :issue:`21708`.)
+
+
+distutils
+~~~~~~~~~
+
+The undocumented ``extra_path`` argument to the
+:class:`~distutils.Distribution` constructor is now considered deprecated
+and will raise a warning if set.   Support for this parameter will be
+removed in a future Python release.  See :issue:`27919` for details.
+
+
+grp
+~~~
+
+The support of non-integer arguments in :func:`~grp.getgrgid` has been
+deprecated.
+(Contributed by Serhiy Storchaka in :issue:`26129`.)
+
+
+importlib
+~~~~~~~~~
+
+The :meth:`importlib.machinery.SourceFileLoader.load_module` and
+:meth:`importlib.machinery.SourcelessFileLoader.load_module` methods
+are now deprecated. They were the only remaining implementations of
+:meth:`importlib.abc.Loader.load_module` in :mod:`importlib` that had not
+been deprecated in previous versions of Python in favour of
+:meth:`importlib.abc.Loader.exec_module`.
+
+os
+~~
+
+Undocumented support of general :term:`bytes-like objects <bytes-like object>`
+as paths in :mod:`os` functions, :func:`compile` and similar functions is
+now deprecated.
+(Contributed by Serhiy Storchaka in :issue:`25791` and :issue:`26754`.)
+
+re
+~~
+
+Support for inline flags ``(?letters)`` in the middle of the regular
+expression has been deprecated and will be removed in a future Python
+version.  Flags at the start of a regular expression are still allowed.
+(Contributed by Serhiy Storchaka in :issue:`22493`.)
+
+ssl
+~~~
+
+OpenSSL 0.9.8, 1.0.0 and 1.0.1 are deprecated and no longer supported.
+In the future the :mod:`ssl` module will require at least OpenSSL 1.0.2 or
+1.1.0.
+
+SSL-related arguments like ``certfile``, ``keyfile`` and ``check_hostname``
+in :mod:`ftplib`, :mod:`http.client`, :mod:`imaplib`, :mod:`poplib`,
+and :mod:`smtplib` have been deprecated in favor of ``context``.
+(Contributed by Christian Heimes in :issue:`28022`.)
+
+A couple of protocols and functions of the :mod:`ssl` module are now
+deprecated. Some features will no longer be available in future versions
+of OpenSSL. Other features are deprecated in favor of a different API.
+(Contributed by Christian Heimes in :issue:`28022` and :issue:`26470`.)
+
+tkinter
+~~~~~~~
+
+The :mod:`tkinter.tix` module is now deprecated.  :mod:`tkinter` users
+should use :mod:`tkinter.ttk` instead.
+
+venv
+~~~~
+
+The ``pyvenv`` script has been deprecated in favour of ``python3 -m venv``.
+This prevents confusion as to what Python interpreter ``pyvenv`` is
+connected to and thus what Python interpreter will be used by the virtual
+environment.  (Contributed by Brett Cannon in :issue:`25154`.)
+
+
+Deprecated functions and types of the C API
+-------------------------------------------
+
+Undocumented functions :c:func:`PyUnicode_AsEncodedObject`,
+:c:func:`PyUnicode_AsDecodedObject`, :c:func:`PyUnicode_AsEncodedUnicode`
+and :c:func:`PyUnicode_AsDecodedUnicode` are deprecated now.
+Use :ref:`generic codec based API <codec-registry>` instead.
+
+
 Deprecated Build Options
 ------------------------
 
-The ``--with-system-ffi`` configure flag is now on by default on non-OSX UNIX
-platforms.  It may be disabled by using ``--without-system-ffi``, but using the
-flag is deprecated and will not be accepted in Python 3.7.  OSX is unaffected
-by this change.  Note that many OS distributors already use the
-``--with-system-ffi`` flag when building their system Python.
-
-
-New Keywords
-------------
-
-``async`` and ``await`` are not recommended to be used as variable, class,
-function or module names.  Introduced by :pep:`492` in Python 3.5, they will
-become proper keywords in Python 3.7.
-
-
-Deprecated Python modules, functions and methods
-------------------------------------------------
-
-* :meth:`importlib.machinery.SourceFileLoader.load_module` and
-  :meth:`importlib.machinery.SourcelessFileLoader.load_module` are now
-  deprecated. They were the only remaining implementations of
-  :meth:`importlib.abc.Loader.load_module` in :mod:`importlib` that had not
-  been deprecated in previous versions of Python in favour of
-  :meth:`importlib.abc.Loader.exec_module`.
-
-* The :mod:`tkinter.tix` module is now deprecated.  :mod:`tkinter` users should
-  use :mod:`tkinter.ttk` instead.
-
-
-Deprecated functions and types of the C API
--------------------------------------------
-
-* Undocumented functions :c:func:`PyUnicode_AsEncodedObject`,
-  :c:func:`PyUnicode_AsDecodedObject`, :c:func:`PyUnicode_AsEncodedUnicode`
-  and :c:func:`PyUnicode_AsDecodedUnicode` are deprecated now.
-  Use :ref:`generic codec based API <codec-registry>` instead.
-
-
-Deprecated features
--------------------
-
-* The ``pyvenv`` script has been deprecated in favour of ``python3 -m venv``.
-  This prevents confusion as to what Python interpreter ``pyvenv`` is
-  connected to and thus what Python interpreter will be used by the virtual
-  environment. (Contributed by Brett Cannon in :issue:`25154`.)
-
-* When performing a relative import, falling back on ``__name__`` and
-  ``__path__`` from the calling module when ``__spec__`` or
-  ``__package__`` are not defined now raises an :exc:`ImportWarning`.
-  (Contributed by Rose Ames in :issue:`25791`.)
-
-* Unlike to other :mod:`dbm` implementations, the :mod:`dbm.dumb` module
-  creates database in ``'r'`` and ``'w'`` modes if it doesn't exist and
-  allows modifying database in ``'r'`` mode.  This behavior is now deprecated
-  and will be removed in 3.8.
-  (Contributed by Serhiy Storchaka in :issue:`21708`.)
-
-* Undocumented support of general :term:`bytes-like objects <bytes-like object>`
-  as paths in :mod:`os` functions, :func:`compile` and similar functions is
-  now deprecated.
-  (Contributed by Serhiy Storchaka in :issue:`25791` and :issue:`26754`.)
-
-* The undocumented ``extra_path`` argument to a distutils Distribution
-  is now considered
-  deprecated, will raise a warning during install if set. Support for this
-  parameter will be dropped in a future Python release and likely earlier
-  through third party tools. See :issue:`27919` for details.
-
-* A backslash-character pair that is not a valid escape sequence now generates
-  a DeprecationWarning.  Although this will eventually become a SyntaxError,
-  that will not be for several Python releases.  (Contributed by Emanuel Barry
-  in :issue:`27364`.)
-
-* Inline flags ``(?letters)`` now should be used only at the start of the
-  regular expression.  Inline flags in the middle of the regular expression
-  affects global flags in Python :mod:`re` module.  This is an exception to
-  other regular expression engines that either apply flags to only part of
-  the regular expression or treat them as an error.  To avoid distinguishing
-  inline flags in the middle of the regular expression now emit a deprecation
-  warning.  It will be an error in future Python releases.
-  (Contributed by Serhiy Storchaka in :issue:`22493`.)
-
-* SSL-related arguments like ``certfile``, ``keyfile`` and ``check_hostname``
-  in :mod:`ftplib`, :mod:`http.client`, :mod:`imaplib`, :mod:`poplib`,
-  and :mod:`smtplib` have been deprecated in favor of ``context``.
-  (Contributed by Christian Heimes in :issue:`28022`.)
-
-* A couple of protocols and functions of the :mod:`ssl` module are now
-  deprecated. Some features will no longer be available in future versions
-  of OpenSSL. Other features are deprecated in favor of a different API.
-  (Contributed by Christian Heimes in :issue:`28022` and :issue:`26470`.)
-
-
-Deprecated Python behavior
---------------------------
-
-* Raising the :exc:`StopIteration` exception inside a generator will now generate a
-  :exc:`DeprecationWarning`, and will trigger a :exc:`RuntimeError` in Python 3.7.
-  See :ref:`whatsnew-pep-479` for details.
+The ``--with-system-ffi`` configure flag is now on by default on non-macOS
+UNIX platforms.  It may be disabled by using ``--without-system-ffi``, but
+using the flag is deprecated and will not be accepted in Python 3.7.
+macOS is unaffected by this change.  Note that many OS distributors already
+use the ``--with-system-ffi`` flag when building their system Python.
 
 
 Removed
@@ -1433,9 +1993,14 @@
 API and Feature Removals
 ------------------------
 
+* Unknown escapes consisting of ``'\'`` and an ASCII letter in
+  regular expressions will now cause an error.  The :const:`re.LOCALE`
+  flag can now only be used with binary patterns.
+
 * ``inspect.getmoduleinfo()`` was removed (was deprecated since CPython 3.3).
   :func:`inspect.getmodulename` should be used for obtaining the module
   name for a given path.
+  (Contributed by Yury Selivanov in :issue:`13248`.)
 
 * ``traceback.Ignore`` class and ``traceback.usage``, ``traceback.modname``,
   ``traceback.fullmodname``, ``traceback.find_lines_from_code``,
@@ -1460,6 +2025,8 @@
   script that created these modules is still available in the source
   distribution at :source:`Tools/scripts/h2py.py`.
 
+* The deprecated ``asynchat.fifo`` class has been removed.
+
 
 Porting to Python 3.6
 =====================
@@ -1480,6 +2047,10 @@
 Changes in the Python API
 -------------------------
 
+* :func:`open() <open>` will no longer allow combining the ``'U'`` mode flag
+  with ``'+'``.
+  (Contributed by Jeff Balogh and John O'Connor in :issue:`2091`.)
+
 * :mod:`sqlite3` no longer implicitly commit an open transaction before DDL
   statements.
 
@@ -1523,7 +2094,8 @@
   :mod:`mimetypes`, :mod:`optparse`, :mod:`plistlib`, :mod:`smtpd`,
   :mod:`subprocess`, :mod:`tarfile`, :mod:`threading` and
   :mod:`wave`.  This means they will export new symbols when ``import *``
-  is used.  See :issue:`23883`.
+  is used.
+  (Contributed by Joel Taddei and Jacek Ko?odziej in :issue:`23883`.)
 
 * When performing a relative import, if ``__package__`` does not compare equal
   to ``__spec__.parent`` then :exc:`ImportWarning` is raised.
@@ -1546,8 +2118,8 @@
   :exc:`KeyError` if the user doesn't have privileges.
 
 * The :meth:`socket.socket.close` method now raises an exception if
-  an error (e.g. EBADF) was reported by the underlying system call.
-  See :issue:`26685`.
+  an error (e.g. ``EBADF``) was reported by the underlying system call.
+  (Contributed by Martin Panter in :issue:`26685`.)
 
 * The *decode_data* argument for :class:`smtpd.SMTPChannel` and
   :class:`smtpd.SMTPServer` constructors is now ``False`` by default.
@@ -1557,13 +2129,16 @@
   Code that has already been updated in accordance with the deprecation
   warning generated by 3.5 will not be affected.
 
-* All optional parameters of the :func:`~json.dump`, :func:`~json.dumps`,
+* All optional arguments of the :func:`~json.dump`, :func:`~json.dumps`,
   :func:`~json.load` and :func:`~json.loads` functions and
   :class:`~json.JSONEncoder` and :class:`~json.JSONDecoder` class
   constructors in the :mod:`json` module are now :ref:`keyword-only
   <keyword-only_parameter>`.
   (Contributed by Serhiy Storchaka in :issue:`18726`.)
 
+* Subclasses of :class:`type` which don't override ``type.__new__`` may no
+  longer use the one-argument form to get the type of an object.
+
 * As part of :pep:`487`, the handling of keyword arguments passed to
   :class:`type` (other than the metaclass hint, ``metaclass``) is now
   consistently delegated to :meth:`object.__init_subclass__`. This means that
@@ -1592,7 +2167,63 @@
   header field has been specified and the request body is a file object,
   it is now sent with HTTP 1.1 chunked encoding. If a file object has to
   be sent to a HTTP 1.0 server, the Content-Length value now has to be
-  specified by the caller. See :issue:`12319`.
+  specified by the caller.
+  (Contributed by Demian Brecht and Rolf Krahl with tweaks from
+  Martin Panter in :issue:`12319`.)
+
+* The :class:`~csv.DictReader` now returns rows of type
+  :class:`~collections.OrderedDict`.
+  (Contributed by Steve Holden in :issue:`27842`.)
+
+* The :const:`crypt.METHOD_CRYPT` will no longer be added to ``crypt.methods``
+  if unsupported by the platform.
+  (Contributed by Victor Stinner in :issue:`25287`.)
+
+* The *verbose* and *rename* arguments for
+  :func:`~collections.namedtuple` are now keyword-only.
+  (Contributed by Raymond Hettinger in :issue:`25628`.)
+
+* The :meth:`~cgi.FieldStorage.read_multi` method now ignores the
+  ``Content-Length`` header in part headers.
+  (Contributed by Peter Landry in :issue:`24764`.)
+
+* On Linux, :func:`ctypes.util.find_library` now looks in
+  ``LD_LIBRARY_PATH`` for shared libraries.
+  (Contributed by Vinay Sajip in :issue:`9998`.)
+
+* The :meth:`datetime.fromtimestamp() <datetime.datetime.fromtimestamp>` and
+  :meth:`datetime.utcfromtimestamp() <datetime.datetime.utcfromtimestamp>`
+  methods now round microseconds to nearest with ties going to
+  nearest even integer (``ROUND_HALF_EVEN``), instead of
+  rounding towards negative infinity (``ROUND_FLOOR``).
+  (Contributed by Victor Stinner in :issue:`23517`.)
+
+* The :class:`imaplib.IMAP4` class now handles flags containing the
+  ``']'`` character in messages sent from the server to improve
+  real-world compatibility.
+  (Contributed by Lita Cho in :issue:`21815`.)
+
+* The :func:`mmap.write() <mmap.write>` function now returns the number
+  of bytes written like other write methods.
+  (Contributed by Jakub Stasiak in :issue:`26335`.)
+
+* The :func:`pkgutil.iter_modules` and :func:`pkgutil.walk_packages`
+  functions now return :class:`~pkgutil.ModuleInfo` named tuples.
+  (Contributed by Ramchandra Apte in :issue:`17211`.)
+
+* :func:`re.sub` now raises an error for invalid numerical group
+  reference in replacement template even if the pattern is not
+  found in the string.  Error message for invalid group reference
+  now includes the group index and the position of the reference.
+  (Contributed by SilentGhost, Serhiy Storchaka in :issue:`25953`.)
+
+* :class:`zipfile.ZipFile` will now raise :exc:`NotImplementedError` for
+  unrecognized compression values.  Previously a plain :exc:`RuntimeError`
+  was raised.  Additionally, calling :class:`~zipfile.ZipFile` methods or
+  on a closed ZipFile or calling :meth:`~zipfile.ZipFile.write` methods
+  on a ZipFile created with mode ``'r'`` will raise a :exc:`ValueError`.
+  Previously, a :exc:`RuntimeError` was raised in those scenarios.
+
 
 Changes in the C API
 --------------------

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 10 15:39:50 2016
From: python-checkins at python.org (yury.selivanov)
Date: Thu, 10 Nov 2016 20:39:50 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg2MzUp?=
Message-ID: <20161110203950.124449.75340.105FA223@psf.io>

https://hg.python.org/cpython/rev/8ebaa546a033
changeset:   105044:8ebaa546a033
parent:      105042:69bdf8bdfd61
parent:      105043:4c81a107ccab
user:        Yury Selivanov <yury at magic.io>
date:        Thu Nov 10 15:39:46 2016 -0500
summary:
  Merge 3.6 (issue #28635)

files:
  Doc/whatsnew/3.6.rst |  40 +------------------------------
  1 files changed, 2 insertions(+), 38 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -93,7 +93,7 @@
 * Customization of class creation has been simplified with the
   :ref:`new protocol <whatsnew36-pep487>`.
 
-* The class attibute definition order is
+* The class attribute definition order is
   :ref:`now preserved  <whatsnew36-pep520>`.
 
 * The order of elements in ``**kwargs`` now
@@ -127,7 +127,7 @@
   :ref:`improvements <whatsnew36-typing>` and is no longer provisional.
 
 * The :mod:`tracemalloc` module has been significantly reworked
-  and is now used to provide better output for :exc:`ResourceWarning`s
+  and is now used to provide better output for :exc:`ResourceWarning`
   as well as provide better diagnostics for memory allocation errors.
   See the :ref:`PYTHONMALLOC section <whatsnew36-pythonmalloc>` for more
   information.
@@ -874,14 +874,6 @@
   (Contributed by Jim Fulton in :issue:`27392`.)
 
 
-base64
-------
-
-The :func:`~base64.a85decode` function no longer requires the leading
-``'<~'`` characters in input when the *adobe* argument is set.
-(Contributed by Swati Jaiswal in :issue:`25913`.)
-
-
 binascii
 --------
 
@@ -984,15 +976,6 @@
 (Contributed by Stefan Krah amd Mark Dickinson in :issue:`25928`.)
 
 
-dis
----
-
-Disassembling a class now disassembles class and static
-methods.  (Contributed by Xiang Zhang in :issue:`26733`.)
-
-The disassembler now decodes ``FORMAT_VALUE`` argument.
-(Contributed by Serhiy Storchaka in :issue:`28317`.)
-
 
 distutils
 ---------
@@ -1299,10 +1282,6 @@
 with underscores.  A space or a colon is added after some completed keywords.
 (Contributed by Serhiy Storchaka in :issue:`25011` and :issue:`25209`.)
 
-Names of most attributes listed by :func:`dir` are now completed.
-Previously, names of properties and slots which were not yet created on
-an instance were excluded.  (Contributed by Martin Panter in :issue:`25590`.)
-
 
 shlex
 -----
@@ -1398,10 +1377,6 @@
 were added.
 (Contributed by Christian Heimes in :issue:`28085`.)
 
-General resource ids (``GEN_RID``) in subject alternative name extensions
-no longer case a SystemError.
-(Contributed by Christian Heimes in :issue:`27691`.)
-
 
 statistics
 ----------
@@ -2183,21 +2158,10 @@
   :func:`~collections.namedtuple` are now keyword-only.
   (Contributed by Raymond Hettinger in :issue:`25628`.)
 
-* The :meth:`~cgi.FieldStorage.read_multi` method now ignores the
-  ``Content-Length`` header in part headers.
-  (Contributed by Peter Landry in :issue:`24764`.)
-
 * On Linux, :func:`ctypes.util.find_library` now looks in
   ``LD_LIBRARY_PATH`` for shared libraries.
   (Contributed by Vinay Sajip in :issue:`9998`.)
 
-* The :meth:`datetime.fromtimestamp() <datetime.datetime.fromtimestamp>` and
-  :meth:`datetime.utcfromtimestamp() <datetime.datetime.utcfromtimestamp>`
-  methods now round microseconds to nearest with ties going to
-  nearest even integer (``ROUND_HALF_EVEN``), instead of
-  rounding towards negative infinity (``ROUND_FLOOR``).
-  (Contributed by Victor Stinner in :issue:`23517`.)
-
 * The :class:`imaplib.IMAP4` class now handles flags containing the
   ``']'`` character in messages sent from the server to improve
   real-world compatibility.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 10 15:39:50 2016
From: python-checkins at python.org (yury.selivanov)
Date: Thu, 10 Nov 2016 20:39:50 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NjM1?=
 =?utf-8?q?=3A_what=27s_new_in_3=2E6=3A_remove_mentions_of_backported_fixe?=
 =?utf-8?q?s=2E?=
Message-ID: <20161110203950.97050.77668.CF93CBE7@psf.io>

https://hg.python.org/cpython/rev/4c81a107ccab
changeset:   105043:4c81a107ccab
branch:      3.6
parent:      105041:c0060567f35d
user:        Yury Selivanov <yury at magic.io>
date:        Thu Nov 10 15:39:27 2016 -0500
summary:
  Issue #28635: what's new in 3.6: remove mentions of backported fixes.

Patch by Elvis Pranskevichus.

files:
  Doc/whatsnew/3.6.rst |  40 +------------------------------
  1 files changed, 2 insertions(+), 38 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -93,7 +93,7 @@
 * Customization of class creation has been simplified with the
   :ref:`new protocol <whatsnew36-pep487>`.
 
-* The class attibute definition order is
+* The class attribute definition order is
   :ref:`now preserved  <whatsnew36-pep520>`.
 
 * The order of elements in ``**kwargs`` now
@@ -127,7 +127,7 @@
   :ref:`improvements <whatsnew36-typing>` and is no longer provisional.
 
 * The :mod:`tracemalloc` module has been significantly reworked
-  and is now used to provide better output for :exc:`ResourceWarning`s
+  and is now used to provide better output for :exc:`ResourceWarning`
   as well as provide better diagnostics for memory allocation errors.
   See the :ref:`PYTHONMALLOC section <whatsnew36-pythonmalloc>` for more
   information.
@@ -874,14 +874,6 @@
   (Contributed by Jim Fulton in :issue:`27392`.)
 
 
-base64
-------
-
-The :func:`~base64.a85decode` function no longer requires the leading
-``'<~'`` characters in input when the *adobe* argument is set.
-(Contributed by Swati Jaiswal in :issue:`25913`.)
-
-
 binascii
 --------
 
@@ -984,15 +976,6 @@
 (Contributed by Stefan Krah amd Mark Dickinson in :issue:`25928`.)
 
 
-dis
----
-
-Disassembling a class now disassembles class and static
-methods.  (Contributed by Xiang Zhang in :issue:`26733`.)
-
-The disassembler now decodes ``FORMAT_VALUE`` argument.
-(Contributed by Serhiy Storchaka in :issue:`28317`.)
-
 
 distutils
 ---------
@@ -1299,10 +1282,6 @@
 with underscores.  A space or a colon is added after some completed keywords.
 (Contributed by Serhiy Storchaka in :issue:`25011` and :issue:`25209`.)
 
-Names of most attributes listed by :func:`dir` are now completed.
-Previously, names of properties and slots which were not yet created on
-an instance were excluded.  (Contributed by Martin Panter in :issue:`25590`.)
-
 
 shlex
 -----
@@ -1398,10 +1377,6 @@
 were added.
 (Contributed by Christian Heimes in :issue:`28085`.)
 
-General resource ids (``GEN_RID``) in subject alternative name extensions
-no longer case a SystemError.
-(Contributed by Christian Heimes in :issue:`27691`.)
-
 
 statistics
 ----------
@@ -2183,21 +2158,10 @@
   :func:`~collections.namedtuple` are now keyword-only.
   (Contributed by Raymond Hettinger in :issue:`25628`.)
 
-* The :meth:`~cgi.FieldStorage.read_multi` method now ignores the
-  ``Content-Length`` header in part headers.
-  (Contributed by Peter Landry in :issue:`24764`.)
-
 * On Linux, :func:`ctypes.util.find_library` now looks in
   ``LD_LIBRARY_PATH`` for shared libraries.
   (Contributed by Vinay Sajip in :issue:`9998`.)
 
-* The :meth:`datetime.fromtimestamp() <datetime.datetime.fromtimestamp>` and
-  :meth:`datetime.utcfromtimestamp() <datetime.datetime.utcfromtimestamp>`
-  methods now round microseconds to nearest with ties going to
-  nearest even integer (``ROUND_HALF_EVEN``), instead of
-  rounding towards negative infinity (``ROUND_FLOOR``).
-  (Contributed by Victor Stinner in :issue:`23517`.)
-
 * The :class:`imaplib.IMAP4` class now handles flags containing the
   ``']'`` character in messages sent from the server to improve
   real-world compatibility.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 10 18:43:13 2016
From: python-checkins at python.org (terry.reedy)
Date: Thu, 10 Nov 2016 23:43:13 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1NTA3?=
 =?utf-8?q?=3A_Add_back_import_needed_for_2=2Ex_encoding_warning_box=2E?=
Message-ID: <20161110234313.39869.16356.0743AB4E@psf.io>

https://hg.python.org/cpython/rev/137c7b92360e
changeset:   105045:137c7b92360e
branch:      2.7
parent:      105009:41613bb27f80
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Thu Nov 10 18:42:58 2016 -0500
summary:
  Issue #25507: Add back import needed for 2.x encoding warning box.
Add pointer to 'Encoding declaration' in Language Reference.

files:
  Lib/idlelib/IOBinding.py |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Lib/idlelib/IOBinding.py b/Lib/idlelib/IOBinding.py
--- a/Lib/idlelib/IOBinding.py
+++ b/Lib/idlelib/IOBinding.py
@@ -13,6 +13,7 @@
 import sys
 import tempfile
 
+from Tkinter import *
 import tkFileDialog
 import tkMessageBox
 from SimpleDialog import SimpleDialog
@@ -91,6 +92,7 @@
         # l2['state'] = DISABLED
         l2.pack(side=TOP, anchor = W, fill=X)
         l3 = Label(top, text="to your file\n"
+                   "See Language Reference, 2.1.4 Encoding declarations.\n"
                    "Choose OK to save this file as %s\n"
                    "Edit your general options to silence this warning" % enc)
         l3.pack(side=TOP, anchor = W)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 10 19:50:47 2016
From: python-checkins at python.org (victor.stinner)
Date: Fri, 11 Nov 2016 00:50:47 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Use_PyThreadState=5FGET=28?=
 =?utf-8?q?=29_in_performance_critical_code?=
Message-ID: <20161111005047.101604.83276.179CD37A@psf.io>

https://hg.python.org/cpython/rev/c2c72616c9f1
changeset:   105046:c2c72616c9f1
parent:      105044:8ebaa546a033
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Fri Nov 11 01:43:56 2016 +0100
summary:
  Use PyThreadState_GET() in performance critical code

It seems like _PyThreadState_UncheckedGet() is not inlined as expected, even
when using gcc -O3.

files:
  Objects/dictobject.c |  2 +-
  Python/errors.c      |  2 +-
  Python/sysmodule.c   |  3 ++-
  3 files changed, 4 insertions(+), 3 deletions(-)


diff --git a/Objects/dictobject.c b/Objects/dictobject.c
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -1409,7 +1409,7 @@
        Let's just hope that no exception occurs then...  This must be
        _PyThreadState_Current and not PyThreadState_GET() because in debug
        mode, the latter complains if tstate is NULL. */
-    tstate = _PyThreadState_UncheckedGet();
+    tstate = PyThreadState_GET();
     if (tstate != NULL && tstate->curexc_type != NULL) {
         /* preserve the existing exception */
         PyObject *err_type, *err_value, *err_tb;
diff --git a/Python/errors.c b/Python/errors.c
--- a/Python/errors.c
+++ b/Python/errors.c
@@ -161,7 +161,7 @@
 PyObject *
 PyErr_Occurred(void)
 {
-    PyThreadState *tstate = _PyThreadState_UncheckedGet();
+    PyThreadState *tstate = PyThreadState_GET();
     return tstate == NULL ? NULL : tstate->curexc_type;
 }
 
diff --git a/Python/sysmodule.c b/Python/sysmodule.c
--- a/Python/sysmodule.c
+++ b/Python/sysmodule.c
@@ -1547,8 +1547,9 @@
     Py_XDECREF(name);
     Py_XDECREF(value);
     /* No return value, therefore clear error state if possible */
-    if (_PyThreadState_UncheckedGet())
+    if (_PyThreadState_UncheckedGet()) {
         PyErr_Clear();
+    }
 }
 
 PyObject *

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 10 20:14:20 2016
From: python-checkins at python.org (victor.stinner)
Date: Fri, 11 Nov 2016 01:14:20 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328618=3A_Make_hot?=
 =?utf-8?b?IGZ1bmN0aW9ucyB1c2luZyBfX2F0dHJpYnV0ZV9fKChob3QpKQ==?=
Message-ID: <20161111011420.13061.52644.B5CE9583@psf.io>

https://hg.python.org/cpython/rev/59b91b4e9506
changeset:   105047:59b91b4e9506
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Fri Nov 11 02:13:35 2016 +0100
summary:
  Issue #28618: Make hot functions using __attribute__((hot))

When Python is not compiled with PGO, the performance of Python on call_simple
and call_method microbenchmarks depend highly on the code placement. In the
worst case, the performance slowdown can be up to 70%.

The GCC __attribute__((hot)) attribute helps to keep hot code close to reduce
the risk of such major slowdown. This attribute is ignored when Python is
compiled with PGO.

The following functions are considered as hot according to statistics collected
by perf record/perf report:

* _PyEval_EvalFrameDefault()
* call_function()
* _PyFunction_FastCall()
* PyFrame_New()
* frame_dealloc()
* PyErr_Occurred()

files:
  Include/pyport.h      |  27 +++++++++++++++++++++++++--
  Objects/frameobject.c |   4 ++--
  Python/ceval.c        |   6 +++---
  Python/errors.c       |   2 +-
  4 files changed, 31 insertions(+), 8 deletions(-)


diff --git a/Include/pyport.h b/Include/pyport.h
--- a/Include/pyport.h
+++ b/Include/pyport.h
@@ -490,13 +490,36 @@
  *    typedef int T1 Py_DEPRECATED(2.4);
  *    extern int x() Py_DEPRECATED(2.5);
  */
-#if defined(__GNUC__) && ((__GNUC__ >= 4) || \
-              (__GNUC__ == 3) && (__GNUC_MINOR__ >= 1))
+#if defined(__GNUC__) \
+    && ((__GNUC__ >= 4) || (__GNUC__ == 3) && (__GNUC_MINOR__ >= 1))
 #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
 #else
 #define Py_DEPRECATED(VERSION_UNUSED)
 #endif
 
+
+/* Py_HOT_FUNCTION
+ * The hot attribute on a function is used to inform the compiler that the
+ * function is a hot spot of the compiled program. The function is optimized
+ * more aggressively and on many target it is placed into special subsection of
+ * the text section so all hot functions appears close together improving
+ * locality.
+ *
+ * Usage:
+ *    int Py_HOT_FUNCTION x() { return 3; }
+ *
+ * Issue #28618: This attribute must not be abused, otherwise it can have a
+ * negative effect on performance. Only the functions were Python spend most of
+ * its time must use it. Use a profiler when running performance benchmark
+ * suite to find these functions.
+ */
+#if defined(__GNUC__) \
+    && ((__GNUC__ >= 5) || (__GNUC__ == 4) && (__GNUC_MINOR__ >= 3))
+#define _Py_HOT_FUNCTION __attribute__((hot))
+#else
+#define _Py_HOT_FUNCTION
+#endif
+
 /**************************************************************************
 Prototypes that are missing from the standard include files on some systems
 (and possibly only some versions of such systems.)
diff --git a/Objects/frameobject.c b/Objects/frameobject.c
--- a/Objects/frameobject.c
+++ b/Objects/frameobject.c
@@ -409,7 +409,7 @@
 /* max value for numfree */
 #define PyFrame_MAXFREELIST 200
 
-static void
+static void _Py_HOT_FUNCTION
 frame_dealloc(PyFrameObject *f)
 {
     PyObject **p, **valuestack;
@@ -605,7 +605,7 @@
     return 1;
 }
 
-PyFrameObject *
+PyFrameObject* _Py_HOT_FUNCTION
 PyFrame_New(PyThreadState *tstate, PyCodeObject *code, PyObject *globals,
             PyObject *locals)
 {
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -718,7 +718,7 @@
     return tstate->interp->eval_frame(f, throwflag);
 }
 
-PyObject *
+PyObject* _Py_HOT_FUNCTION
 _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
 {
 #ifdef DXPAIRS
@@ -4771,7 +4771,7 @@
     x = call; \
     }
 
-static PyObject *
+static PyObject* _Py_HOT_FUNCTION
 call_function(PyObject ***pp_stack, Py_ssize_t oparg, PyObject *kwnames)
 {
     PyObject **pfunc = (*pp_stack) - oparg - 1;
@@ -4844,7 +4844,7 @@
    done before evaluating the frame.
 */
 
-static PyObject*
+static PyObject* _Py_HOT_FUNCTION
 _PyFunction_FastCall(PyCodeObject *co, PyObject **args, Py_ssize_t nargs,
                      PyObject *globals)
 {
diff --git a/Python/errors.c b/Python/errors.c
--- a/Python/errors.c
+++ b/Python/errors.c
@@ -158,7 +158,7 @@
 }
 
 
-PyObject *
+PyObject* _Py_HOT_FUNCTION
 PyErr_Occurred(void)
 {
     PyThreadState *tstate = PyThreadState_GET();

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Fri Nov 11 04:04:33 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Fri, 11 Nov 2016 09:04:33 +0000
Subject: [Python-checkins] Daily reference leaks (59b91b4e9506): sum=-2
Message-ID: <20161111090433.13983.2207.428C4E69@psf.io>

results for 59b91b4e9506 on branch "default"
--------------------------------------------

test_collections leaked [-7, 1, 0] memory blocks, sum=-6
test_functools leaked [0, 3, 1] memory blocks, sum=4


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog0qO1Dm', '--timeout', '7200']

From python-checkins at python.org  Fri Nov 11 04:49:17 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Fri, 11 Nov 2016 09:49:17 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIzODM5?=
 =?utf-8?q?=3A_Various_caches_now_are_cleared_before_running_every_test_fi?=
 =?utf-8?b?bGUu?=
Message-ID: <20161111094917.14653.26179.7D324A57@psf.io>

https://hg.python.org/cpython/rev/bc81f2137706
changeset:   105048:bc81f2137706
branch:      2.7
parent:      105045:137c7b92360e
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Fri Nov 11 11:42:25 2016 +0200
summary:
  Issue #23839: Various caches now are cleared before running every test file.

files:
  Lib/test/regrtest.py |  116 ++++++++++++++++++++++++------
  Misc/NEWS            |    2 +
  2 files changed, 94 insertions(+), 24 deletions(-)


diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
--- a/Lib/test/regrtest.py
+++ b/Lib/test/regrtest.py
@@ -931,6 +931,7 @@
             else:
                 # Always import it from the test package
                 abstest = 'test.' + test
+            clear_caches()
             with saved_test_environment(test, verbose, quiet, pgo) as environment:
                 start_time = time.time()
                 the_package = __import__(abstest, globals(), locals(), [])
@@ -1096,16 +1097,6 @@
 
 def dash_R_cleanup(fs, ps, pic, zdc, abcs):
     import gc, copy_reg
-    import _strptime, linecache
-    dircache = test_support.import_module('dircache', deprecated=True)
-    import urlparse, urllib, urllib2, mimetypes, doctest
-    import struct, filecmp
-    from distutils.dir_util import _path_created
-
-    # Clear the warnings registry, so they can be displayed again
-    for mod in sys.modules.values():
-        if hasattr(mod, '__warningregistry__'):
-            del mod.__warningregistry__
 
     # Restore some original values.
     warnings.filters[:] = fs
@@ -1130,23 +1121,100 @@
         abc._abc_cache.clear()
         abc._abc_negative_cache.clear()
 
+    clear_caches()
+
+def clear_caches():
+    import gc
+
+    # Clear the warnings registry, so they can be displayed again
+    for mod in sys.modules.values():
+        if hasattr(mod, '__warningregistry__'):
+            del mod.__warningregistry__
+
     # Clear assorted module caches.
-    _path_created.clear()
+    # Don't worry about resetting the cache if the module is not loaded
+    try:
+        distutils_dir_util = sys.modules['distutils.dir_util']
+    except KeyError:
+        pass
+    else:
+        distutils_dir_util._path_created.clear()
+
     re.purge()
-    _strptime._regex_cache.clear()
-    urlparse.clear_cache()
-    urllib.urlcleanup()
-    urllib2.install_opener(None)
-    dircache.reset()
-    linecache.clearcache()
-    mimetypes._default_mime_types()
-    filecmp._cache.clear()
-    struct._clearcache()
-    doctest.master = None
+
     try:
-        import ctypes
-    except ImportError:
-        # Don't worry about resetting the cache if ctypes is not supported
+        _strptime = sys.modules['_strptime']
+    except KeyError:
+        pass
+    else:
+        _strptime._regex_cache.clear()
+
+    try:
+        urlparse = sys.modules['urlparse']
+    except KeyError:
+        pass
+    else:
+        urlparse.clear_cache()
+
+    try:
+        urllib = sys.modules['urllib']
+    except KeyError:
+        pass
+    else:
+        urllib.urlcleanup()
+
+    try:
+        urllib2 = sys.modules['urllib2']
+    except KeyError:
+        pass
+    else:
+        urllib2.install_opener(None)
+
+    try:
+        dircache = sys.modules['dircache']
+    except KeyError:
+        pass
+    else:
+        dircache.reset()
+
+    try:
+        linecache = sys.modules['linecache']
+    except KeyError:
+        pass
+    else:
+        linecache.clearcache()
+
+    try:
+        mimetypes = sys.modules['mimetypes']
+    except KeyError:
+        pass
+    else:
+        mimetypes._default_mime_types()
+
+    try:
+        filecmp = sys.modules['filecmp']
+    except KeyError:
+        pass
+    else:
+        filecmp._cache.clear()
+
+    try:
+        struct = sys.modules['struct']
+    except KeyError:
+        pass
+    else:
+        struct._clearcache()
+
+    try:
+        doctest = sys.modules['doctest']
+    except KeyError:
+        pass
+    else:
+        doctest.master = None
+
+    try:
+        ctypes = sys.modules['ctypes']
+    except KeyError:
         pass
     else:
         ctypes._reset_cache()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -253,6 +253,8 @@
 Tests
 -----
 
+- Issue #23839: Various caches now are cleared before running every test file.
+
 - Issue #27369: In test_pyexpat, avoid testing an error message detail that
   changed in Expat 2.2.0.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 04:49:17 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Fri, 11 Nov 2016 09:49:17 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2323839=3A_Various_caches_now_are_cleared_before_runnin?=
 =?utf-8?q?g_every_test_file=2E?=
Message-ID: <20161111094917.12693.23716.602D0E37@psf.io>

https://hg.python.org/cpython/rev/c89f213b21e8
changeset:   105050:c89f213b21e8
branch:      3.6
parent:      105043:4c81a107ccab
parent:      105049:89776a40e0ec
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Fri Nov 11 11:46:44 2016 +0200
summary:
  Issue #23839: Various caches now are cleared before running every test file.

files:
  Lib/test/libregrtest/refleak.py |  109 +++++++++++++++----
  Lib/test/libregrtest/runtest.py |    3 +-
  Misc/NEWS                       |    5 +
  3 files changed, 91 insertions(+), 26 deletions(-)


diff --git a/Lib/test/libregrtest/refleak.py b/Lib/test/libregrtest/refleak.py
--- a/Lib/test/libregrtest/refleak.py
+++ b/Lib/test/libregrtest/refleak.py
@@ -122,17 +122,9 @@
 
 def dash_R_cleanup(fs, ps, pic, zdc, abcs):
     import gc, copyreg
-    import _strptime, linecache
-    import urllib.parse, urllib.request, mimetypes, doctest
-    import struct, filecmp, collections.abc
-    from distutils.dir_util import _path_created
+    import collections.abc
     from weakref import WeakSet
 
-    # Clear the warnings registry, so they can be displayed again
-    for mod in sys.modules.values():
-        if hasattr(mod, '__warningregistry__'):
-            del mod.__warningregistry__
-
     # Restore some original values.
     warnings.filters[:] = fs
     copyreg.dispatch_table.clear()
@@ -159,6 +151,23 @@
             obj._abc_cache.clear()
             obj._abc_negative_cache.clear()
 
+    clear_caches()
+
+    # Collect cyclic trash and read memory statistics immediately after.
+    func1 = sys.getallocatedblocks
+    func2 = sys.gettotalrefcount
+    gc.collect()
+    return func1(), func2(), fd_count()
+
+
+def clear_caches():
+    import gc
+
+    # Clear the warnings registry, so they can be displayed again
+    for mod in sys.modules.values():
+        if hasattr(mod, '__warningregistry__'):
+            del mod.__warningregistry__
+
     # Flush standard output, so that buffered data is sent to the OS and
     # associated Python objects are reclaimed.
     for stream in (sys.stdout, sys.stderr, sys.__stdout__, sys.__stderr__):
@@ -166,20 +175,74 @@
             stream.flush()
 
     # Clear assorted module caches.
-    _path_created.clear()
+    # Don't worry about resetting the cache if the module is not loaded
+    try:
+        distutils_dir_util = sys.modules['distutils.dir_util']
+    except KeyError:
+        pass
+    else:
+        distutils_dir_util._path_created.clear()
     re.purge()
-    _strptime._regex_cache.clear()
-    urllib.parse.clear_cache()
-    urllib.request.urlcleanup()
-    linecache.clearcache()
-    mimetypes._default_mime_types()
-    filecmp._cache.clear()
-    struct._clearcache()
-    doctest.master = None
+
     try:
-        import ctypes
-    except ImportError:
-        # Don't worry about resetting the cache if ctypes is not supported
+        _strptime = sys.modules['_strptime']
+    except KeyError:
+        pass
+    else:
+        _strptime._regex_cache.clear()
+
+    try:
+        urllib_parse = sys.modules['urllib.parse']
+    except KeyError:
+        pass
+    else:
+        urllib_parse.clear_cache()
+
+    try:
+        urllib_request = sys.modules['urllib.request']
+    except KeyError:
+        pass
+    else:
+        urllib_request.urlcleanup()
+
+    try:
+        linecache = sys.modules['linecache']
+    except KeyError:
+        pass
+    else:
+        linecache.clearcache()
+
+    try:
+        mimetypes = sys.modules['mimetypes']
+    except KeyError:
+        pass
+    else:
+        mimetypes._default_mime_types()
+
+    try:
+        filecmp = sys.modules['filecmp']
+    except KeyError:
+        pass
+    else:
+        filecmp._cache.clear()
+
+    try:
+        struct = sys.modules['struct']
+    except KeyError:
+        pass
+    else:
+        struct._clearcache()
+
+    try:
+        doctest = sys.modules['doctest']
+    except KeyError:
+        pass
+    else:
+        doctest.master = None
+
+    try:
+        ctypes = sys.modules['ctypes']
+    except KeyError:
         pass
     else:
         ctypes._reset_cache()
@@ -192,11 +255,7 @@
         for f in typing._cleanups:
             f()
 
-    # Collect cyclic trash and read memory statistics immediately after.
-    func1 = sys.getallocatedblocks
-    func2 = sys.gettotalrefcount
     gc.collect()
-    return func1(), func2(), fd_count()
 
 
 def warm_caches():
diff --git a/Lib/test/libregrtest/runtest.py b/Lib/test/libregrtest/runtest.py
--- a/Lib/test/libregrtest/runtest.py
+++ b/Lib/test/libregrtest/runtest.py
@@ -7,7 +7,7 @@
 import traceback
 import unittest
 from test import support
-from test.libregrtest.refleak import dash_R
+from test.libregrtest.refleak import dash_R, clear_caches
 from test.libregrtest.save_env import saved_test_environment
 
 
@@ -146,6 +146,7 @@
         else:
             # Always import it from the test package
             abstest = 'test.' + test
+        clear_caches()
         with saved_test_environment(test, ns.verbose, ns.quiet, pgo=ns.pgo) as environment:
             start_time = time.time()
             the_module = importlib.import_module(abstest)
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -55,6 +55,11 @@
 
 - Issue #28513: Documented command-line interface of zipfile.
 
+Tests
+-----
+
+- Issue #23839: Various caches now are cleared before running every test file.
+
 
 What's New in Python 3.6.0 beta 3
 =================================

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 04:49:17 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Fri, 11 Nov 2016 09:49:17 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIzODM5?=
 =?utf-8?q?=3A_Various_caches_now_are_cleared_before_running_every_test_fi?=
 =?utf-8?b?bGUu?=
Message-ID: <20161111094917.14135.874.ABC33DE5@psf.io>

https://hg.python.org/cpython/rev/89776a40e0ec
changeset:   105049:89776a40e0ec
branch:      3.5
parent:      105036:555f0ca31587
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Fri Nov 11 11:42:47 2016 +0200
summary:
  Issue #23839: Various caches now are cleared before running every test file.

files:
  Lib/test/regrtest.py |  110 +++++++++++++++++++++++-------
  Misc/NEWS            |    2 +
  2 files changed, 87 insertions(+), 25 deletions(-)


diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
--- a/Lib/test/regrtest.py
+++ b/Lib/test/regrtest.py
@@ -1292,6 +1292,7 @@
         else:
             # Always import it from the test package
             abstest = 'test.' + test
+        clear_caches()
         with saved_test_environment(test, verbose, quiet, pgo=pgo) as environment:
             start_time = time.time()
             the_module = importlib.import_module(abstest)
@@ -1464,17 +1465,9 @@
 
 def dash_R_cleanup(fs, ps, pic, zdc, abcs):
     import gc, copyreg
-    import _strptime, linecache
-    import urllib.parse, urllib.request, mimetypes, doctest
-    import struct, filecmp, collections.abc
-    from distutils.dir_util import _path_created
+    import collections.abc
     from weakref import WeakSet
 
-    # Clear the warnings registry, so they can be displayed again
-    for mod in sys.modules.values():
-        if hasattr(mod, '__warningregistry__'):
-            del mod.__warningregistry__
-
     # Restore some original values.
     warnings.filters[:] = fs
     copyreg.dispatch_table.clear()
@@ -1501,6 +1494,22 @@
             obj._abc_cache.clear()
             obj._abc_negative_cache.clear()
 
+    clear_caches()
+
+    # Collect cyclic trash and read memory statistics immediately after.
+    func1 = sys.getallocatedblocks
+    func2 = sys.gettotalrefcount
+    gc.collect()
+    return func1(), func2()
+
+def clear_caches():
+    import gc
+
+    # Clear the warnings registry, so they can be displayed again
+    for mod in sys.modules.values():
+        if hasattr(mod, '__warningregistry__'):
+            del mod.__warningregistry__
+
     # Flush standard output, so that buffered data is sent to the OS and
     # associated Python objects are reclaimed.
     for stream in (sys.stdout, sys.stderr, sys.__stdout__, sys.__stderr__):
@@ -1508,20 +1517,75 @@
             stream.flush()
 
     # Clear assorted module caches.
-    _path_created.clear()
+    # Don't worry about resetting the cache if the module is not loaded
+    try:
+        distutils_dir_util = sys.modules['distutils.dir_util']
+    except KeyError:
+        pass
+    else:
+        distutils_dir_util._path_created.clear()
+
     re.purge()
-    _strptime._regex_cache.clear()
-    urllib.parse.clear_cache()
-    urllib.request.urlcleanup()
-    linecache.clearcache()
-    mimetypes._default_mime_types()
-    filecmp._cache.clear()
-    struct._clearcache()
-    doctest.master = None
+
     try:
-        import ctypes
-    except ImportError:
-        # Don't worry about resetting the cache if ctypes is not supported
+        _strptime = sys.modules['_strptime']
+    except KeyError:
+        pass
+    else:
+        _strptime._regex_cache.clear()
+
+    try:
+        urllib_parse = sys.modules['urllib.parse']
+    except KeyError:
+        pass
+    else:
+        urllib_parse.clear_cache()
+
+    try:
+        urllib_request = sys.modules['urllib.request']
+    except KeyError:
+        pass
+    else:
+        urllib_request.urlcleanup()
+
+    try:
+        linecache = sys.modules['linecache']
+    except KeyError:
+        pass
+    else:
+        linecache.clearcache()
+
+    try:
+        mimetypes = sys.modules['mimetypes']
+    except KeyError:
+        pass
+    else:
+        mimetypes._default_mime_types()
+
+    try:
+        filecmp = sys.modules['filecmp']
+    except KeyError:
+        pass
+    else:
+        filecmp._cache.clear()
+
+    try:
+        struct = sys.modules['struct']
+    except KeyError:
+        pass
+    else:
+        struct._clearcache()
+
+    try:
+        doctest = sys.modules['doctest']
+    except KeyError:
+        pass
+    else:
+        doctest.master = None
+
+    try:
+        ctypes = sys.modules['ctypes']
+    except KeyError:
         pass
     else:
         ctypes._reset_cache()
@@ -1534,11 +1598,7 @@
         for f in typing._cleanups:
             f()
 
-    # Collect cyclic trash and read memory statistics immediately after.
-    func1 = sys.getallocatedblocks
-    func2 = sys.gettotalrefcount
     gc.collect()
-    return func1(), func2()
 
 def warm_caches():
     # char cache
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -501,6 +501,8 @@
 Tests
 -----
 
+- Issue #23839: Various caches now are cleared before running every test file.
+
 - Issue #28409: regrtest: fix the parser of command line arguments.
 
 - Issue #27787: Call gc.collect() before checking each test for "dangling

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 04:49:17 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Fri, 11 Nov 2016 09:49:17 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2323839=3A_Various_caches_now_are_cleared_before_?=
 =?utf-8?q?running_every_test_file=2E?=
Message-ID: <20161111094917.13306.75752.F4BD1C8C@psf.io>

https://hg.python.org/cpython/rev/5d1067e89717
changeset:   105051:5d1067e89717
parent:      105047:59b91b4e9506
parent:      105050:c89f213b21e8
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Fri Nov 11 11:48:22 2016 +0200
summary:
  Issue #23839: Various caches now are cleared before running every test file.

files:
  Lib/test/libregrtest/refleak.py |  109 +++++++++++++++----
  Lib/test/libregrtest/runtest.py |    3 +-
  Misc/NEWS                       |    2 +
  3 files changed, 88 insertions(+), 26 deletions(-)


diff --git a/Lib/test/libregrtest/refleak.py b/Lib/test/libregrtest/refleak.py
--- a/Lib/test/libregrtest/refleak.py
+++ b/Lib/test/libregrtest/refleak.py
@@ -122,17 +122,9 @@
 
 def dash_R_cleanup(fs, ps, pic, zdc, abcs):
     import gc, copyreg
-    import _strptime, linecache
-    import urllib.parse, urllib.request, mimetypes, doctest
-    import struct, filecmp, collections.abc
-    from distutils.dir_util import _path_created
+    import collections.abc
     from weakref import WeakSet
 
-    # Clear the warnings registry, so they can be displayed again
-    for mod in sys.modules.values():
-        if hasattr(mod, '__warningregistry__'):
-            del mod.__warningregistry__
-
     # Restore some original values.
     warnings.filters[:] = fs
     copyreg.dispatch_table.clear()
@@ -159,6 +151,23 @@
             obj._abc_cache.clear()
             obj._abc_negative_cache.clear()
 
+    clear_caches()
+
+    # Collect cyclic trash and read memory statistics immediately after.
+    func1 = sys.getallocatedblocks
+    func2 = sys.gettotalrefcount
+    gc.collect()
+    return func1(), func2(), fd_count()
+
+
+def clear_caches():
+    import gc
+
+    # Clear the warnings registry, so they can be displayed again
+    for mod in sys.modules.values():
+        if hasattr(mod, '__warningregistry__'):
+            del mod.__warningregistry__
+
     # Flush standard output, so that buffered data is sent to the OS and
     # associated Python objects are reclaimed.
     for stream in (sys.stdout, sys.stderr, sys.__stdout__, sys.__stderr__):
@@ -166,20 +175,74 @@
             stream.flush()
 
     # Clear assorted module caches.
-    _path_created.clear()
+    # Don't worry about resetting the cache if the module is not loaded
+    try:
+        distutils_dir_util = sys.modules['distutils.dir_util']
+    except KeyError:
+        pass
+    else:
+        distutils_dir_util._path_created.clear()
     re.purge()
-    _strptime._regex_cache.clear()
-    urllib.parse.clear_cache()
-    urllib.request.urlcleanup()
-    linecache.clearcache()
-    mimetypes._default_mime_types()
-    filecmp._cache.clear()
-    struct._clearcache()
-    doctest.master = None
+
     try:
-        import ctypes
-    except ImportError:
-        # Don't worry about resetting the cache if ctypes is not supported
+        _strptime = sys.modules['_strptime']
+    except KeyError:
+        pass
+    else:
+        _strptime._regex_cache.clear()
+
+    try:
+        urllib_parse = sys.modules['urllib.parse']
+    except KeyError:
+        pass
+    else:
+        urllib_parse.clear_cache()
+
+    try:
+        urllib_request = sys.modules['urllib.request']
+    except KeyError:
+        pass
+    else:
+        urllib_request.urlcleanup()
+
+    try:
+        linecache = sys.modules['linecache']
+    except KeyError:
+        pass
+    else:
+        linecache.clearcache()
+
+    try:
+        mimetypes = sys.modules['mimetypes']
+    except KeyError:
+        pass
+    else:
+        mimetypes._default_mime_types()
+
+    try:
+        filecmp = sys.modules['filecmp']
+    except KeyError:
+        pass
+    else:
+        filecmp._cache.clear()
+
+    try:
+        struct = sys.modules['struct']
+    except KeyError:
+        pass
+    else:
+        struct._clearcache()
+
+    try:
+        doctest = sys.modules['doctest']
+    except KeyError:
+        pass
+    else:
+        doctest.master = None
+
+    try:
+        ctypes = sys.modules['ctypes']
+    except KeyError:
         pass
     else:
         ctypes._reset_cache()
@@ -192,11 +255,7 @@
         for f in typing._cleanups:
             f()
 
-    # Collect cyclic trash and read memory statistics immediately after.
-    func1 = sys.getallocatedblocks
-    func2 = sys.gettotalrefcount
     gc.collect()
-    return func1(), func2(), fd_count()
 
 
 def warm_caches():
diff --git a/Lib/test/libregrtest/runtest.py b/Lib/test/libregrtest/runtest.py
--- a/Lib/test/libregrtest/runtest.py
+++ b/Lib/test/libregrtest/runtest.py
@@ -7,7 +7,7 @@
 import traceback
 import unittest
 from test import support
-from test.libregrtest.refleak import dash_R
+from test.libregrtest.refleak import dash_R, clear_caches
 from test.libregrtest.save_env import saved_test_environment
 
 
@@ -146,6 +146,7 @@
         else:
             # Always import it from the test package
             abstest = 'test.' + test
+        clear_caches()
         with saved_test_environment(test, ns.verbose, ns.quiet, pgo=ns.pgo) as environment:
             start_time = time.time()
             the_module = importlib.import_module(abstest)
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -443,6 +443,8 @@
 Tests
 -----
 
+- Issue #23839: Various caches now are cleared before running every test file.
+
 - Issue #26944: Fix test_posix for Android where 'id -G' is entirely wrong or
   missing the effective gid.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 05:12:25 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Fri, 11 Nov 2016 10:12:25 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2319398=3A_Extra_slash_no_longer_added_to_sys=2Epath_co?=
 =?utf-8?q?mponents_in_case_of?=
Message-ID: <20161111101225.101224.96282.2E3B4DEB@psf.io>

https://hg.python.org/cpython/rev/1a88baaed7a0
changeset:   105053:1a88baaed7a0
branch:      3.6
parent:      105050:c89f213b21e8
parent:      105052:db220f2df5a9
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Fri Nov 11 12:06:38 2016 +0200
summary:
  Issue #19398: Extra slash no longer added to sys.path components in case of
empty compile-time PYTHONPATH components.  This fixes some tests in -S or -I
modes.

files:
  Lib/test/test_trace.py |  8 ++++----
  Misc/NEWS              |  3 +++
  Modules/getpath.c      |  5 ++++-
  3 files changed, 11 insertions(+), 5 deletions(-)


diff --git a/Lib/test/test_trace.py b/Lib/test/test_trace.py
--- a/Lib/test/test_trace.py
+++ b/Lib/test/test_trace.py
@@ -311,11 +311,11 @@
         with captured_stdout() as stdout:
             self._coverage(tracer)
         stdout = stdout.getvalue()
-        self.assertTrue("pprint.py" in stdout)
-        self.assertTrue("case.py" in stdout)   # from unittest
+        self.assertIn("pprint.py", stdout)
+        self.assertIn("case.py", stdout)   # from unittest
         files = os.listdir(TESTFN)
-        self.assertTrue("pprint.cover" in files)
-        self.assertTrue("unittest.case.cover" in files)
+        self.assertIn("pprint.cover", files)
+        self.assertIn("unittest.case.cover", files)
 
     def test_coverage_ignore(self):
         # Ignore all files, nothing should be traced nor printed
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #19398: Extra slash no longer added to sys.path components in case of
+  empty compile-time PYTHONPATH components.
+
 - Issue #28583: PyDict_SetDefault didn't combine split table when needed.
   Patch by Xiang Zhang.
 
diff --git a/Modules/getpath.c b/Modules/getpath.c
--- a/Modules/getpath.c
+++ b/Modules/getpath.c
@@ -762,7 +762,10 @@
 
         if (defpath[0] != SEP) {
             wcscat(buf, prefix);
-            wcscat(buf, separator);
+            if (prefixsz >= 2 && prefix[prefixsz - 2] != SEP &&
+                defpath[0] != (delim ? DELIM : L'\0')) {  /* not empty */
+                wcscat(buf, separator);
+            }
         }
 
         if (delim) {

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 05:12:25 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Fri, 11 Nov 2016 10:12:25 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzE5Mzk4?=
 =?utf-8?q?=3A_Extra_slash_no_longer_added_to_sys=2Epath_components_in_cas?=
 =?utf-8?q?e_of?=
Message-ID: <20161111101225.12831.98742.5805D600@psf.io>

https://hg.python.org/cpython/rev/db220f2df5a9
changeset:   105052:db220f2df5a9
branch:      3.5
parent:      105049:89776a40e0ec
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Fri Nov 11 12:05:01 2016 +0200
summary:
  Issue #19398: Extra slash no longer added to sys.path components in case of
empty compile-time PYTHONPATH components.  This fixes some tests in -S or -I
modes.

files:
  Lib/test/test_trace.py |  8 ++++----
  Misc/NEWS              |  3 +++
  Modules/getpath.c      |  5 ++++-
  3 files changed, 11 insertions(+), 5 deletions(-)


diff --git a/Lib/test/test_trace.py b/Lib/test/test_trace.py
--- a/Lib/test/test_trace.py
+++ b/Lib/test/test_trace.py
@@ -311,11 +311,11 @@
         with captured_stdout() as stdout:
             self._coverage(tracer)
         stdout = stdout.getvalue()
-        self.assertTrue("pprint.py" in stdout)
-        self.assertTrue("case.py" in stdout)   # from unittest
+        self.assertIn("pprint.py", stdout)
+        self.assertIn("case.py", stdout)   # from unittest
         files = os.listdir(TESTFN)
-        self.assertTrue("pprint.cover" in files)
-        self.assertTrue("unittest.case.cover" in files)
+        self.assertIn("pprint.cover", files)
+        self.assertIn("unittest.case.cover", files)
 
     def test_coverage_ignore(self):
         # Ignore all files, nothing should be traced nor printed
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #19398: Extra slash no longer added to sys.path components in case of
+  empty compile-time PYTHONPATH components.
+
 - Issue #28426: Fixed potential crash in PyUnicode_AsDecodedObject() in debug
   build.
 
diff --git a/Modules/getpath.c b/Modules/getpath.c
--- a/Modules/getpath.c
+++ b/Modules/getpath.c
@@ -762,7 +762,10 @@
 
         if (defpath[0] != SEP) {
             wcscat(buf, prefix);
-            wcscat(buf, separator);
+            if (prefixsz >= 2 && prefix[prefixsz - 2] != SEP &&
+                defpath[0] != (delim ? DELIM : L'\0')) {  /* not empty */
+                wcscat(buf, separator);
+            }
         }
 
         if (delim) {

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 05:12:26 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Fri, 11 Nov 2016 10:12:26 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2319398=3A_Extra_slash_no_longer_added_to_sys=2Ep?=
 =?utf-8?q?ath_components_in_case_of?=
Message-ID: <20161111101225.13690.35668.4E533BC6@psf.io>

https://hg.python.org/cpython/rev/82607e7c24c7
changeset:   105054:82607e7c24c7
parent:      105051:5d1067e89717
parent:      105053:1a88baaed7a0
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Fri Nov 11 12:07:48 2016 +0200
summary:
  Issue #19398: Extra slash no longer added to sys.path components in case of
empty compile-time PYTHONPATH components.  This fixes some tests in -S or -I
modes.

files:
  Lib/test/test_trace.py |  8 ++++----
  Misc/NEWS              |  3 +++
  Modules/getpath.c      |  5 ++++-
  3 files changed, 11 insertions(+), 5 deletions(-)


diff --git a/Lib/test/test_trace.py b/Lib/test/test_trace.py
--- a/Lib/test/test_trace.py
+++ b/Lib/test/test_trace.py
@@ -311,11 +311,11 @@
         with captured_stdout() as stdout:
             self._coverage(tracer)
         stdout = stdout.getvalue()
-        self.assertTrue("pprint.py" in stdout)
-        self.assertTrue("case.py" in stdout)   # from unittest
+        self.assertIn("pprint.py", stdout)
+        self.assertIn("case.py", stdout)   # from unittest
         files = os.listdir(TESTFN)
-        self.assertTrue("pprint.cover" in files)
-        self.assertTrue("unittest.case.cover" in files)
+        self.assertIn("pprint.cover", files)
+        self.assertIn("unittest.case.cover", files)
 
     def test_coverage_ignore(self):
         # Ignore all files, nothing should be traced nor printed
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #19398: Extra slash no longer added to sys.path components in case of
+  empty compile-time PYTHONPATH components.
+
 - Issue #28621: Sped up converting int to float by reusing faster bits counting
   implementation.  Patch by Adrian Wielgosik.
 
diff --git a/Modules/getpath.c b/Modules/getpath.c
--- a/Modules/getpath.c
+++ b/Modules/getpath.c
@@ -762,7 +762,10 @@
 
         if (defpath[0] != SEP) {
             wcscat(buf, prefix);
-            wcscat(buf, separator);
+            if (prefixsz >= 2 && prefix[prefixsz - 2] != SEP &&
+                defpath[0] != (delim ? DELIM : L'\0')) {  /* not empty */
+                wcscat(buf, separator);
+            }
         }
 
         if (delim) {

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 05:12:27 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Fri, 11 Nov 2016 10:12:27 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE5Mzk4?=
 =?utf-8?q?=3A_Extra_slash_no_longer_added_to_sys=2Epath_components_in_cas?=
 =?utf-8?q?e_of?=
Message-ID: <20161111101227.12831.90815.3FA470FC@psf.io>

https://hg.python.org/cpython/rev/237ef36fb1bb
changeset:   105055:237ef36fb1bb
branch:      2.7
parent:      105048:bc81f2137706
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Fri Nov 11 12:11:55 2016 +0200
summary:
  Issue #19398: Extra slash no longer added to sys.path components in case of
empty compile-time PYTHONPATH components.  This fixes some tests in -S or -I
modes.

files:
  Lib/test/test_trace.py |  8 ++++----
  Misc/NEWS              |  3 +++
  Modules/getpath.c      |  5 ++++-
  3 files changed, 11 insertions(+), 5 deletions(-)


diff --git a/Lib/test/test_trace.py b/Lib/test/test_trace.py
--- a/Lib/test/test_trace.py
+++ b/Lib/test/test_trace.py
@@ -293,11 +293,11 @@
         with captured_stdout() as stdout:
             self._coverage(tracer)
         stdout = stdout.getvalue()
-        self.assertTrue("pprint.py" in stdout)
-        self.assertTrue("case.py" in stdout)   # from unittest
+        self.assertIn("pprint.py", stdout)
+        self.assertIn("case.py", stdout)   # from unittest
         files = os.listdir(TESTFN)
-        self.assertTrue("pprint.cover" in files)
-        self.assertTrue("unittest.case.cover" in files)
+        self.assertIn("pprint.cover", files)
+        self.assertIn("unittest.case.cover", files)
 
     def test_coverage_ignore(self):
         # Ignore all files, nothing should be traced nor printed
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #19398: Extra slash no longer added to sys.path components in case of
+  empty compile-time PYTHONPATH components.
+
 - Issue #21720: Improve exception message when the type of fromlist is unicode.
   fromlist parameter of __import__() only accepts str in Python 2 and this
   will help to identify the problem especially when the unicode_literals
diff --git a/Modules/getpath.c b/Modules/getpath.c
--- a/Modules/getpath.c
+++ b/Modules/getpath.c
@@ -597,7 +597,10 @@
 
             if (defpath[0] != SEP) {
                 strcat(buf, prefix);
-                strcat(buf, separator);
+                if (prefixsz >= 2 && prefix[prefixsz - 2] != SEP &&
+                    defpath[0] != (delim ? DELIM : L'\0')) {  /* not empty */
+                    strcat(buf, separator);
+                }
             }
 
             if (delim) {

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 07:32:21 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Fri, 11 Nov 2016 12:32:21 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_merge?=
Message-ID: <20161111123221.101177.93921.B5BD1544@psf.io>

https://hg.python.org/cpython/rev/955164a24535
changeset:   105057:955164a24535
parent:      105054:82607e7c24c7
user:        Raymond Hettinger <python at rcn.com>
date:        Fri Nov 11 04:32:11 2016 -0800
summary:
  merge

files:
  Python/ceval.c |  5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -2468,8 +2468,9 @@
         TARGET(STORE_DEREF) {
             PyObject *v = POP();
             PyObject *cell = freevars[oparg];
-            PyCell_Set(cell, v);
-            Py_DECREF(v);
+            PyObject *oldobj = PyCell_GET(cell);
+            PyCell_SET(cell, v);
+            Py_XDECREF(oldobj);
             DISPATCH();
         }
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 07:32:21 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Fri, 11 Nov 2016 12:32:21 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NjY1?=
 =?utf-8?q?=3A_Harmonize_STORE=5FDEREF_with_STORE=5FFAST_and_LOAD=5FDEREF_?=
 =?utf-8?q?giving_a_40=25?=
Message-ID: <20161111123221.8458.92633.F0A21B4F@psf.io>

https://hg.python.org/cpython/rev/d78d45436753
changeset:   105056:d78d45436753
branch:      3.6
parent:      105053:1a88baaed7a0
user:        Raymond Hettinger <python at rcn.com>
date:        Fri Nov 11 04:31:18 2016 -0800
summary:
  Issue #28665: Harmonize STORE_DEREF with STORE_FAST and LOAD_DEREF giving a 40% speedup.

files:
  Misc/NEWS      |  2 ++
  Python/ceval.c |  5 +++--
  2 files changed, 5 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -13,6 +13,8 @@
 - Issue #19398: Extra slash no longer added to sys.path components in case of
   empty compile-time PYTHONPATH components.
 
+- Issue #28665:  Improve speed of the STORE_DEREF opcode by 40%.
+
 - Issue #28583: PyDict_SetDefault didn't combine split table when needed.
   Patch by Xiang Zhang.
 
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -2462,8 +2462,9 @@
         TARGET(STORE_DEREF) {
             PyObject *v = POP();
             PyObject *cell = freevars[oparg];
-            PyCell_Set(cell, v);
-            Py_DECREF(v);
+            PyObject *oldobj = PyCell_GET(cell);
+            PyCell_SET(cell, v);
+            Py_XDECREF(oldobj);
             DISPATCH();
         }
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 09:50:47 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Fri, 11 Nov 2016 14:50:47 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgZnJvbSAzLjYgKGlzc3VlICMyODY2NSku?=
Message-ID: <20161111145037.13690.47358.64C0EEE8@psf.io>

https://hg.python.org/cpython/rev/7ec45e7d2194
changeset:   105058:7ec45e7d2194
parent:      105057:955164a24535
parent:      105056:d78d45436753
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Fri Nov 11 16:50:13 2016 +0200
summary:
  Merge from 3.6 (issue #28665).

files:
  Misc/NEWS |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,8 @@
 Core and Builtins
 -----------------
 
+- Issue #28665: Improve speed of the STORE_DEREF opcode by 40%.
+
 - Issue #19398: Extra slash no longer added to sys.path components in case of
   empty compile-time PYTHONPATH components.
 

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Fri Nov 11 09:56:57 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Fri, 11 Nov 2016 14:56:57 +0000
Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-11-11
Message-ID: <d9ca80fe-ccbf-43ec-91f6-a64962dd6ac5@irsmsx106.ger.corp.intel.com>

Results for project Python 2.7, build date 2016-11-11 03:47:09 +0000
commit:		137c7b92360e
previous commit:	41613bb27f80
revision date:	2016-11-10 23:42:58 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.16%          0.48%          5.02%             6.45%
:-)             pybench      0.24%         -0.06%          5.88%             4.84%
:-(            regex_v8      0.78%         -0.44%         -2.49%            11.25%
:-)               nbody      0.24%         -3.24%          5.30%             6.00%
:-)        json_dump_v2      0.30%          0.12%          2.91%             8.23%
:-|      normal_startup      0.56%         -0.77%         -0.97%             2.86%
:-)             ssbench      0.23%         -0.06%          2.31%             2.06%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-11-11/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Fri Nov 11 10:12:21 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Fri, 11 Nov 2016 15:12:21 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328664=3A_test=5Fbz2_now_works_on_non-Windows_pl?=
 =?utf-8?q?atforms_without_bunzip2?=
Message-ID: <20161111151221.101144.37733.BDA57342@psf.io>

https://hg.python.org/cpython/rev/969e85a7a943
changeset:   105061:969e85a7a943
parent:      105058:7ec45e7d2194
parent:      105060:9184f7f11b30
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Fri Nov 11 17:11:48 2016 +0200
summary:
  Issue #28664: test_bz2 now works on non-Windows platforms without bunzip2
(e.g. on Android).

files:
  Lib/test/test_bz2.py |  59 ++++++++++++++-----------------
  1 files changed, 26 insertions(+), 33 deletions(-)


diff --git a/Lib/test/test_bz2.py b/Lib/test/test_bz2.py
--- a/Lib/test/test_bz2.py
+++ b/Lib/test/test_bz2.py
@@ -8,6 +8,7 @@
 import glob
 import pathlib
 import random
+import shutil
 import subprocess
 import sys
 from test.support import unlink
@@ -22,6 +23,16 @@
 bz2 = support.import_module('bz2')
 from bz2 import BZ2File, BZ2Compressor, BZ2Decompressor
 
+has_cmdline_bunzip2 = None
+
+def ext_decompress(data):
+    global has_cmdline_bunzip2
+    if has_cmdline_bunzip2 is None:
+        has_cmdline_bunzip2 = bool(shutil.which('bunzip2'))
+    if has_cmdline_bunzip2:
+        return subprocess.check_output(['bunzip2'], input=data)
+    else:
+        return bz2.decompress(data)
 
 class BaseTest(unittest.TestCase):
     "Base for other testcases."
@@ -74,24 +85,6 @@
         if os.path.isfile(self.filename):
             os.unlink(self.filename)
 
-    if sys.platform == "win32":
-        # bunzip2 isn't available to run on Windows.
-        def decompress(self, data):
-            return bz2.decompress(data)
-    else:
-        def decompress(self, data):
-            pop = subprocess.Popen("bunzip2", shell=True,
-                                   stdin=subprocess.PIPE,
-                                   stdout=subprocess.PIPE,
-                                   stderr=subprocess.STDOUT)
-            pop.stdin.write(data)
-            pop.stdin.close()
-            ret = pop.stdout.read()
-            pop.stdout.close()
-            if pop.wait() != 0:
-                ret = bz2.decompress(data)
-            return ret
-
 
 class BZ2FileTest(BaseTest):
     "Test the BZ2File class."
@@ -252,7 +245,7 @@
             self.assertRaises(TypeError, bz2f.write)
             bz2f.write(self.TEXT)
         with open(self.filename, 'rb') as f:
-            self.assertEqual(self.decompress(f.read()), self.TEXT)
+            self.assertEqual(ext_decompress(f.read()), self.TEXT)
 
     def testWriteChunks10(self):
         with BZ2File(self.filename, "w") as bz2f:
@@ -264,7 +257,7 @@
                 bz2f.write(str)
                 n += 1
         with open(self.filename, 'rb') as f:
-            self.assertEqual(self.decompress(f.read()), self.TEXT)
+            self.assertEqual(ext_decompress(f.read()), self.TEXT)
 
     def testWriteNonDefaultCompressLevel(self):
         expected = bz2.compress(self.TEXT, compresslevel=5)
@@ -281,7 +274,7 @@
         # should raise an exception.
         self.assertRaises(ValueError, bz2f.writelines, ["a"])
         with open(self.filename, 'rb') as f:
-            self.assertEqual(self.decompress(f.read()), self.TEXT)
+            self.assertEqual(ext_decompress(f.read()), self.TEXT)
 
     def testWriteMethodsOnReadOnlyFile(self):
         with BZ2File(self.filename, "w") as bz2f:
@@ -299,7 +292,7 @@
             self.assertRaises(TypeError, bz2f.write)
             bz2f.write(self.TEXT)
         with open(self.filename, 'rb') as f:
-            self.assertEqual(self.decompress(f.read()), self.TEXT * 2)
+            self.assertEqual(ext_decompress(f.read()), self.TEXT * 2)
 
     def testSeekForward(self):
         self.createTempFile()
@@ -602,7 +595,7 @@
             with BZ2File(bio, "w") as bz2f:
                 self.assertRaises(TypeError, bz2f.write)
                 bz2f.write(self.TEXT)
-            self.assertEqual(self.decompress(bio.getvalue()), self.TEXT)
+            self.assertEqual(ext_decompress(bio.getvalue()), self.TEXT)
             self.assertFalse(bio.closed)
 
     def testSeekForwardBytesIO(self):
@@ -639,7 +632,7 @@
         self.assertRaises(TypeError, bz2c.compress)
         data = bz2c.compress(self.TEXT)
         data += bz2c.flush()
-        self.assertEqual(self.decompress(data), self.TEXT)
+        self.assertEqual(ext_decompress(data), self.TEXT)
 
     def testCompressEmptyString(self):
         bz2c = BZ2Compressor()
@@ -658,7 +651,7 @@
             data += bz2c.compress(str)
             n += 1
         data += bz2c.flush()
-        self.assertEqual(self.decompress(data), self.TEXT)
+        self.assertEqual(ext_decompress(data), self.TEXT)
 
     @bigmemtest(size=_4G + 100, memuse=2)
     def testCompress4G(self, size):
@@ -838,7 +831,7 @@
 class CompressDecompressTest(BaseTest):
     def testCompress(self):
         data = bz2.compress(self.TEXT)
-        self.assertEqual(self.decompress(data), self.TEXT)
+        self.assertEqual(ext_decompress(data), self.TEXT)
 
     def testCompressEmptyString(self):
         text = bz2.compress(b'')
@@ -888,14 +881,14 @@
             with self.open(self.filename, mode) as f:
                 f.write(self.TEXT)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read())
+                file_data = ext_decompress(f.read())
                 self.assertEqual(file_data, self.TEXT)
             with self.open(self.filename, "rb") as f:
                 self.assertEqual(f.read(), self.TEXT)
             with self.open(self.filename, "ab") as f:
                 f.write(self.TEXT)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read())
+                file_data = ext_decompress(f.read())
                 self.assertEqual(file_data, self.TEXT * 2)
 
     def test_implicit_binary_modes(self):
@@ -906,14 +899,14 @@
             with self.open(self.filename, mode) as f:
                 f.write(self.TEXT)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read())
+                file_data = ext_decompress(f.read())
                 self.assertEqual(file_data, self.TEXT)
             with self.open(self.filename, "r") as f:
                 self.assertEqual(f.read(), self.TEXT)
             with self.open(self.filename, "a") as f:
                 f.write(self.TEXT)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read())
+                file_data = ext_decompress(f.read())
                 self.assertEqual(file_data, self.TEXT * 2)
 
     def test_text_modes(self):
@@ -925,14 +918,14 @@
             with self.open(self.filename, mode) as f:
                 f.write(text)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read()).decode("ascii")
+                file_data = ext_decompress(f.read()).decode("ascii")
                 self.assertEqual(file_data, text_native_eol)
             with self.open(self.filename, "rt") as f:
                 self.assertEqual(f.read(), text)
             with self.open(self.filename, "at") as f:
                 f.write(text)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read()).decode("ascii")
+                file_data = ext_decompress(f.read()).decode("ascii")
                 self.assertEqual(file_data, text_native_eol * 2)
 
     def test_x_mode(self):
@@ -973,7 +966,7 @@
         with self.open(self.filename, "wt", encoding="utf-16-le") as f:
             f.write(text)
         with open(self.filename, "rb") as f:
-            file_data = self.decompress(f.read()).decode("utf-16-le")
+            file_data = ext_decompress(f.read()).decode("utf-16-le")
             self.assertEqual(file_data, text_native_eol)
         with self.open(self.filename, "rt", encoding="utf-16-le") as f:
             self.assertEqual(f.read(), text)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 10:12:21 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Fri, 11 Nov 2016 15:12:21 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjY0?=
 =?utf-8?q?=3A_test=5Fbz2_now_works_on_non-Windows_platforms_without_bunzi?=
 =?utf-8?q?p2?=
Message-ID: <20161111151221.13461.77611.1EAE29E3@psf.io>

https://hg.python.org/cpython/rev/648cd8450f4f
changeset:   105059:648cd8450f4f
branch:      3.5
parent:      105052:db220f2df5a9
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Fri Nov 11 17:10:24 2016 +0200
summary:
  Issue #28664: test_bz2 now works on non-Windows platforms without bunzip2
(e.g. on Android).

files:
  Lib/test/test_bz2.py |  59 ++++++++++++++-----------------
  1 files changed, 26 insertions(+), 33 deletions(-)


diff --git a/Lib/test/test_bz2.py b/Lib/test/test_bz2.py
--- a/Lib/test/test_bz2.py
+++ b/Lib/test/test_bz2.py
@@ -7,6 +7,7 @@
 import pickle
 import glob
 import random
+import shutil
 import subprocess
 import sys
 from test.support import unlink
@@ -21,6 +22,16 @@
 bz2 = support.import_module('bz2')
 from bz2 import BZ2File, BZ2Compressor, BZ2Decompressor
 
+has_cmdline_bunzip2 = None
+
+def ext_decompress(data):
+    global has_cmdline_bunzip2
+    if has_cmdline_bunzip2 is None:
+        has_cmdline_bunzip2 = bool(shutil.which('bunzip2'))
+    if has_cmdline_bunzip2:
+        return subprocess.check_output(['bunzip2'], input=data)
+    else:
+        return bz2.decompress(data)
 
 class BaseTest(unittest.TestCase):
     "Base for other testcases."
@@ -73,24 +84,6 @@
         if os.path.isfile(self.filename):
             os.unlink(self.filename)
 
-    if sys.platform == "win32":
-        # bunzip2 isn't available to run on Windows.
-        def decompress(self, data):
-            return bz2.decompress(data)
-    else:
-        def decompress(self, data):
-            pop = subprocess.Popen("bunzip2", shell=True,
-                                   stdin=subprocess.PIPE,
-                                   stdout=subprocess.PIPE,
-                                   stderr=subprocess.STDOUT)
-            pop.stdin.write(data)
-            pop.stdin.close()
-            ret = pop.stdout.read()
-            pop.stdout.close()
-            if pop.wait() != 0:
-                ret = bz2.decompress(data)
-            return ret
-
 
 class BZ2FileTest(BaseTest):
     "Test the BZ2File class."
@@ -251,7 +244,7 @@
             self.assertRaises(TypeError, bz2f.write)
             bz2f.write(self.TEXT)
         with open(self.filename, 'rb') as f:
-            self.assertEqual(self.decompress(f.read()), self.TEXT)
+            self.assertEqual(ext_decompress(f.read()), self.TEXT)
 
     def testWriteChunks10(self):
         with BZ2File(self.filename, "w") as bz2f:
@@ -263,7 +256,7 @@
                 bz2f.write(str)
                 n += 1
         with open(self.filename, 'rb') as f:
-            self.assertEqual(self.decompress(f.read()), self.TEXT)
+            self.assertEqual(ext_decompress(f.read()), self.TEXT)
 
     def testWriteNonDefaultCompressLevel(self):
         expected = bz2.compress(self.TEXT, compresslevel=5)
@@ -280,7 +273,7 @@
         # should raise an exception.
         self.assertRaises(ValueError, bz2f.writelines, ["a"])
         with open(self.filename, 'rb') as f:
-            self.assertEqual(self.decompress(f.read()), self.TEXT)
+            self.assertEqual(ext_decompress(f.read()), self.TEXT)
 
     def testWriteMethodsOnReadOnlyFile(self):
         with BZ2File(self.filename, "w") as bz2f:
@@ -298,7 +291,7 @@
             self.assertRaises(TypeError, bz2f.write)
             bz2f.write(self.TEXT)
         with open(self.filename, 'rb') as f:
-            self.assertEqual(self.decompress(f.read()), self.TEXT * 2)
+            self.assertEqual(ext_decompress(f.read()), self.TEXT * 2)
 
     def testSeekForward(self):
         self.createTempFile()
@@ -594,7 +587,7 @@
             with BZ2File(bio, "w") as bz2f:
                 self.assertRaises(TypeError, bz2f.write)
                 bz2f.write(self.TEXT)
-            self.assertEqual(self.decompress(bio.getvalue()), self.TEXT)
+            self.assertEqual(ext_decompress(bio.getvalue()), self.TEXT)
             self.assertFalse(bio.closed)
 
     def testSeekForwardBytesIO(self):
@@ -631,7 +624,7 @@
         self.assertRaises(TypeError, bz2c.compress)
         data = bz2c.compress(self.TEXT)
         data += bz2c.flush()
-        self.assertEqual(self.decompress(data), self.TEXT)
+        self.assertEqual(ext_decompress(data), self.TEXT)
 
     def testCompressEmptyString(self):
         bz2c = BZ2Compressor()
@@ -650,7 +643,7 @@
             data += bz2c.compress(str)
             n += 1
         data += bz2c.flush()
-        self.assertEqual(self.decompress(data), self.TEXT)
+        self.assertEqual(ext_decompress(data), self.TEXT)
 
     @bigmemtest(size=_4G + 100, memuse=2)
     def testCompress4G(self, size):
@@ -830,7 +823,7 @@
 class CompressDecompressTest(BaseTest):
     def testCompress(self):
         data = bz2.compress(self.TEXT)
-        self.assertEqual(self.decompress(data), self.TEXT)
+        self.assertEqual(ext_decompress(data), self.TEXT)
 
     def testCompressEmptyString(self):
         text = bz2.compress(b'')
@@ -880,14 +873,14 @@
             with self.open(self.filename, mode) as f:
                 f.write(self.TEXT)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read())
+                file_data = ext_decompress(f.read())
                 self.assertEqual(file_data, self.TEXT)
             with self.open(self.filename, "rb") as f:
                 self.assertEqual(f.read(), self.TEXT)
             with self.open(self.filename, "ab") as f:
                 f.write(self.TEXT)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read())
+                file_data = ext_decompress(f.read())
                 self.assertEqual(file_data, self.TEXT * 2)
 
     def test_implicit_binary_modes(self):
@@ -898,14 +891,14 @@
             with self.open(self.filename, mode) as f:
                 f.write(self.TEXT)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read())
+                file_data = ext_decompress(f.read())
                 self.assertEqual(file_data, self.TEXT)
             with self.open(self.filename, "r") as f:
                 self.assertEqual(f.read(), self.TEXT)
             with self.open(self.filename, "a") as f:
                 f.write(self.TEXT)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read())
+                file_data = ext_decompress(f.read())
                 self.assertEqual(file_data, self.TEXT * 2)
 
     def test_text_modes(self):
@@ -917,14 +910,14 @@
             with self.open(self.filename, mode) as f:
                 f.write(text)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read()).decode("ascii")
+                file_data = ext_decompress(f.read()).decode("ascii")
                 self.assertEqual(file_data, text_native_eol)
             with self.open(self.filename, "rt") as f:
                 self.assertEqual(f.read(), text)
             with self.open(self.filename, "at") as f:
                 f.write(text)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read()).decode("ascii")
+                file_data = ext_decompress(f.read()).decode("ascii")
                 self.assertEqual(file_data, text_native_eol * 2)
 
     def test_x_mode(self):
@@ -965,7 +958,7 @@
         with self.open(self.filename, "wt", encoding="utf-16-le") as f:
             f.write(text)
         with open(self.filename, "rb") as f:
-            file_data = self.decompress(f.read()).decode("utf-16-le")
+            file_data = ext_decompress(f.read()).decode("utf-16-le")
             self.assertEqual(file_data, text_native_eol)
         with self.open(self.filename, "rt", encoding="utf-16-le") as f:
             self.assertEqual(f.read(), text)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 10:12:22 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Fri, 11 Nov 2016 15:12:22 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328664=3A_test=5Fbz2_now_works_on_non-Windows_platform?=
 =?utf-8?q?s_without_bunzip2?=
Message-ID: <20161111151221.6983.83094.EC387988@psf.io>

https://hg.python.org/cpython/rev/9184f7f11b30
changeset:   105060:9184f7f11b30
branch:      3.6
parent:      105056:d78d45436753
parent:      105059:648cd8450f4f
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Fri Nov 11 17:11:33 2016 +0200
summary:
  Issue #28664: test_bz2 now works on non-Windows platforms without bunzip2
(e.g. on Android).

files:
  Lib/test/test_bz2.py |  59 ++++++++++++++-----------------
  1 files changed, 26 insertions(+), 33 deletions(-)


diff --git a/Lib/test/test_bz2.py b/Lib/test/test_bz2.py
--- a/Lib/test/test_bz2.py
+++ b/Lib/test/test_bz2.py
@@ -8,6 +8,7 @@
 import glob
 import pathlib
 import random
+import shutil
 import subprocess
 import sys
 from test.support import unlink
@@ -22,6 +23,16 @@
 bz2 = support.import_module('bz2')
 from bz2 import BZ2File, BZ2Compressor, BZ2Decompressor
 
+has_cmdline_bunzip2 = None
+
+def ext_decompress(data):
+    global has_cmdline_bunzip2
+    if has_cmdline_bunzip2 is None:
+        has_cmdline_bunzip2 = bool(shutil.which('bunzip2'))
+    if has_cmdline_bunzip2:
+        return subprocess.check_output(['bunzip2'], input=data)
+    else:
+        return bz2.decompress(data)
 
 class BaseTest(unittest.TestCase):
     "Base for other testcases."
@@ -74,24 +85,6 @@
         if os.path.isfile(self.filename):
             os.unlink(self.filename)
 
-    if sys.platform == "win32":
-        # bunzip2 isn't available to run on Windows.
-        def decompress(self, data):
-            return bz2.decompress(data)
-    else:
-        def decompress(self, data):
-            pop = subprocess.Popen("bunzip2", shell=True,
-                                   stdin=subprocess.PIPE,
-                                   stdout=subprocess.PIPE,
-                                   stderr=subprocess.STDOUT)
-            pop.stdin.write(data)
-            pop.stdin.close()
-            ret = pop.stdout.read()
-            pop.stdout.close()
-            if pop.wait() != 0:
-                ret = bz2.decompress(data)
-            return ret
-
 
 class BZ2FileTest(BaseTest):
     "Test the BZ2File class."
@@ -252,7 +245,7 @@
             self.assertRaises(TypeError, bz2f.write)
             bz2f.write(self.TEXT)
         with open(self.filename, 'rb') as f:
-            self.assertEqual(self.decompress(f.read()), self.TEXT)
+            self.assertEqual(ext_decompress(f.read()), self.TEXT)
 
     def testWriteChunks10(self):
         with BZ2File(self.filename, "w") as bz2f:
@@ -264,7 +257,7 @@
                 bz2f.write(str)
                 n += 1
         with open(self.filename, 'rb') as f:
-            self.assertEqual(self.decompress(f.read()), self.TEXT)
+            self.assertEqual(ext_decompress(f.read()), self.TEXT)
 
     def testWriteNonDefaultCompressLevel(self):
         expected = bz2.compress(self.TEXT, compresslevel=5)
@@ -281,7 +274,7 @@
         # should raise an exception.
         self.assertRaises(ValueError, bz2f.writelines, ["a"])
         with open(self.filename, 'rb') as f:
-            self.assertEqual(self.decompress(f.read()), self.TEXT)
+            self.assertEqual(ext_decompress(f.read()), self.TEXT)
 
     def testWriteMethodsOnReadOnlyFile(self):
         with BZ2File(self.filename, "w") as bz2f:
@@ -299,7 +292,7 @@
             self.assertRaises(TypeError, bz2f.write)
             bz2f.write(self.TEXT)
         with open(self.filename, 'rb') as f:
-            self.assertEqual(self.decompress(f.read()), self.TEXT * 2)
+            self.assertEqual(ext_decompress(f.read()), self.TEXT * 2)
 
     def testSeekForward(self):
         self.createTempFile()
@@ -602,7 +595,7 @@
             with BZ2File(bio, "w") as bz2f:
                 self.assertRaises(TypeError, bz2f.write)
                 bz2f.write(self.TEXT)
-            self.assertEqual(self.decompress(bio.getvalue()), self.TEXT)
+            self.assertEqual(ext_decompress(bio.getvalue()), self.TEXT)
             self.assertFalse(bio.closed)
 
     def testSeekForwardBytesIO(self):
@@ -639,7 +632,7 @@
         self.assertRaises(TypeError, bz2c.compress)
         data = bz2c.compress(self.TEXT)
         data += bz2c.flush()
-        self.assertEqual(self.decompress(data), self.TEXT)
+        self.assertEqual(ext_decompress(data), self.TEXT)
 
     def testCompressEmptyString(self):
         bz2c = BZ2Compressor()
@@ -658,7 +651,7 @@
             data += bz2c.compress(str)
             n += 1
         data += bz2c.flush()
-        self.assertEqual(self.decompress(data), self.TEXT)
+        self.assertEqual(ext_decompress(data), self.TEXT)
 
     @bigmemtest(size=_4G + 100, memuse=2)
     def testCompress4G(self, size):
@@ -838,7 +831,7 @@
 class CompressDecompressTest(BaseTest):
     def testCompress(self):
         data = bz2.compress(self.TEXT)
-        self.assertEqual(self.decompress(data), self.TEXT)
+        self.assertEqual(ext_decompress(data), self.TEXT)
 
     def testCompressEmptyString(self):
         text = bz2.compress(b'')
@@ -888,14 +881,14 @@
             with self.open(self.filename, mode) as f:
                 f.write(self.TEXT)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read())
+                file_data = ext_decompress(f.read())
                 self.assertEqual(file_data, self.TEXT)
             with self.open(self.filename, "rb") as f:
                 self.assertEqual(f.read(), self.TEXT)
             with self.open(self.filename, "ab") as f:
                 f.write(self.TEXT)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read())
+                file_data = ext_decompress(f.read())
                 self.assertEqual(file_data, self.TEXT * 2)
 
     def test_implicit_binary_modes(self):
@@ -906,14 +899,14 @@
             with self.open(self.filename, mode) as f:
                 f.write(self.TEXT)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read())
+                file_data = ext_decompress(f.read())
                 self.assertEqual(file_data, self.TEXT)
             with self.open(self.filename, "r") as f:
                 self.assertEqual(f.read(), self.TEXT)
             with self.open(self.filename, "a") as f:
                 f.write(self.TEXT)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read())
+                file_data = ext_decompress(f.read())
                 self.assertEqual(file_data, self.TEXT * 2)
 
     def test_text_modes(self):
@@ -925,14 +918,14 @@
             with self.open(self.filename, mode) as f:
                 f.write(text)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read()).decode("ascii")
+                file_data = ext_decompress(f.read()).decode("ascii")
                 self.assertEqual(file_data, text_native_eol)
             with self.open(self.filename, "rt") as f:
                 self.assertEqual(f.read(), text)
             with self.open(self.filename, "at") as f:
                 f.write(text)
             with open(self.filename, "rb") as f:
-                file_data = self.decompress(f.read()).decode("ascii")
+                file_data = ext_decompress(f.read()).decode("ascii")
                 self.assertEqual(file_data, text_native_eol * 2)
 
     def test_x_mode(self):
@@ -973,7 +966,7 @@
         with self.open(self.filename, "wt", encoding="utf-16-le") as f:
             f.write(text)
         with open(self.filename, "rb") as f:
-            file_data = self.decompress(f.read()).decode("utf-16-le")
+            file_data = ext_decompress(f.read()).decode("utf-16-le")
             self.assertEqual(file_data, text_native_eol)
         with self.open(self.filename, "rt", encoding="utf-16-le") as f:
             self.assertEqual(f.read(), text)

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Fri Nov 11 10:13:26 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Fri, 11 Nov 2016 15:13:26 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python Default
 2016-11-11
Message-ID: <cab429a5-32a4-4750-a0af-3ffd5812084f@irsmsx106.ger.corp.intel.com>

Results for project Python default, build date 2016-11-11 03:01:38 +0000
commit:		59b91b4e9506
previous commit:	784fea019cab
revision date:	2016-11-11 01:13:35 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.18%          0.18%          7.06%            15.86%
:-)             pybench      0.28%          0.36%          6.13%             4.67%
:-|            regex_v8      2.73%         -0.03%         -1.34%             3.99%
:-)               nbody      0.07%          3.57%          4.40%             2.25%
:-)        json_dump_v2      0.42%          6.20%         -2.25%             8.61%
:-|      normal_startup      0.98%          0.52%          0.30%             6.52%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-11-11/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Fri Nov 11 12:03:25 2016
From: python-checkins at python.org (terry.reedy)
Date: Fri, 11 Nov 2016 17:03:25 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3ODU0?=
 =?utf-8?q?=3A_Include_idlelib/help=2Ehtml_in_2=2E7_Windows_installer=2E?=
Message-ID: <20161111170321.8391.93572.B7D5EA9F@psf.io>

https://hg.python.org/cpython/rev/2776720f549c
changeset:   105062:2776720f549c
branch:      2.7
parent:      105055:237ef36fb1bb
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Fri Nov 11 12:03:09 2016 -0500
summary:
  Issue #27854: Include idlelib/help.html in 2.7 Windows installer.
Without this file, clicking Help => IDLE Help did nothing.

files:
  Lib/idlelib/NEWS.txt |  6 ++++++
  Misc/NEWS            |  6 ++++++
  Tools/msi/msi.py     |  1 +
  3 files changed, 13 insertions(+), 0 deletions(-)


diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt
--- a/Lib/idlelib/NEWS.txt
+++ b/Lib/idlelib/NEWS.txt
@@ -2,6 +2,12 @@
 ==========================
 *Release date: 2017-01-01?*
 
+- Issue #27854: Make Help => IDLE Help work again on Windows.
+  Include idlelib/help.html in 2.7 Windows installer.
+
+- Issue #25507: Add back import needed for 2.x encoding warning box.
+  Add pointer to 'Encoding declaration' in Language Reference.
+
 - Issue #15308: Add 'interrupt execution' (^C) to Shell menu.
   Patch by Roger Serwy, updated by Bayard Randel.
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -220,6 +220,12 @@
 IDLE
 ----
 
+- Issue #27854: Make Help => IDLE Help work again on Windows.
+  Include idlelib/help.html in 2.7 Windows installer.
+
+- Issue #25507: Add back import needed for 2.x encoding warning box.
+  Add pointer to 'Encoding declaration' in Language Reference.
+
 - Issue #15308: Add 'interrupt execution' (^C) to Shell menu.
   Patch by Roger Serwy, updated by Bayard Randel.
 
diff --git a/Tools/msi/msi.py b/Tools/msi/msi.py
--- a/Tools/msi/msi.py
+++ b/Tools/msi/msi.py
@@ -1076,6 +1076,7 @@
         if dir=='idlelib':
             lib.glob("*.def")
             lib.add_file("idle.bat")
+            lib.add_file("help.html")
         if dir=="Icons":
             lib.glob("*.gif")
             lib.glob("*.ico")

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 18:58:36 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Fri, 11 Nov 2016 23:58:36 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_28644=3A_Document_recent_changes_in_typing=2Epy_?=
 =?utf-8?b?KEl2YW4gTCkgKDMuNi0+My43KQ==?=
Message-ID: <20161111235836.25322.28326.2645A305@psf.io>

https://hg.python.org/cpython/rev/c4394da344b7
changeset:   105065:c4394da344b7
parent:      105061:969e85a7a943
parent:      105064:72a2c90abdec
user:        Guido van Rossum <guido at python.org>
date:        Fri Nov 11 15:58:03 2016 -0800
summary:
  Issue 28644: Document recent changes in typing.py (Ivan L) (3.6->3.7)

files:
  Doc/library/typing.rst |  65 +++++++++++++++++++++++------
  1 files changed, 50 insertions(+), 15 deletions(-)


diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst
--- a/Doc/library/typing.rst
+++ b/Doc/library/typing.rst
@@ -11,7 +11,7 @@
 --------------
 
 This module supports type hints as specified by :pep:`484` and :pep:`526`.
-The most fundamental support consists of the type :data:`Any`, :data:`Union`,
+The most fundamental support consists of the types :data:`Any`, :data:`Union`,
 :data:`Tuple`, :data:`Callable`, :class:`TypeVar`, and
 :class:`Generic`.  For full specification please see :pep:`484`.  For
 a simplified introduction to type hints see :pep:`483`.
@@ -274,6 +274,22 @@
 
    class MyIterable(Iterable): # Same as Iterable[Any]
 
+User defined generic type aliases are also supported. Examples::
+
+   from typing import TypeVar, Iterable, Tuple, Union
+   S = TypeVar('S')
+   Response = Union[Iterable[S], int]
+
+   # Return type here is same as Union[Iterable[str], int]
+   def response(query: str) -> Response[str]:
+       ...
+
+   T = TypeVar('T', int, float, complex)
+   Vec = Iterable[Tuple[T, T]]
+
+   def inproduct(v: Vec[T]) -> T: # Same as Iterable[Tuple[T, T]]
+       return sum(x*y for x, y in v)
+
 The metaclass used by :class:`Generic` is a subclass of :class:`abc.ABCMeta`.
 A generic class can be an ABC by including abstract methods or properties,
 and generic classes can also have ABCs as base classes without a metaclass
@@ -588,6 +604,19 @@
 
    A generic version of :class:`collections.abc.Awaitable`.
 
+.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co])
+
+   A generic version of :class:`collections.abc.Coroutine`.
+   The variance and order of type variables
+   correspond to those of :class:`Generator`, for example::
+
+      from typing import List, Coroutine
+      c = None # type: Coroutine[List[str], str, int]
+      ...
+      x = c.send('hi') # type: List[str]
+      async def bar() -> None:
+          x = await c # type: int
+
 .. class:: AsyncIterable(Generic[T_co])
 
    A generic version of :class:`collections.abc.AsyncIterable`.
@@ -721,13 +750,18 @@
    runtime we intentionally don't check anything (we want this
    to be as fast as possible).
 
-.. function:: get_type_hints(obj)
+.. function:: get_type_hints(obj[, globals[, locals])
 
-   Return type hints for a class, module, function or method object.
+   Return a dictionary containing type hints for a function, method, module
+   or class object.
 
-   This is often the same as ``obj.__annotations__``, but it handles
-   forward references encoded as string literals, and if necessary
-   adds ``Optional[t]`` if a default value equal to ``None`` is set.
+   This is often the same as ``obj.__annotations__``. In addition,
+   forward references encoded as string literals are handled by evaluating
+   them in ``globals`` and ``locals`` namespaces. If necessary,
+   ``Optional[t]`` is added for function and method annotations if a default
+   value equal to ``None`` is set. For a class ``C``, return
+   a dictionary constructed by merging all the ``__annotations__`` along
+   ``C.__mro__`` in reverse order.
 
 .. decorator:: overload
 
@@ -830,16 +864,16 @@
 
 .. data:: Tuple
 
-  Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items
-  with the first item of type X and the second of type Y.
+   Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items
+   with the first item of type X and the second of type Y.
 
-  Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding
-  to type variables T1 and T2.  ``Tuple[int, float, str]`` is a tuple
-  of an int, a float and a string.
+   Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding
+   to type variables T1 and T2.  ``Tuple[int, float, str]`` is a tuple
+   of an int, a float and a string.
 
-  To specify a variable-length tuple of homogeneous type,
-  use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple`
-  is equivalent to ``Tuple[Any, ...]``, and in turn to :data:`tuple`.
+   To specify a variable-length tuple of homogeneous type,
+   use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple`
+   is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`.
 
 .. data:: Callable
 
@@ -847,7 +881,8 @@
 
    The subscription syntax must always be used with exactly two
    values: the argument list and the return type.  The argument list
-   must be a list of types; the return type must be a single type.
+   must be a list of types or an ellipsis; the return type must be
+   a single type.
 
    There is no syntax to indicate optional or keyword arguments;
    such function types are rarely used as callback types.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 18:58:36 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Fri, 11 Nov 2016 23:58:36 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_28644=3A_Document_recent_changes_in_typing=2Epy_=28Ivan_?=
 =?utf-8?b?TCkgKDMuNS0+My42KQ==?=
Message-ID: <20161111235836.12178.15166.DFE2CB24@psf.io>

https://hg.python.org/cpython/rev/72a2c90abdec
changeset:   105064:72a2c90abdec
branch:      3.6
parent:      105060:9184f7f11b30
parent:      105063:5bf2ea0d3830
user:        Guido van Rossum <guido at python.org>
date:        Fri Nov 11 15:57:09 2016 -0800
summary:
  Issue 28644: Document recent changes in typing.py (Ivan L) (3.5->3.6)

files:
  Doc/library/typing.rst |  65 +++++++++++++++++++++++------
  1 files changed, 50 insertions(+), 15 deletions(-)


diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst
--- a/Doc/library/typing.rst
+++ b/Doc/library/typing.rst
@@ -11,7 +11,7 @@
 --------------
 
 This module supports type hints as specified by :pep:`484` and :pep:`526`.
-The most fundamental support consists of the type :data:`Any`, :data:`Union`,
+The most fundamental support consists of the types :data:`Any`, :data:`Union`,
 :data:`Tuple`, :data:`Callable`, :class:`TypeVar`, and
 :class:`Generic`.  For full specification please see :pep:`484`.  For
 a simplified introduction to type hints see :pep:`483`.
@@ -274,6 +274,22 @@
 
    class MyIterable(Iterable): # Same as Iterable[Any]
 
+User defined generic type aliases are also supported. Examples::
+
+   from typing import TypeVar, Iterable, Tuple, Union
+   S = TypeVar('S')
+   Response = Union[Iterable[S], int]
+
+   # Return type here is same as Union[Iterable[str], int]
+   def response(query: str) -> Response[str]:
+       ...
+
+   T = TypeVar('T', int, float, complex)
+   Vec = Iterable[Tuple[T, T]]
+
+   def inproduct(v: Vec[T]) -> T: # Same as Iterable[Tuple[T, T]]
+       return sum(x*y for x, y in v)
+
 The metaclass used by :class:`Generic` is a subclass of :class:`abc.ABCMeta`.
 A generic class can be an ABC by including abstract methods or properties,
 and generic classes can also have ABCs as base classes without a metaclass
@@ -588,6 +604,19 @@
 
    A generic version of :class:`collections.abc.Awaitable`.
 
+.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co])
+
+   A generic version of :class:`collections.abc.Coroutine`.
+   The variance and order of type variables
+   correspond to those of :class:`Generator`, for example::
+
+      from typing import List, Coroutine
+      c = None # type: Coroutine[List[str], str, int]
+      ...
+      x = c.send('hi') # type: List[str]
+      async def bar() -> None:
+          x = await c # type: int
+
 .. class:: AsyncIterable(Generic[T_co])
 
    A generic version of :class:`collections.abc.AsyncIterable`.
@@ -721,13 +750,18 @@
    runtime we intentionally don't check anything (we want this
    to be as fast as possible).
 
-.. function:: get_type_hints(obj)
+.. function:: get_type_hints(obj[, globals[, locals])
 
-   Return type hints for a class, module, function or method object.
+   Return a dictionary containing type hints for a function, method, module
+   or class object.
 
-   This is often the same as ``obj.__annotations__``, but it handles
-   forward references encoded as string literals, and if necessary
-   adds ``Optional[t]`` if a default value equal to ``None`` is set.
+   This is often the same as ``obj.__annotations__``. In addition,
+   forward references encoded as string literals are handled by evaluating
+   them in ``globals`` and ``locals`` namespaces. If necessary,
+   ``Optional[t]`` is added for function and method annotations if a default
+   value equal to ``None`` is set. For a class ``C``, return
+   a dictionary constructed by merging all the ``__annotations__`` along
+   ``C.__mro__`` in reverse order.
 
 .. decorator:: overload
 
@@ -830,16 +864,16 @@
 
 .. data:: Tuple
 
-  Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items
-  with the first item of type X and the second of type Y.
+   Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items
+   with the first item of type X and the second of type Y.
 
-  Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding
-  to type variables T1 and T2.  ``Tuple[int, float, str]`` is a tuple
-  of an int, a float and a string.
+   Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding
+   to type variables T1 and T2.  ``Tuple[int, float, str]`` is a tuple
+   of an int, a float and a string.
 
-  To specify a variable-length tuple of homogeneous type,
-  use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple`
-  is equivalent to ``Tuple[Any, ...]``, and in turn to :data:`tuple`.
+   To specify a variable-length tuple of homogeneous type,
+   use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple`
+   is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`.
 
 .. data:: Callable
 
@@ -847,7 +881,8 @@
 
    The subscription syntax must always be used with exactly two
    values: the argument list and the return type.  The argument list
-   must be a list of types; the return type must be a single type.
+   must be a list of types or an ellipsis; the return type must be
+   a single type.
 
    There is no syntax to indicate optional or keyword arguments;
    such function types are rarely used as callback types.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 18:58:36 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Fri, 11 Nov 2016 23:58:36 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgMjg2NDQ6?=
 =?utf-8?q?_Document_recent_changes_in_typing=2Epy_=28Ivan_L=29?=
Message-ID: <20161111235836.4882.49408.50805DED@psf.io>

https://hg.python.org/cpython/rev/5bf2ea0d3830
changeset:   105063:5bf2ea0d3830
branch:      3.5
parent:      105059:648cd8450f4f
user:        Guido van Rossum <guido at python.org>
date:        Fri Nov 11 15:54:04 2016 -0800
summary:
  Issue 28644: Document recent changes in typing.py (Ivan L)

files:
  Doc/library/typing.rst |  65 +++++++++++++++++++++++------
  1 files changed, 50 insertions(+), 15 deletions(-)


diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst
--- a/Doc/library/typing.rst
+++ b/Doc/library/typing.rst
@@ -11,7 +11,7 @@
 --------------
 
 This module supports type hints as specified by :pep:`484`.  The most
-fundamental support consists of the type :data:`Any`, :data:`Union`,
+fundamental support consists of the types :data:`Any`, :data:`Union`,
 :data:`Tuple`, :data:`Callable`, :class:`TypeVar`, and
 :class:`Generic`.  For full specification please see :pep:`484`.  For
 a simplified introduction to type hints see :pep:`483`.
@@ -274,6 +274,22 @@
 
    class MyIterable(Iterable): # Same as Iterable[Any]
 
+User defined generic type aliases are also supported. Examples::
+
+   from typing import TypeVar, Iterable, Tuple, Union
+   S = TypeVar('S')
+   Response = Union[Iterable[S], int]
+
+   # Return type here is same as Union[Iterable[str], int]
+   def response(query: str) -> Response[str]:
+       ...
+
+   T = TypeVar('T', int, float, complex)
+   Vec = Iterable[Tuple[T, T]]
+
+   def inproduct(v: Vec[T]) -> T: # Same as Iterable[Tuple[T, T]]
+       return sum(x*y for x, y in v)
+
 The metaclass used by :class:`Generic` is a subclass of :class:`abc.ABCMeta`.
 A generic class can be an ABC by including abstract methods or properties,
 and generic classes can also have ABCs as base classes without a metaclass
@@ -582,6 +598,19 @@
 
    A generic version of :class:`collections.abc.Awaitable`.
 
+.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co])
+
+   A generic version of :class:`collections.abc.Coroutine`.
+   The variance and order of type variables
+   correspond to those of :class:`Generator`, for example::
+
+      from typing import List, Coroutine
+      c = None # type: Coroutine[List[str], str, int]
+      ...
+      x = c.send('hi') # type: List[str]
+      async def bar() -> None:
+          x = await c # type: int
+
 .. class:: AsyncIterable(Generic[T_co])
 
    A generic version of :class:`collections.abc.AsyncIterable`.
@@ -700,13 +729,18 @@
    runtime we intentionally don't check anything (we want this
    to be as fast as possible).
 
-.. function:: get_type_hints(obj)
+.. function:: get_type_hints(obj[, globals[, locals])
 
-   Return type hints for a function or method object.
+   Return a dictionary containing type hints for a function, method, module
+   or class object.
 
-   This is often the same as ``obj.__annotations__``, but it handles
-   forward references encoded as string literals, and if necessary
-   adds ``Optional[t]`` if a default value equal to ``None`` is set.
+   This is often the same as ``obj.__annotations__``. In addition,
+   forward references encoded as string literals are handled by evaluating
+   them in ``globals`` and ``locals`` namespaces. If necessary,
+   ``Optional[t]`` is added for function and method annotations if a default
+   value equal to ``None`` is set. For a class ``C``, return
+   a dictionary constructed by merging all the ``__annotations__`` along
+   ``C.__mro__`` in reverse order.
 
 .. decorator:: overload
 
@@ -809,16 +843,16 @@
 
 .. data:: Tuple
 
-  Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items
-  with the first item of type X and the second of type Y.
+   Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items
+   with the first item of type X and the second of type Y.
 
-  Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding
-  to type variables T1 and T2.  ``Tuple[int, float, str]`` is a tuple
-  of an int, a float and a string.
+   Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding
+   to type variables T1 and T2.  ``Tuple[int, float, str]`` is a tuple
+   of an int, a float and a string.
 
-  To specify a variable-length tuple of homogeneous type,
-  use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple`
-  is equivalent to ``Tuple[Any, ...]``, and in turn to :data:`tuple`.
+   To specify a variable-length tuple of homogeneous type,
+   use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple`
+   is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`.
 
 .. data:: Callable
 
@@ -826,7 +860,8 @@
 
    The subscription syntax must always be used with exactly two
    values: the argument list and the return type.  The argument list
-   must be a list of types; the return type must be a single type.
+   must be a list of types or an ellipsis; the return type must be
+   a single type.
 
    There is no syntax to indicate optional or keyword arguments;
    such function types are rarely used as callback types.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 11 19:10:01 2016
From: python-checkins at python.org (terry.reedy)
Date: Sat, 12 Nov 2016 00:10:01 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4NjE1?=
 =?utf-8?q?=3A_Backport_imaginary/complex_number_text_from_3=2Ex=2E?=
Message-ID: <20161112001001.60852.35430.2BD1D36B@psf.io>

https://hg.python.org/cpython/rev/f8d12cb7d0fd
changeset:   105066:f8d12cb7d0fd
branch:      2.7
parent:      105062:2776720f549c
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Fri Nov 11 19:09:50 2016 -0500
summary:
  Issue #28615: Backport imaginary/complex number text from 3.x.
Patch by Mariatta Wijaya.

files:
  Doc/library/stdtypes.rst |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst
--- a/Doc/library/stdtypes.rst
+++ b/Doc/library/stdtypes.rst
@@ -266,9 +266,9 @@
 Integer literals with an ``'L'`` or ``'l'`` suffix yield long integers (``'L'``
 is preferred because ``1l`` looks too much like eleven!).  Numeric literals
 containing a decimal point or an exponent sign yield floating point numbers.
-Appending ``'j'`` or ``'J'`` to a numeric literal yields a complex number with a
-zero real part. A complex numeric literal is the sum of a real and an imaginary
-part.
+Appending ``'j'`` or ``'J'`` to a numeric literal yields an imaginary number
+(a complex number with a zero real part) which you can add to an integer or
+float to get a complex number with real and imaginary parts.
 
 .. index::
    single: arithmetic

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Sat Nov 12 04:08:38 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sat, 12 Nov 2016 09:08:38 +0000
Subject: [Python-checkins] Daily reference leaks (c4394da344b7): sum=5
Message-ID: <20161112090838.60983.48429.31468F12@psf.io>

results for c4394da344b7 on branch "default"
--------------------------------------------

test_collections leaked [7, -7, 1] memory blocks, sum=1
test_functools leaked [0, 3, 1] memory blocks, sum=4


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogKPtVeX', '--timeout', '7200']

From python-checkins at python.org  Sat Nov 12 04:10:40 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Sat, 12 Nov 2016 09:10:40 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328665=3A__Use_mac?=
 =?utf-8?q?ro_form_of_PyCell=5FGET/SET?=
Message-ID: <20161112091040.25048.33844.84FAA2CA@psf.io>

https://hg.python.org/cpython/rev/5f3b7ceb394c
changeset:   105067:5f3b7ceb394c
parent:      105065:c4394da344b7
user:        Raymond Hettinger <python at rcn.com>
date:        Sat Nov 12 04:10:35 2016 -0500
summary:
  Issue #28665:  Use macro form of PyCell_GET/SET

files:
  Python/ceval.c |  12 ++++++++----
  1 files changed, 8 insertions(+), 4 deletions(-)


diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -2405,8 +2405,10 @@
 
         TARGET(DELETE_DEREF) {
             PyObject *cell = freevars[oparg];
-            if (PyCell_GET(cell) != NULL) {
-                PyCell_Set(cell, NULL);
+            PyObject *oldobj = PyCell_GET(cell);
+            if (oldobj != NULL) {
+                PyCell_SET(cell, NULL);
+                Py_DECREF(oldobj);
                 DISPATCH();
             }
             format_exc_unbound(co, oparg);
@@ -5351,8 +5353,10 @@
             PyObject **freevars = (f->f_localsplus +
                                    f->f_code->co_nlocals);
             PyObject *c = freevars[oparg];
-            if (PyCell_GET(c) == v)
-                PyCell_Set(c, NULL);
+            if (PyCell_GET(c) ==  v) {
+                PyCell_SET(c, NULL);
+                Py_DECREF(v);
+            }
             break;
         }
         case STORE_NAME:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 12 07:37:36 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sat, 12 Nov 2016 12:37:36 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzI4NjQ4?=
 =?utf-8?q?=3A_Fixed_crash_in_Py=5FDecodeLocale=28=29_in_debug_build_on_Ma?=
 =?utf-8?q?c_OS_X?=
Message-ID: <20161112123735.61667.83442.F84C9905@psf.io>

https://hg.python.org/cpython/rev/9bf1ca6ce1fe
changeset:   105068:9bf1ca6ce1fe
branch:      3.3
parent:      104977:7e66c5dc4218
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sat Nov 12 14:28:06 2016 +0200
summary:
  Issue #28648: Fixed crash in Py_DecodeLocale() in debug build on Mac OS X
when decode astral characters.

files:
  Misc/NEWS               |  3 +++
  Objects/unicodeobject.c |  2 +-
  2 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #28648: Fixed crash in Py_DecodeLocale() in debug build on Mac OS X
+  when decode astral characters.  Patch by Xiang Zhang.
+
 - Issue #26171: Fix possible integer overflow and heap corruption in
   zipimporter.get_data().
 
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -4896,7 +4896,7 @@
 #if SIZEOF_WCHAR_T == 4
             assert(0);
 #else
-            assert(Py_UNICODE_IS_SURROGATE(ch));
+            assert(ch > 0xFFFF && ch <= MAX_UNICODE);
             /*  compute and append the two surrogates: */
             unicode[outpos++] = (wchar_t)Py_UNICODE_HIGH_SURROGATE(ch);
             unicode[outpos++] = (wchar_t)Py_UNICODE_LOW_SURROGATE(ch);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 12 07:37:36 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sat, 12 Nov 2016 12:37:36 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?=
 =?utf-8?q?_Issue_=2328648=3A_Fixed_crash_in_Py=5FDecodeLocale=28=29_in_de?=
 =?utf-8?q?bug_build_on_Mac_OS_X?=
Message-ID: <20161112123736.5669.37498.8BEF4ADF@psf.io>

https://hg.python.org/cpython/rev/bfd0da08438f
changeset:   105069:bfd0da08438f
branch:      3.4
parent:      104978:730cf8cdf02c
parent:      105068:9bf1ca6ce1fe
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sat Nov 12 14:29:48 2016 +0200
summary:
  Issue #28648: Fixed crash in Py_DecodeLocale() in debug build on Mac OS X
when decode astral characters.

files:
  Misc/NEWS               |  3 +++
  Objects/unicodeobject.c |  2 +-
  2 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #28648: Fixed crash in Py_DecodeLocale() in debug build on Mac OS X
+  when decode astral characters.  Patch by Xiang Zhang.
+
 - Issue #28426: Fixed potential crash in PyUnicode_AsDecodedObject() in debug
   build.
 
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -4856,7 +4856,7 @@
 #if SIZEOF_WCHAR_T == 4
             assert(0);
 #else
-            assert(Py_UNICODE_IS_SURROGATE(ch));
+            assert(ch > 0xFFFF && ch <= MAX_UNICODE);
             /*  compute and append the two surrogates: */
             unicode[outpos++] = (wchar_t)Py_UNICODE_HIGH_SURROGATE(ch);
             unicode[outpos++] = (wchar_t)Py_UNICODE_LOW_SURROGATE(ch);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 12 07:37:36 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sat, 12 Nov 2016 12:37:36 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?=
 =?utf-8?q?_Issue_=2328648=3A_Fixed_crash_in_Py=5FDecodeLocale=28=29_in_de?=
 =?utf-8?q?bug_build_on_Mac_OS_X?=
Message-ID: <20161112123736.61578.6176.18B32C0D@psf.io>

https://hg.python.org/cpython/rev/65b5518da6e2
changeset:   105070:65b5518da6e2
branch:      3.5
parent:      105063:5bf2ea0d3830
parent:      105069:bfd0da08438f
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sat Nov 12 14:35:46 2016 +0200
summary:
  Issue #28648: Fixed crash in Py_DecodeLocale() in debug build on Mac OS X
when decode astral characters.

files:
  Misc/NEWS               |  3 +++
  Objects/unicodeobject.c |  2 +-
  2 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #28648: Fixed crash in Py_DecodeLocale() in debug build on Mac OS X
+  when decode astral characters.  Patch by Xiang Zhang.
+
 - Issue #19398: Extra slash no longer added to sys.path components in case of
   empty compile-time PYTHONPATH components.
 
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -4816,7 +4816,7 @@
 #if SIZEOF_WCHAR_T == 4
             assert(0);
 #else
-            assert(Py_UNICODE_IS_SURROGATE(ch));
+            assert(ch > 0xFFFF && ch <= MAX_UNICODE);
             /*  compute and append the two surrogates: */
             unicode[outpos++] = (wchar_t)Py_UNICODE_HIGH_SURROGATE(ch);
             unicode[outpos++] = (wchar_t)Py_UNICODE_LOW_SURROGATE(ch);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 12 07:37:36 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sat, 12 Nov 2016 12:37:36 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328648=3A_Fixed_crash_in_Py=5FDecodeLocale=28=29_in_de?=
 =?utf-8?q?bug_build_on_Mac_OS_X?=
Message-ID: <20161112123736.8391.84282.C088E1B0@psf.io>

https://hg.python.org/cpython/rev/2cbd2ec6307d
changeset:   105071:2cbd2ec6307d
branch:      3.6
parent:      105064:72a2c90abdec
parent:      105070:65b5518da6e2
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sat Nov 12 14:36:02 2016 +0200
summary:
  Issue #28648: Fixed crash in Py_DecodeLocale() in debug build on Mac OS X
when decode astral characters.

files:
  Misc/NEWS               |  3 +++
  Objects/unicodeobject.c |  2 +-
  2 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #28648: Fixed crash in Py_DecodeLocale() in debug build on Mac OS X
+  when decode astral characters.  Patch by Xiang Zhang.
+
 - Issue #19398: Extra slash no longer added to sys.path components in case of
   empty compile-time PYTHONPATH components.
 
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -5120,7 +5120,7 @@
 #if SIZEOF_WCHAR_T == 4
             assert(0);
 #else
-            assert(Py_UNICODE_IS_SURROGATE(ch));
+            assert(ch > 0xFFFF && ch <= MAX_UNICODE);
             /*  compute and append the two surrogates: */
             unicode[outpos++] = (wchar_t)Py_UNICODE_HIGH_SURROGATE(ch);
             unicode[outpos++] = (wchar_t)Py_UNICODE_LOW_SURROGATE(ch);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 12 07:37:36 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sat, 12 Nov 2016 12:37:36 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328648=3A_Fixed_crash_in_Py=5FDecodeLocale=28=29?=
 =?utf-8?q?_in_debug_build_on_Mac_OS_X?=
Message-ID: <20161112123736.12137.28586.D6046655@psf.io>

https://hg.python.org/cpython/rev/0b576ab589c5
changeset:   105072:0b576ab589c5
parent:      105067:5f3b7ceb394c
parent:      105071:2cbd2ec6307d
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sat Nov 12 14:37:11 2016 +0200
summary:
  Issue #28648: Fixed crash in Py_DecodeLocale() in debug build on Mac OS X
when decode astral characters.

files:
  Misc/NEWS               |  3 +++
  Objects/unicodeobject.c |  2 +-
  2 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #28648: Fixed crash in Py_DecodeLocale() in debug build on Mac OS X
+  when decode astral characters.  Patch by Xiang Zhang.
+
 - Issue #28665: Improve speed of the STORE_DEREF opcode by 40%.
 
 - Issue #19398: Extra slash no longer added to sys.path components in case of
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -5101,7 +5101,7 @@
 #if SIZEOF_WCHAR_T == 4
             assert(0);
 #else
-            assert(Py_UNICODE_IS_SURROGATE(ch));
+            assert(ch > 0xFFFF && ch <= MAX_UNICODE);
             /*  compute and append the two surrogates: */
             unicode[outpos++] = (wchar_t)Py_UNICODE_HIGH_SURROGATE(ch);
             unicode[outpos++] = (wchar_t)Py_UNICODE_LOW_SURROGATE(ch);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 12 15:49:06 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sat, 12 Nov 2016 20:49:06 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fixed_the_docu?=
 =?utf-8?q?mentation_of_parse=5Fconstant_argument_in_json=2Eload=28=29=2E?=
Message-ID: <20161112204906.14231.69675.DDF4ABD4@psf.io>

https://hg.python.org/cpython/rev/61b419b77752
changeset:   105073:61b419b77752
branch:      3.5
parent:      105070:65b5518da6e2
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sat Nov 12 22:47:16 2016 +0200
summary:
  Fixed the documentation of parse_constant argument in json.load().
parse_constant doesn't get called on 'null', 'true', 'false' since 3.1/2.7.

files:
  Doc/library/json.rst |  4 ++--
  Lib/json/__init__.py |  2 +-
  2 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/json.rst b/Doc/library/json.rst
--- a/Doc/library/json.rst
+++ b/Doc/library/json.rst
@@ -332,8 +332,8 @@
    (e.g. :class:`float`).
 
    *parse_constant*, if specified, will be called with one of the following
-   strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``, ``'null'``, ``'true'``,
-   ``'false'``.  This can be used to raise an exception if invalid JSON numbers
+   strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``.
+   This can be used to raise an exception if invalid JSON numbers
    are encountered.
 
    If *strict* is false (``True`` is the default), then control characters
diff --git a/Lib/json/__init__.py b/Lib/json/__init__.py
--- a/Lib/json/__init__.py
+++ b/Lib/json/__init__.py
@@ -297,7 +297,7 @@
     for JSON integers (e.g. float).
 
     ``parse_constant``, if specified, will be called with one of the
-    following strings: -Infinity, Infinity, NaN, null, true, false.
+    following strings: -Infinity, Infinity, NaN.
     This can be used to raise an exception if invalid JSON numbers
     are encountered.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 12 15:49:06 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sat, 12 Nov 2016 20:49:06 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fixed_the_docu?=
 =?utf-8?q?mentation_of_parse=5Fconstant_argument_in_json=2Eload=28=29=2E?=
Message-ID: <20161112204906.61037.64396.D8268F77@psf.io>

https://hg.python.org/cpython/rev/b968faeb8d1b
changeset:   105074:b968faeb8d1b
branch:      2.7
parent:      105066:f8d12cb7d0fd
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sat Nov 12 22:47:16 2016 +0200
summary:
  Fixed the documentation of parse_constant argument in json.load().
parse_constant doesn't get called on 'null', 'true', 'false' since 3.1/2.7.

files:
  Doc/library/json.rst |  4 ++--
  Lib/json/__init__.py |  2 +-
  2 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/json.rst b/Doc/library/json.rst
--- a/Doc/library/json.rst
+++ b/Doc/library/json.rst
@@ -344,8 +344,8 @@
    (e.g. :class:`float`).
 
    *parse_constant*, if specified, will be called with one of the following
-   strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``, ``'null'``, ``'true'``,
-   ``'false'``.  This can be used to raise an exception if invalid JSON numbers
+   strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``.
+   This can be used to raise an exception if invalid JSON numbers
    are encountered.
 
    If *strict* is false (``True`` is the default), then control characters
diff --git a/Lib/json/__init__.py b/Lib/json/__init__.py
--- a/Lib/json/__init__.py
+++ b/Lib/json/__init__.py
@@ -325,7 +325,7 @@
     for JSON integers (e.g. float).
 
     ``parse_constant``, if specified, will be called with one of the
-    following strings: -Infinity, Infinity, NaN, null, true, false.
+    following strings: -Infinity, Infinity, NaN.
     This can be used to raise an exception if invalid JSON numbers
     are encountered.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 12 15:49:06 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sat, 12 Nov 2016 20:49:06 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Fixed_the_documentation_of_parse=5Fconstant_argument_in_json?=
 =?utf-8?b?LmxvYWQoKS4=?=
Message-ID: <20161112204906.4882.95115.FE48F8C7@psf.io>

https://hg.python.org/cpython/rev/51a9c1c9272c
changeset:   105075:51a9c1c9272c
branch:      3.6
parent:      105071:2cbd2ec6307d
parent:      105073:61b419b77752
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sat Nov 12 22:48:33 2016 +0200
summary:
  Fixed the documentation of parse_constant argument in json.load().
parse_constant doesn't get called on 'null', 'true', 'false' since 3.1/2.7.

files:
  Doc/library/json.rst |  4 ++--
  Lib/json/__init__.py |  2 +-
  2 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/json.rst b/Doc/library/json.rst
--- a/Doc/library/json.rst
+++ b/Doc/library/json.rst
@@ -339,8 +339,8 @@
    (e.g. :class:`float`).
 
    *parse_constant*, if specified, will be called with one of the following
-   strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``, ``'null'``, ``'true'``,
-   ``'false'``.  This can be used to raise an exception if invalid JSON numbers
+   strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``.
+   This can be used to raise an exception if invalid JSON numbers
    are encountered.
 
    If *strict* is false (``True`` is the default), then control characters
diff --git a/Lib/json/__init__.py b/Lib/json/__init__.py
--- a/Lib/json/__init__.py
+++ b/Lib/json/__init__.py
@@ -328,7 +328,7 @@
     for JSON integers (e.g. float).
 
     ``parse_constant``, if specified, will be called with one of the
-    following strings: -Infinity, Infinity, NaN, null, true, false.
+    following strings: -Infinity, Infinity, NaN.
     This can be used to raise an exception if invalid JSON numbers
     are encountered.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 12 15:49:06 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sat, 12 Nov 2016 20:49:06 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Fixed_the_documentation_of_parse=5Fconstant_argument_in_?=
 =?utf-8?b?anNvbi5sb2FkKCku?=
Message-ID: <20161112204906.7819.15590.F6BBCB50@psf.io>

https://hg.python.org/cpython/rev/c674dade719f
changeset:   105076:c674dade719f
parent:      105072:0b576ab589c5
parent:      105075:51a9c1c9272c
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sat Nov 12 22:48:50 2016 +0200
summary:
  Fixed the documentation of parse_constant argument in json.load().
parse_constant doesn't get called on 'null', 'true', 'false' since 3.1/2.7.

files:
  Doc/library/json.rst |  4 ++--
  Lib/json/__init__.py |  2 +-
  2 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/json.rst b/Doc/library/json.rst
--- a/Doc/library/json.rst
+++ b/Doc/library/json.rst
@@ -339,8 +339,8 @@
    (e.g. :class:`float`).
 
    *parse_constant*, if specified, will be called with one of the following
-   strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``, ``'null'``, ``'true'``,
-   ``'false'``.  This can be used to raise an exception if invalid JSON numbers
+   strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``.
+   This can be used to raise an exception if invalid JSON numbers
    are encountered.
 
    If *strict* is false (``True`` is the default), then control characters
diff --git a/Lib/json/__init__.py b/Lib/json/__init__.py
--- a/Lib/json/__init__.py
+++ b/Lib/json/__init__.py
@@ -328,7 +328,7 @@
     for JSON integers (e.g. float).
 
     ``parse_constant``, if specified, will be called with one of the
-    following strings: -Infinity, Infinity, NaN, null, true, false.
+    following strings: -Infinity, Infinity, NaN.
     This can be used to raise an exception if invalid JSON numbers
     are encountered.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 12 16:40:38 2016
From: python-checkins at python.org (ned.deily)
Date: Sat, 12 Nov 2016 21:40:38 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4Njc2?=
 =?utf-8?q?=3A_Prevent_missing_=27getentropy=27_declaration_warning_on_mac?=
 =?utf-8?b?T1Mu?=
Message-ID: <20161112214038.60162.48821.72324E65@psf.io>

https://hg.python.org/cpython/rev/0efd48d4c47c
changeset:   105078:0efd48d4c47c
branch:      3.5
parent:      105073:61b419b77752
user:        Ned Deily <nad at python.org>
date:        Sat Nov 12 16:35:48 2016 -0500
summary:
  Issue #28676: Prevent missing 'getentropy' declaration warning on macOS.
Patch by Gareth Rees.

files:
  Misc/NEWS       |  3 +++
  Python/random.c |  5 +++--
  2 files changed, 6 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -597,6 +597,9 @@
 - Issue #10910: Avoid C++ compilation errors on FreeBSD and OS X.
   Also update FreedBSD version checks for the original ctype UTF-8 workaround.
 
+- Issue #28676: Prevent missing 'getentropy' declaration warning on macOS.
+  Patch by Gareth Rees.
+
 
 What's New in Python 3.5.2?
 ===========================
diff --git a/Python/random.c b/Python/random.c
--- a/Python/random.c
+++ b/Python/random.c
@@ -9,9 +9,10 @@
 #  ifdef HAVE_LINUX_RANDOM_H
 #    include <linux/random.h>
 #  endif
-#  ifdef HAVE_GETRANDOM
+#  if defined(HAVE_GETRANDOM) || defined(HAVE_GETENTROPY)
 #    include <sys/random.h>
-#  elif defined(HAVE_GETRANDOM_SYSCALL)
+#  endif
+#  if !defined(HAVE_GETRANDOM) && defined(HAVE_GETRANDOM_SYSCALL)
 #    include <sys/syscall.h>
 #  endif
 #endif

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 12 16:40:38 2016
From: python-checkins at python.org (ned.deily)
Date: Sat, 12 Nov 2016 21:40:38 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4Njc2?=
 =?utf-8?q?=3A_Prevent_missing_=27getentropy=27_declaration_warning_on_mac?=
 =?utf-8?b?T1Mu?=
Message-ID: <20161112214038.14178.77344.452E1D64@psf.io>

https://hg.python.org/cpython/rev/828251c2bccf
changeset:   105077:828251c2bccf
branch:      2.7
parent:      105074:b968faeb8d1b
user:        Ned Deily <nad at python.org>
date:        Sat Nov 12 16:34:25 2016 -0500
summary:
  Issue #28676: Prevent missing 'getentropy' declaration warning on macOS.
Initial patch by Gareth Rees.

files:
  Misc/NEWS       |  3 +++
  Python/random.c |  3 +++
  2 files changed, 6 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -295,6 +295,9 @@
 - Issue #27806: Fix 32-bit builds on macOS Sierra 10.12 broken by removal of
   deprecated QuickTime/QuickTime.h header file.  Patch by Aleks Bunin.
 
+- Issue #28676: Prevent missing 'getentropy' declaration warning on macOS.
+  Initial patch by Gareth Rees.
+
 Tools/Demos
 -----------
 
diff --git a/Python/random.c b/Python/random.c
--- a/Python/random.c
+++ b/Python/random.c
@@ -3,6 +3,9 @@
 #include <windows.h>
 #else
 #include <fcntl.h>
+#if defined(HAVE_GETRANDOM) || defined(HAVE_GETENTROPY)
+#include <sys/random.h>
+#endif
 #endif
 
 #ifdef Py_DEBUG

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 12 16:40:38 2016
From: python-checkins at python.org (ned.deily)
Date: Sat, 12 Nov 2016 21:40:38 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328676=3A_merge_from_3=2E6?=
Message-ID: <20161112214038.61134.67498.87030F1B@psf.io>

https://hg.python.org/cpython/rev/b31a7efd8b31
changeset:   105080:b31a7efd8b31
parent:      105076:c674dade719f
parent:      105079:e2faa8a22b69
user:        Ned Deily <nad at python.org>
date:        Sat Nov 12 16:39:52 2016 -0500
summary:
  Issue #28676: merge from 3.6

files:
  Misc/NEWS       |  3 +++
  Python/random.c |  5 +++--
  2 files changed, 6 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -437,6 +437,9 @@
 - Issue #15819: Remove redundant include search directory option for building
   outside the source tree.
 
+- Issue #28676: Prevent missing 'getentropy' declaration warning on macOS.
+  Patch by Gareth Rees.
+
 Tools/Demos
 -----------
 
diff --git a/Python/random.c b/Python/random.c
--- a/Python/random.c
+++ b/Python/random.c
@@ -12,9 +12,10 @@
 #  ifdef HAVE_LINUX_RANDOM_H
 #    include <linux/random.h>
 #  endif
-#  ifdef HAVE_GETRANDOM
+#  if defined(HAVE_GETRANDOM) || defined(HAVE_GETENTROPY)
 #    include <sys/random.h>
-#  elif defined(HAVE_GETRANDOM_SYSCALL)
+#  endif
+#  if !defined(HAVE_GETRANDOM) && defined(HAVE_GETRANDOM_SYSCALL)
 #    include <sys/syscall.h>
 #  endif
 #endif

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 12 16:40:38 2016
From: python-checkins at python.org (ned.deily)
Date: Sat, 12 Nov 2016 21:40:38 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328676=3A_merge_from_3=2E5?=
Message-ID: <20161112214038.5379.25612.F1C2B60D@psf.io>

https://hg.python.org/cpython/rev/e2faa8a22b69
changeset:   105079:e2faa8a22b69
branch:      3.6
parent:      105075:51a9c1c9272c
parent:      105078:0efd48d4c47c
user:        Ned Deily <nad at python.org>
date:        Sat Nov 12 16:38:03 2016 -0500
summary:
  Issue #28676: merge from 3.5

files:
  Misc/NEWS       |  6 ++++++
  Python/random.c |  5 +++--
  2 files changed, 9 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -68,6 +68,12 @@
 
 - Issue #23839: Various caches now are cleared before running every test file.
 
+Build
+-----
+
+- Issue #28676: Prevent missing 'getentropy' declaration warning on macOS.
+  Patch by Gareth Rees.
+
 
 What's New in Python 3.6.0 beta 3
 =================================
diff --git a/Python/random.c b/Python/random.c
--- a/Python/random.c
+++ b/Python/random.c
@@ -12,9 +12,10 @@
 #  ifdef HAVE_LINUX_RANDOM_H
 #    include <linux/random.h>
 #  endif
-#  ifdef HAVE_GETRANDOM
+#  if defined(HAVE_GETRANDOM) || defined(HAVE_GETENTROPY)
 #    include <sys/random.h>
-#  elif defined(HAVE_GETRANDOM_SYSCALL)
+#  endif
+#  if !defined(HAVE_GETRANDOM) && defined(HAVE_GETRANDOM_SYSCALL)
 #    include <sys/syscall.h>
 #  endif
 #endif

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 13 00:44:29 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Sun, 13 Nov 2016 05:44:29 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Fix_typos?=
Message-ID: <20161113054429.8391.21300.3EE37AFB@psf.io>

https://hg.python.org/cpython/rev/8820387f2667
changeset:   105081:8820387f2667
branch:      3.6
parent:      105079:e2faa8a22b69
user:        Raymond Hettinger <python at rcn.com>
date:        Sun Nov 13 00:42:56 2016 -0500
summary:
  Fix typos

files:
  Lib/random.py           |  2 +-
  Lib/test/test_random.py |  2 +-
  Misc/NEWS               |  2 +-
  3 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Lib/random.py b/Lib/random.py
--- a/Lib/random.py
+++ b/Lib/random.py
@@ -352,7 +352,7 @@
                 return [population[_int(random() * total)] for i in range(k)]
             cum_weights = list(_itertools.accumulate(weights))
         elif weights is not None:
-            raise TypeError('Cannot specify both weights and cumulative_weights')
+            raise TypeError('Cannot specify both weights and cumulative weights')
         if len(cum_weights) != len(population):
             raise ValueError('The number of weights does not match the population')
         bisect = _bisect.bisect
diff --git a/Lib/test/test_random.py b/Lib/test/test_random.py
--- a/Lib/test/test_random.py
+++ b/Lib/test/test_random.py
@@ -630,7 +630,7 @@
         self.assertEqual((x+stop)%step, 0)
 
     def test_choices_algorithms(self):
-        # The various ways of specifing weights should produce the same results
+        # The various ways of specifying weights should produce the same results
         choices = self.gen.choices
         n = 13132817
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,7 +123,7 @@
   by representing the scale as float value internally in Tk.  tkinter.IntVar
   now works if float value is set to underlying Tk variable.
 
-- Issue #18844: The various ways of specifing weights for random.choices()
+- Issue #18844: The various ways of specifying weights for random.choices()
   now produce the same result sequences.
 
 - Issue #28255: calendar.TextCalendar().prmonth() no longer prints a space

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 13 00:44:29 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Sun, 13 Nov 2016 05:44:29 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161113054429.59751.85440.A11F753D@psf.io>

https://hg.python.org/cpython/rev/51d07aaa70ef
changeset:   105082:51d07aaa70ef
parent:      105080:b31a7efd8b31
parent:      105081:8820387f2667
user:        Raymond Hettinger <python at rcn.com>
date:        Sun Nov 13 00:44:24 2016 -0500
summary:
  merge

files:
  Lib/random.py           |  2 +-
  Lib/test/test_random.py |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/random.py b/Lib/random.py
--- a/Lib/random.py
+++ b/Lib/random.py
@@ -352,7 +352,7 @@
                 return [population[_int(random() * total)] for i in range(k)]
             cum_weights = list(_itertools.accumulate(weights))
         elif weights is not None:
-            raise TypeError('Cannot specify both weights and cumulative_weights')
+            raise TypeError('Cannot specify both weights and cumulative weights')
         if len(cum_weights) != len(population):
             raise ValueError('The number of weights does not match the population')
         bisect = _bisect.bisect
diff --git a/Lib/test/test_random.py b/Lib/test/test_random.py
--- a/Lib/test/test_random.py
+++ b/Lib/test/test_random.py
@@ -626,7 +626,7 @@
         self.assertEqual((x+stop)%step, 0)
 
     def test_choices_algorithms(self):
-        # The various ways of specifing weights should produce the same results
+        # The various ways of specifying weights should produce the same results
         choices = self.gen.choices
         n = 13132817
 

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Sun Nov 13 04:07:48 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sun, 13 Nov 2016 09:07:48 +0000
Subject: [Python-checkins] Daily reference leaks (51d07aaa70ef): sum=4
Message-ID: <20161113090739.12137.110.6F94E979@psf.io>

results for 51d07aaa70ef on branch "default"
--------------------------------------------

test_collections leaked [0, 7, -7] memory blocks, sum=0
test_functools leaked [0, 3, 1] memory blocks, sum=4


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogxlLTaR', '--timeout', '7200']

From python-checkins at python.org  Sun Nov 13 13:43:04 2016
From: python-checkins at python.org (xavier.degaye)
Date: Sun, 13 Nov 2016 18:43:04 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5?=
Message-ID: <20161113184304.8458.72297.FA531BCC@psf.io>

https://hg.python.org/cpython/rev/f043683f9b40
changeset:   105084:f043683f9b40
branch:      3.6
parent:      105081:8820387f2667
parent:      105083:d82f43d8927a
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Sun Nov 13 19:41:07 2016 +0100
summary:
  Merge 3.5

files:
  Lib/test/test_socket.py |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -1480,7 +1480,8 @@
         # type and populates the socket object.
         #
         # On Windows this trick won't work, so the test is skipped.
-        fd, _ = tempfile.mkstemp()
+        fd, path = tempfile.mkstemp()
+        self.addCleanup(os.unlink, path)
         with socket.socket(family=42424, type=13331, fileno=fd) as s:
             self.assertEqual(s.family, 42424)
             self.assertEqual(s.type, 13331)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 13 13:43:04 2016
From: python-checkins at python.org (xavier.degaye)
Date: Sun, 13 Nov 2016 18:43:04 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42?=
Message-ID: <20161113184304.7841.49679.4D21965A@psf.io>

https://hg.python.org/cpython/rev/facef53c1d1d
changeset:   105085:facef53c1d1d
parent:      105082:51d07aaa70ef
parent:      105084:f043683f9b40
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Sun Nov 13 19:42:09 2016 +0100
summary:
  Merge 3.6

files:
  Lib/test/test_socket.py |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -1490,7 +1490,8 @@
         # type and populates the socket object.
         #
         # On Windows this trick won't work, so the test is skipped.
-        fd, _ = tempfile.mkstemp()
+        fd, path = tempfile.mkstemp()
+        self.addCleanup(os.unlink, path)
         with socket.socket(family=42424, type=13331, fileno=fd) as s:
             self.assertEqual(s.family, 42424)
             self.assertEqual(s.type, 13331)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 13 13:43:04 2016
From: python-checkins at python.org (xavier.degaye)
Date: Sun, 13 Nov 2016 18:43:04 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_temporary_?=
 =?utf-8?q?file_not_deleted_in_test=5Fsocket?=
Message-ID: <20161113184304.6853.38119.7B4882D5@psf.io>

https://hg.python.org/cpython/rev/d82f43d8927a
changeset:   105083:d82f43d8927a
branch:      3.5
parent:      105078:0efd48d4c47c
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Sun Nov 13 19:39:48 2016 +0100
summary:
  Fix temporary file not deleted in test_socket

files:
  Lib/test/test_socket.py |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -1442,7 +1442,8 @@
         # type and populates the socket object.
         #
         # On Windows this trick won't work, so the test is skipped.
-        fd, _ = tempfile.mkstemp()
+        fd, path = tempfile.mkstemp()
+        self.addCleanup(os.unlink, path)
         with socket.socket(family=42424, type=13331, fileno=fd) as s:
             self.assertEqual(s.family, 42424)
             self.assertEqual(s.type, 13331)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 13 14:48:18 2016
From: python-checkins at python.org (xavier.degaye)
Date: Sun, 13 Nov 2016 19:48:18 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42?=
Message-ID: <20161113194818.59784.30686.4D173E55@psf.io>

https://hg.python.org/cpython/rev/892f13827219
changeset:   105087:892f13827219
parent:      105085:facef53c1d1d
parent:      105086:f37ac1a003f3
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Sun Nov 13 20:47:39 2016 +0100
summary:
  Merge 3.6

files:
  Lib/test/support/__init__.py  |  12 ++++++++++--
  Lib/test/test_faulthandler.py |  14 +++++++++++++-
  2 files changed, 23 insertions(+), 3 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -90,7 +90,7 @@
     "bigmemtest", "bigaddrspacetest", "cpython_only", "get_attribute",
     "requires_IEEE_754", "skip_unless_xattr", "requires_zlib",
     "anticipate_failure", "load_package_tests", "detect_api_mismatch",
-    "check__all__",
+    "check__all__", "requires_android_level",
     # sys
     "is_jython", "is_android", "check_impl_detail", "unix_shell",
     # network
@@ -735,7 +735,8 @@
 
 is_jython = sys.platform.startswith('java')
 
-is_android = bool(sysconfig.get_config_var('ANDROID_API_LEVEL'))
+_ANDROID_API_LEVEL = sysconfig.get_config_var('ANDROID_API_LEVEL')
+is_android = (_ANDROID_API_LEVEL > 0)
 
 if sys.platform != 'win32':
     unix_shell = '/system/bin/sh' if is_android else '/bin/sh'
@@ -1725,6 +1726,13 @@
     else:
         return unittest.skip("resource {0!r} is not enabled".format(resource))
 
+def requires_android_level(level, reason):
+    if is_android and _ANDROID_API_LEVEL < level:
+        return unittest.skip('%s at Android API level %d' %
+                             (reason, _ANDROID_API_LEVEL))
+    else:
+        return _id
+
 def cpython_only(test):
     """
     Decorator for tests only applicable on CPython.
diff --git a/Lib/test/test_faulthandler.py b/Lib/test/test_faulthandler.py
--- a/Lib/test/test_faulthandler.py
+++ b/Lib/test/test_faulthandler.py
@@ -7,7 +7,7 @@
 import subprocess
 import sys
 from test import support
-from test.support import script_helper
+from test.support import script_helper, is_android, requires_android_level
 import tempfile
 import unittest
 from textwrap import dedent
@@ -42,6 +42,10 @@
     finally:
         support.unlink(filename)
 
+def requires_raise(test):
+    return (test if not is_android else
+                    requires_android_level(24, 'raise() is buggy')(test))
+
 class FaultHandlerTests(unittest.TestCase):
     def get_output(self, code, filename=None, fd=None):
         """
@@ -141,6 +145,7 @@
                 3,
                 'access violation')
 
+    @requires_raise
     def test_sigsegv(self):
         self.check_fatal_error("""
             import faulthandler
@@ -183,6 +188,7 @@
 
     @unittest.skipIf(_testcapi is None, 'need _testcapi')
     @unittest.skipUnless(hasattr(signal, 'SIGBUS'), 'need signal.SIGBUS')
+    @requires_raise
     def test_sigbus(self):
         self.check_fatal_error("""
             import _testcapi
@@ -197,6 +203,7 @@
 
     @unittest.skipIf(_testcapi is None, 'need _testcapi')
     @unittest.skipUnless(hasattr(signal, 'SIGILL'), 'need signal.SIGILL')
+    @requires_raise
     def test_sigill(self):
         self.check_fatal_error("""
             import _testcapi
@@ -240,6 +247,7 @@
             '(?:Segmentation fault|Bus error)',
             other_regex='unable to raise a stack overflow')
 
+    @requires_raise
     def test_gil_released(self):
         self.check_fatal_error("""
             import faulthandler
@@ -249,6 +257,7 @@
             3,
             'Segmentation fault')
 
+    @requires_raise
     def test_enable_file(self):
         with temporary_filename() as filename:
             self.check_fatal_error("""
@@ -263,6 +272,7 @@
 
     @unittest.skipIf(sys.platform == "win32",
                      "subprocess doesn't support pass_fds on Windows")
+    @requires_raise
     def test_enable_fd(self):
         with tempfile.TemporaryFile('wb+') as fp:
             fd = fp.fileno()
@@ -276,6 +286,7 @@
                 'Segmentation fault',
                 fd=fd)
 
+    @requires_raise
     def test_enable_single_thread(self):
         self.check_fatal_error("""
             import faulthandler
@@ -286,6 +297,7 @@
             'Segmentation fault',
             all_threads=False)
 
+    @requires_raise
     def test_disable(self):
         code = """
             import faulthandler

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 13 14:48:18 2016
From: python-checkins at python.org (xavier.degaye)
Date: Sun, 13 Nov 2016 19:48:18 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogRml4IHRlc3RfZmF1?=
 =?utf-8?q?lthandler_on_Android_where_raise=28=29_exits_with_0?=
Message-ID: <20161113194818.5669.37428.07AA6CB4@psf.io>

https://hg.python.org/cpython/rev/f37ac1a003f3
changeset:   105086:f37ac1a003f3
branch:      3.6
parent:      105084:f043683f9b40
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Sun Nov 13 20:46:46 2016 +0100
summary:
  Fix test_faulthandler on Android where raise() exits with 0

files:
  Lib/test/support/__init__.py  |  12 ++++++++++--
  Lib/test/test_faulthandler.py |  14 +++++++++++++-
  2 files changed, 23 insertions(+), 3 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -90,7 +90,7 @@
     "bigmemtest", "bigaddrspacetest", "cpython_only", "get_attribute",
     "requires_IEEE_754", "skip_unless_xattr", "requires_zlib",
     "anticipate_failure", "load_package_tests", "detect_api_mismatch",
-    "check__all__",
+    "check__all__", "requires_android_level",
     # sys
     "is_jython", "is_android", "check_impl_detail", "unix_shell",
     # network
@@ -735,7 +735,8 @@
 
 is_jython = sys.platform.startswith('java')
 
-is_android = bool(sysconfig.get_config_var('ANDROID_API_LEVEL'))
+_ANDROID_API_LEVEL = sysconfig.get_config_var('ANDROID_API_LEVEL')
+is_android = (_ANDROID_API_LEVEL > 0)
 
 if sys.platform != 'win32':
     unix_shell = '/system/bin/sh' if is_android else '/bin/sh'
@@ -1725,6 +1726,13 @@
     else:
         return unittest.skip("resource {0!r} is not enabled".format(resource))
 
+def requires_android_level(level, reason):
+    if is_android and _ANDROID_API_LEVEL < level:
+        return unittest.skip('%s at Android API level %d' %
+                             (reason, _ANDROID_API_LEVEL))
+    else:
+        return _id
+
 def cpython_only(test):
     """
     Decorator for tests only applicable on CPython.
diff --git a/Lib/test/test_faulthandler.py b/Lib/test/test_faulthandler.py
--- a/Lib/test/test_faulthandler.py
+++ b/Lib/test/test_faulthandler.py
@@ -7,7 +7,7 @@
 import subprocess
 import sys
 from test import support
-from test.support import script_helper
+from test.support import script_helper, is_android, requires_android_level
 import tempfile
 import unittest
 from textwrap import dedent
@@ -42,6 +42,10 @@
     finally:
         support.unlink(filename)
 
+def requires_raise(test):
+    return (test if not is_android else
+                    requires_android_level(24, 'raise() is buggy')(test))
+
 class FaultHandlerTests(unittest.TestCase):
     def get_output(self, code, filename=None, fd=None):
         """
@@ -141,6 +145,7 @@
                 3,
                 'access violation')
 
+    @requires_raise
     def test_sigsegv(self):
         self.check_fatal_error("""
             import faulthandler
@@ -183,6 +188,7 @@
 
     @unittest.skipIf(_testcapi is None, 'need _testcapi')
     @unittest.skipUnless(hasattr(signal, 'SIGBUS'), 'need signal.SIGBUS')
+    @requires_raise
     def test_sigbus(self):
         self.check_fatal_error("""
             import _testcapi
@@ -197,6 +203,7 @@
 
     @unittest.skipIf(_testcapi is None, 'need _testcapi')
     @unittest.skipUnless(hasattr(signal, 'SIGILL'), 'need signal.SIGILL')
+    @requires_raise
     def test_sigill(self):
         self.check_fatal_error("""
             import _testcapi
@@ -240,6 +247,7 @@
             '(?:Segmentation fault|Bus error)',
             other_regex='unable to raise a stack overflow')
 
+    @requires_raise
     def test_gil_released(self):
         self.check_fatal_error("""
             import faulthandler
@@ -249,6 +257,7 @@
             3,
             'Segmentation fault')
 
+    @requires_raise
     def test_enable_file(self):
         with temporary_filename() as filename:
             self.check_fatal_error("""
@@ -263,6 +272,7 @@
 
     @unittest.skipIf(sys.platform == "win32",
                      "subprocess doesn't support pass_fds on Windows")
+    @requires_raise
     def test_enable_fd(self):
         with tempfile.TemporaryFile('wb+') as fp:
             fd = fp.fileno()
@@ -276,6 +286,7 @@
                 'Segmentation fault',
                 fd=fd)
 
+    @requires_raise
     def test_enable_single_thread(self):
         self.check_fatal_error("""
             import faulthandler
@@ -286,6 +297,7 @@
             'Segmentation fault',
             all_threads=False)
 
+    @requires_raise
     def test_disable(self):
         code = """
             import faulthandler

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 13 15:14:36 2016
From: python-checkins at python.org (xavier.degaye)
Date: Sun, 13 Nov 2016 20:14:36 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326934=3A_Fix_test?=
 =?utf-8?q?=5Ffaulthandler_on_Android_where_raise=28=29_exits_with_0=2C?=
Message-ID: <20161113201436.60852.9758.298D592C@psf.io>

https://hg.python.org/cpython/rev/66aac9676a28
changeset:   105088:66aac9676a28
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Sun Nov 13 21:14:03 2016 +0100
summary:
  Issue #26934: Fix test_faulthandler on Android where raise() exits with 0,
with a cosmetic change to add a commit message with the issue number,
missing from the previous two commits.

files:
  Lib/test/test_faulthandler.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_faulthandler.py b/Lib/test/test_faulthandler.py
--- a/Lib/test/test_faulthandler.py
+++ b/Lib/test/test_faulthandler.py
@@ -44,7 +44,7 @@
 
 def requires_raise(test):
     return (test if not is_android else
-                    requires_android_level(24, 'raise() is buggy')(test))
+                   requires_android_level(24, 'raise() is buggy')(test))
 
 class FaultHandlerTests(unittest.TestCase):
     def get_output(self, code, filename=None, fd=None):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 13 15:57:29 2016
From: python-checkins at python.org (xavier.degaye)
Date: Sun, 13 Nov 2016 20:57:29 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI2OTM0?=
 =?utf-8?q?=3A_Handle_=5FANDROID=5FAPI=5FLEVEL_is_None_on_Windows?=
Message-ID: <20161113205729.13630.79463.150714A2@psf.io>

https://hg.python.org/cpython/rev/2639afcedaad
changeset:   105089:2639afcedaad
branch:      3.6
parent:      105086:f37ac1a003f3
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Sun Nov 13 21:55:52 2016 +0100
summary:
  Issue #26934: Handle _ANDROID_API_LEVEL is None on Windows

files:
  Lib/test/support/__init__.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -736,7 +736,7 @@
 is_jython = sys.platform.startswith('java')
 
 _ANDROID_API_LEVEL = sysconfig.get_config_var('ANDROID_API_LEVEL')
-is_android = (_ANDROID_API_LEVEL > 0)
+is_android = (_ANDROID_API_LEVEL is not None and _ANDROID_API_LEVEL > 0)
 
 if sys.platform != 'win32':
     unix_shell = '/system/bin/sh' if is_android else '/bin/sh'

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 13 15:57:29 2016
From: python-checkins at python.org (xavier.degaye)
Date: Sun, 13 Nov 2016 20:57:29 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2326934=3A_Merge_3=2E6?=
Message-ID: <20161113205729.60457.56836.7436C313@psf.io>

https://hg.python.org/cpython/rev/4012e28539c4
changeset:   105090:4012e28539c4
parent:      105088:66aac9676a28
parent:      105089:2639afcedaad
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Sun Nov 13 21:56:59 2016 +0100
summary:
  Issue #26934: Merge 3.6

files:
  Lib/test/support/__init__.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -736,7 +736,7 @@
 is_jython = sys.platform.startswith('java')
 
 _ANDROID_API_LEVEL = sysconfig.get_config_var('ANDROID_API_LEVEL')
-is_android = (_ANDROID_API_LEVEL > 0)
+is_android = (_ANDROID_API_LEVEL is not None and _ANDROID_API_LEVEL > 0)
 
 if sys.platform != 'win32':
     unix_shell = '/system/bin/sh' if is_android else '/bin/sh'

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 13 18:53:21 2016
From: python-checkins at python.org (martin.panter)
Date: Sun, 13 Nov 2016 23:53:21 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4Njc4?=
 =?utf-8?q?=3A_Fix_references_to_numeric=5Fowner_parameter?=
Message-ID: <20161113235321.60012.37616.FFFCD7B7@psf.io>

https://hg.python.org/cpython/rev/4aca14dbb66d
changeset:   105091:4aca14dbb66d
branch:      3.5
parent:      105083:d82f43d8927a
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sun Nov 13 23:25:06 2016 +0000
summary:
  Issue #28678: Fix references to numeric_owner parameter

files:
  Doc/library/tarfile.rst |  4 ++--
  Doc/whatsnew/3.5.rst    |  2 +-
  2 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst
--- a/Doc/library/tarfile.rst
+++ b/Doc/library/tarfile.rst
@@ -388,7 +388,7 @@
       dots ``".."``.
 
    .. versionchanged:: 3.5
-      Added the *numeric_only* parameter.
+      Added the *numeric_owner* parameter.
 
 
 .. method:: TarFile.extract(member, path="", set_attrs=True, *, numeric_owner=False)
@@ -416,7 +416,7 @@
       Added the *set_attrs* parameter.
 
    .. versionchanged:: 3.5
-      Added the *numeric_only* parameter.
+      Added the *numeric_owner* parameter.
 
 .. method:: TarFile.extractfile(member)
 
diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst
--- a/Doc/whatsnew/3.5.rst
+++ b/Doc/whatsnew/3.5.rst
@@ -1891,7 +1891,7 @@
 
 The :meth:`TarFile.extractall() <tarfile.TarFile.extractall>` and
 :meth:`TarFile.extract() <tarfile.TarFile.extract>` methods now take a keyword
-argument *numeric_only*.  If set to ``True``, the extracted files and
+argument *numeric_owner*.  If set to ``True``, the extracted files and
 directories will be owned by the numeric ``uid`` and ``gid`` from the tarfile.
 If set to ``False`` (the default, and the behavior in versions prior to 3.5),
 they will be owned by the named user and group in the tarfile.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 13 18:53:21 2016
From: python-checkins at python.org (martin.panter)
Date: Sun, 13 Nov 2016 23:53:21 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328678=3A_Merge_parameter_name_from_3=2E6?=
Message-ID: <20161113235321.59665.52322.8E9DFA4F@psf.io>

https://hg.python.org/cpython/rev/817a003ecdc6
changeset:   105093:817a003ecdc6
parent:      105090:4012e28539c4
parent:      105092:5efa1a39ee59
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sun Nov 13 23:36:30 2016 +0000
summary:
  Issue #28678: Merge parameter name from 3.6

files:
  Doc/library/tarfile.rst |  4 ++--
  Doc/whatsnew/3.5.rst    |  2 +-
  2 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst
--- a/Doc/library/tarfile.rst
+++ b/Doc/library/tarfile.rst
@@ -388,7 +388,7 @@
       dots ``".."``.
 
    .. versionchanged:: 3.5
-      Added the *numeric_only* parameter.
+      Added the *numeric_owner* parameter.
 
 
 .. method:: TarFile.extract(member, path="", set_attrs=True, *, numeric_owner=False)
@@ -416,7 +416,7 @@
       Added the *set_attrs* parameter.
 
    .. versionchanged:: 3.5
-      Added the *numeric_only* parameter.
+      Added the *numeric_owner* parameter.
 
 .. method:: TarFile.extractfile(member)
 
diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst
--- a/Doc/whatsnew/3.5.rst
+++ b/Doc/whatsnew/3.5.rst
@@ -1892,7 +1892,7 @@
 
 The :meth:`TarFile.extractall() <tarfile.TarFile.extractall>` and
 :meth:`TarFile.extract() <tarfile.TarFile.extract>` methods now take a keyword
-argument *numeric_only*.  If set to ``True``, the extracted files and
+argument *numeric_owner*.  If set to ``True``, the extracted files and
 directories will be owned by the numeric ``uid`` and ``gid`` from the tarfile.
 If set to ``False`` (the default, and the behavior in versions prior to 3.5),
 they will be owned by the named user and group in the tarfile.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 13 18:53:21 2016
From: python-checkins at python.org (martin.panter)
Date: Sun, 13 Nov 2016 23:53:21 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328678=3A_Merge_parameter_name_from_3=2E5_into_3=2E6?=
Message-ID: <20161113235321.61850.82607.A59F9929@psf.io>

https://hg.python.org/cpython/rev/5efa1a39ee59
changeset:   105092:5efa1a39ee59
branch:      3.6
parent:      105089:2639afcedaad
parent:      105091:4aca14dbb66d
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sun Nov 13 23:36:07 2016 +0000
summary:
  Issue #28678: Merge parameter name from 3.5 into 3.6

files:
  Doc/library/tarfile.rst |  4 ++--
  Doc/whatsnew/3.5.rst    |  2 +-
  2 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst
--- a/Doc/library/tarfile.rst
+++ b/Doc/library/tarfile.rst
@@ -388,7 +388,7 @@
       dots ``".."``.
 
    .. versionchanged:: 3.5
-      Added the *numeric_only* parameter.
+      Added the *numeric_owner* parameter.
 
 
 .. method:: TarFile.extract(member, path="", set_attrs=True, *, numeric_owner=False)
@@ -416,7 +416,7 @@
       Added the *set_attrs* parameter.
 
    .. versionchanged:: 3.5
-      Added the *numeric_only* parameter.
+      Added the *numeric_owner* parameter.
 
 .. method:: TarFile.extractfile(member)
 
diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst
--- a/Doc/whatsnew/3.5.rst
+++ b/Doc/whatsnew/3.5.rst
@@ -1892,7 +1892,7 @@
 
 The :meth:`TarFile.extractall() <tarfile.TarFile.extractall>` and
 :meth:`TarFile.extract() <tarfile.TarFile.extract>` methods now take a keyword
-argument *numeric_only*.  If set to ``True``, the extracted files and
+argument *numeric_owner*.  If set to ``True``, the extracted files and
 directories will be owned by the numeric ``uid`` and ``gid`` from the tarfile.
 If set to ``False`` (the default, and the behavior in versions prior to 3.5),
 they will be owned by the named user and group in the tarfile.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 00:39:11 2016
From: python-checkins at python.org (martin.panter)
Date: Mon, 14 Nov 2016 05:39:11 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4MDAw?=
 =?utf-8?q?=3A_Fix_gethostbyname=5Fr=28=29_usage_on_AIX_with_=5FLINUX=5FSO?=
 =?utf-8?q?URCE=5FCOMPAT?=
Message-ID: <20161114053911.59932.24720.D4EA2813@psf.io>

https://hg.python.org/cpython/rev/7598e65a924e
changeset:   105096:7598e65a924e
branch:      3.5
parent:      105094:a2ba15f05ed1
user:        Martin Panter <vadmium+py at gmail.com>
date:        Mon Nov 14 04:26:36 2016 +0000
summary:
  Issue #28000: Fix gethostbyname_r() usage on AIX with _LINUX_SOURCE_COMPAT

Patch by Matthieu S.

files:
  Misc/ACKS              |  1 +
  Modules/socketmodule.c |  4 +++-
  2 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -1290,6 +1290,7 @@
 Chris Ryland
 Bernt R?skar Brenna
 Constantina S.
+Matthieu S
 Patrick Sabin
 S?bastien Sabl?
 Suman Saha
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -168,12 +168,14 @@
 #endif
 
 #ifdef HAVE_GETHOSTBYNAME_R
-# if defined(_AIX)
+# if defined(_AIX) && !defined(_LINUX_SOURCE_COMPAT)
 #  define HAVE_GETHOSTBYNAME_R_3_ARG
 # elif defined(__sun) || defined(__sgi)
 #  define HAVE_GETHOSTBYNAME_R_5_ARG
 # elif defined(linux)
 /* Rely on the configure script */
+# elif defined(_LINUX_SOURCE_COMPAT) /* Linux compatibility on AIX */
+#  define HAVE_GETHOSTBYNAME_R_6_ARG
 # else
 #  undef HAVE_GETHOSTBYNAME_R
 # endif

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 00:39:10 2016
From: python-checkins at python.org (martin.panter)
Date: Mon, 14 Nov 2016 05:39:10 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4MDE2?=
 =?utf-8?q?=3A_Skip_/dev/tty_seekable=28=29_test_on_AIX?=
Message-ID: <20161114053910.59885.47514.BA930896@psf.io>

https://hg.python.org/cpython/rev/a2ba15f05ed1
changeset:   105094:a2ba15f05ed1
branch:      3.5
parent:      105091:4aca14dbb66d
user:        Martin Panter <vadmium+py at gmail.com>
date:        Mon Nov 14 01:58:57 2016 +0000
summary:
  Issue #28016: Skip /dev/tty seekable() test on AIX

files:
  Lib/test/test_fileio.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_fileio.py b/Lib/test/test_fileio.py
--- a/Lib/test/test_fileio.py
+++ b/Lib/test/test_fileio.py
@@ -377,7 +377,7 @@
                     self.assertEqual(f.writable(), True)
                     if sys.platform != "darwin" and \
                        'bsd' not in sys.platform and \
-                       not sys.platform.startswith('sunos'):
+                       not sys.platform.startswith(('sunos', 'aix')):
                         # Somehow /dev/tty appears seekable on some BSDs
                         self.assertEqual(f.seekable(), False)
                     self.assertEqual(f.isatty(), True)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 00:39:11 2016
From: python-checkins at python.org (martin.panter)
Date: Mon, 14 Nov 2016 05:39:11 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_AIX_fixes_from_3=2E5_into_3=2E6?=
Message-ID: <20161114053911.61134.63380.640B4294@psf.io>

https://hg.python.org/cpython/rev/6312f1eca2ff
changeset:   105098:6312f1eca2ff
branch:      3.6
parent:      105092:5efa1a39ee59
parent:      105096:7598e65a924e
user:        Martin Panter <vadmium+py at gmail.com>
date:        Mon Nov 14 05:04:12 2016 +0000
summary:
  Merge AIX fixes from 3.5 into 3.6

files:
  Lib/test/test_fileio.py |  2 +-
  Misc/ACKS               |  1 +
  Modules/socketmodule.c  |  4 +++-
  3 files changed, 5 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_fileio.py b/Lib/test/test_fileio.py
--- a/Lib/test/test_fileio.py
+++ b/Lib/test/test_fileio.py
@@ -377,7 +377,7 @@
                     self.assertEqual(f.writable(), True)
                     if sys.platform != "darwin" and \
                        'bsd' not in sys.platform and \
-                       not sys.platform.startswith('sunos'):
+                       not sys.platform.startswith(('sunos', 'aix')):
                         # Somehow /dev/tty appears seekable on some BSDs
                         self.assertEqual(f.seekable(), False)
                     self.assertEqual(f.isatty(), True)
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -1308,6 +1308,7 @@
 Chris Ryland
 Bernt R?skar Brenna
 Constantina S.
+Matthieu S
 Patrick Sabin
 S?bastien Sabl?
 Amit Saha
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -172,12 +172,14 @@
 #endif
 
 #ifdef HAVE_GETHOSTBYNAME_R
-# if defined(_AIX)
+# if defined(_AIX) && !defined(_LINUX_SOURCE_COMPAT)
 #  define HAVE_GETHOSTBYNAME_R_3_ARG
 # elif defined(__sun) || defined(__sgi)
 #  define HAVE_GETHOSTBYNAME_R_5_ARG
 # elif defined(__linux__)
 /* Rely on the configure script */
+# elif defined(_LINUX_SOURCE_COMPAT) /* Linux compatibility on AIX */
+#  define HAVE_GETHOSTBYNAME_R_6_ARG
 # else
 #  undef HAVE_GETHOSTBYNAME_R
 # endif

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 00:39:11 2016
From: python-checkins at python.org (martin.panter)
Date: Mon, 14 Nov 2016 05:39:11 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4MDAw?=
 =?utf-8?q?=3A_Fix_gethostbyname=5Fr=28=29_usage_on_AIX_with_=5FLINUX=5FSO?=
 =?utf-8?q?URCE=5FCOMPAT?=
Message-ID: <20161114053911.5153.91680.1CBBFA5E@psf.io>

https://hg.python.org/cpython/rev/77cb8fa47138
changeset:   105097:77cb8fa47138
branch:      2.7
parent:      105095:b1f99e09bd77
user:        Martin Panter <vadmium+py at gmail.com>
date:        Mon Nov 14 04:13:55 2016 +0000
summary:
  Issue #28000: Fix gethostbyname_r() usage on AIX with _LINUX_SOURCE_COMPAT

Patch by Matthieu S.

files:
  Misc/ACKS              |  1 +
  Modules/socketmodule.c |  4 +++-
  2 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -1214,6 +1214,7 @@
 James Rutherford
 Chris Ryland
 Constantina S.
+Matthieu S
 Patrick Sabin
 S?bastien Sabl?
 Suman Saha
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -167,12 +167,14 @@
 #endif
 
 #ifdef HAVE_GETHOSTBYNAME_R
-# if defined(_AIX) || defined(__osf__)
+# if defined(_AIX) && !defined(_LINUX_SOURCE_COMPAT) || defined(__osf__)
 #  define HAVE_GETHOSTBYNAME_R_3_ARG
 # elif defined(__sun) || defined(__sgi)
 #  define HAVE_GETHOSTBYNAME_R_5_ARG
 # elif defined(linux)
 /* Rely on the configure script */
+# elif defined(_LINUX_SOURCE_COMPAT) /* Linux compatibility on AIX */
+#  define HAVE_GETHOSTBYNAME_R_6_ARG
 # else
 #  undef HAVE_GETHOSTBYNAME_R
 # endif

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 00:39:11 2016
From: python-checkins at python.org (martin.panter)
Date: Mon, 14 Nov 2016 05:39:11 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4MDE2?=
 =?utf-8?q?=3A_Skip_/dev/tty_seekable=28=29_test_on_AIX?=
Message-ID: <20161114053910.5520.56063.076452A1@psf.io>

https://hg.python.org/cpython/rev/b1f99e09bd77
changeset:   105095:b1f99e09bd77
branch:      2.7
parent:      105077:828251c2bccf
user:        Martin Panter <vadmium+py at gmail.com>
date:        Mon Nov 14 03:35:59 2016 +0000
summary:
  Issue #28016: Skip /dev/tty seekable() test on AIX

files:
  Lib/test/test_fileio.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_fileio.py b/Lib/test/test_fileio.py
--- a/Lib/test/test_fileio.py
+++ b/Lib/test/test_fileio.py
@@ -308,7 +308,7 @@
             self.assertEqual(f.writable(), True)
             if sys.platform != "darwin" and \
                'bsd' not in sys.platform and \
-               not sys.platform.startswith('sunos'):
+               not sys.platform.startswith(('sunos', 'aix')):
                 # Somehow /dev/tty appears seekable on some BSDs
                 self.assertEqual(f.seekable(), False)
             self.assertEqual(f.isatty(), True)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 00:39:11 2016
From: python-checkins at python.org (martin.panter)
Date: Mon, 14 Nov 2016 05:39:11 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_AIX_fixes_from_3=2E6?=
Message-ID: <20161114053911.5055.10039.A4A5DEC9@psf.io>

https://hg.python.org/cpython/rev/fcf3ae3e7db4
changeset:   105099:fcf3ae3e7db4
parent:      105093:817a003ecdc6
parent:      105098:6312f1eca2ff
user:        Martin Panter <vadmium+py at gmail.com>
date:        Mon Nov 14 05:04:36 2016 +0000
summary:
  Merge AIX fixes from 3.6

files:
  Lib/test/test_fileio.py |  2 +-
  Misc/ACKS               |  1 +
  Modules/socketmodule.c  |  4 +++-
  3 files changed, 5 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_fileio.py b/Lib/test/test_fileio.py
--- a/Lib/test/test_fileio.py
+++ b/Lib/test/test_fileio.py
@@ -377,7 +377,7 @@
                     self.assertEqual(f.writable(), True)
                     if sys.platform != "darwin" and \
                        'bsd' not in sys.platform and \
-                       not sys.platform.startswith('sunos'):
+                       not sys.platform.startswith(('sunos', 'aix')):
                         # Somehow /dev/tty appears seekable on some BSDs
                         self.assertEqual(f.seekable(), False)
                     self.assertEqual(f.isatty(), True)
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -1309,6 +1309,7 @@
 Chris Ryland
 Bernt R?skar Brenna
 Constantina S.
+Matthieu S
 Patrick Sabin
 S?bastien Sabl?
 Amit Saha
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -172,12 +172,14 @@
 #endif
 
 #ifdef HAVE_GETHOSTBYNAME_R
-# if defined(_AIX)
+# if defined(_AIX) && !defined(_LINUX_SOURCE_COMPAT)
 #  define HAVE_GETHOSTBYNAME_R_3_ARG
 # elif defined(__sun) || defined(__sgi)
 #  define HAVE_GETHOSTBYNAME_R_5_ARG
 # elif defined(__linux__)
 /* Rely on the configure script */
+# elif defined(_LINUX_SOURCE_COMPAT) /* Linux compatibility on AIX */
+#  define HAVE_GETHOSTBYNAME_R_6_ARG
 # else
 #  undef HAVE_GETHOSTBYNAME_R
 # endif

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 03:15:58 2016
From: python-checkins at python.org (benjamin.peterson)
Date: Mon, 14 Nov 2016 08:15:58 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogbWVyZ2UgMy42?=
Message-ID: <20161114081558.59885.78416.800B39DC@psf.io>

https://hg.python.org/cpython/rev/71c1970f27b6
changeset:   105101:71c1970f27b6
parent:      105099:fcf3ae3e7db4
parent:      105100:3ea121235ede
user:        Benjamin Peterson <benjamin at python.org>
date:        Mon Nov 14 00:15:52 2016 -0800
summary:
  merge 3.6

files:
  Lib/test/test_asyncio/test_futures.py |   9 ++
  Modules/_asynciomodule.c              |  55 +++++++++-----
  2 files changed, 44 insertions(+), 20 deletions(-)


diff --git a/Lib/test/test_asyncio/test_futures.py b/Lib/test/test_asyncio/test_futures.py
--- a/Lib/test/test_asyncio/test_futures.py
+++ b/Lib/test/test_asyncio/test_futures.py
@@ -466,6 +466,15 @@
             self.fail('StopIteration was expected')
         self.assertEqual(result, (1, 2))
 
+    def test_future_iter_throw(self):
+        fut = self._new_future(loop=self.loop)
+        fi = iter(fut)
+        self.assertRaises(TypeError, fi.throw,
+                          Exception, Exception("elephant"), 32)
+        self.assertRaises(TypeError, fi.throw,
+                          Exception("elephant"), Exception("elephant"))
+        self.assertRaises(TypeError, fi.throw, list)
+
 
 @unittest.skipUnless(hasattr(futures, '_CFuture'),
                      'requires the C _asyncio module')
diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -1031,31 +1031,46 @@
     }
     if (tb == Py_None) {
         tb = NULL;
+    } else if (tb != NULL && !PyTraceBack_Check(tb)) {
+        PyErr_SetString(PyExc_TypeError, "throw() third argument must be a traceback");
+        return NULL;
+    }
+
+    Py_INCREF(type);
+    Py_XINCREF(val);
+    Py_XINCREF(tb);
+
+    if (PyExceptionClass_Check(type)) {
+        PyErr_NormalizeException(&type, &val, &tb);
+    } else if (PyExceptionInstance_Check(type)) {
+        if (val) {
+            PyErr_SetString(PyExc_TypeError,
+                            "instance exception may not have a separate value");
+            goto fail;
+        }
+        val = type;
+        type = PyExceptionInstance_Class(type);
+        Py_INCREF(type);
+        if (tb == NULL)
+            tb = PyException_GetTraceback(val);
+    } else {
+        PyErr_SetString(PyExc_TypeError,
+                        "exceptions must be classes deriving BaseException or "
+                        "instances of such a class");
+        goto fail;
     }
 
     Py_CLEAR(self->future);
 
-    if (tb != NULL) {
-        PyErr_Restore(type, val, tb);
-    }
-    else if (val != NULL) {
-        PyErr_SetObject(type, val);
-    }
-    else {
-        if (PyExceptionClass_Check(type)) {
-            val = PyObject_CallObject(type, NULL);
-            PyErr_SetObject(type, val);
-            Py_DECREF(val);
-        }
-        else {
-            val = type;
-            assert (PyExceptionInstance_Check(val));
-            type = (PyObject*)Py_TYPE(val);
-            assert (PyExceptionClass_Check(type));
-            PyErr_SetObject(type, val);
-        }
-    }
+    PyErr_Restore(type, val, tb);
+
     return FutureIter_iternext(self);
+
+  fail:
+    Py_DECREF(type);
+    Py_XDECREF(val);
+    Py_XDECREF(tb);
+    return NULL;
 }
 
 static PyObject *

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 03:15:58 2016
From: python-checkins at python.org (benjamin.peterson)
Date: Mon, 14 Nov 2016 08:15:58 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_correctly_emul?=
 =?utf-8?q?ate_error_semantics_of_gen=2Ethrow_in_FutureIter=5Fthrow?=
Message-ID: <20161114081558.59784.61294.956403BF@psf.io>

https://hg.python.org/cpython/rev/3ea121235ede
changeset:   105100:3ea121235ede
branch:      3.6
parent:      105098:6312f1eca2ff
user:        Benjamin Peterson <benjamin at python.org>
date:        Mon Nov 14 00:15:44 2016 -0800
summary:
  correctly emulate error semantics of gen.throw in FutureIter_throw

files:
  Lib/test/test_asyncio/test_futures.py |   9 ++
  Modules/_asynciomodule.c              |  55 +++++++++-----
  2 files changed, 44 insertions(+), 20 deletions(-)


diff --git a/Lib/test/test_asyncio/test_futures.py b/Lib/test/test_asyncio/test_futures.py
--- a/Lib/test/test_asyncio/test_futures.py
+++ b/Lib/test/test_asyncio/test_futures.py
@@ -466,6 +466,15 @@
             self.fail('StopIteration was expected')
         self.assertEqual(result, (1, 2))
 
+    def test_future_iter_throw(self):
+        fut = self._new_future(loop=self.loop)
+        fi = iter(fut)
+        self.assertRaises(TypeError, fi.throw,
+                          Exception, Exception("elephant"), 32)
+        self.assertRaises(TypeError, fi.throw,
+                          Exception("elephant"), Exception("elephant"))
+        self.assertRaises(TypeError, fi.throw, list)
+
 
 @unittest.skipUnless(hasattr(futures, '_CFuture'),
                      'requires the C _asyncio module')
diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -1031,31 +1031,46 @@
     }
     if (tb == Py_None) {
         tb = NULL;
+    } else if (tb != NULL && !PyTraceBack_Check(tb)) {
+        PyErr_SetString(PyExc_TypeError, "throw() third argument must be a traceback");
+        return NULL;
+    }
+
+    Py_INCREF(type);
+    Py_XINCREF(val);
+    Py_XINCREF(tb);
+
+    if (PyExceptionClass_Check(type)) {
+        PyErr_NormalizeException(&type, &val, &tb);
+    } else if (PyExceptionInstance_Check(type)) {
+        if (val) {
+            PyErr_SetString(PyExc_TypeError,
+                            "instance exception may not have a separate value");
+            goto fail;
+        }
+        val = type;
+        type = PyExceptionInstance_Class(type);
+        Py_INCREF(type);
+        if (tb == NULL)
+            tb = PyException_GetTraceback(val);
+    } else {
+        PyErr_SetString(PyExc_TypeError,
+                        "exceptions must be classes deriving BaseException or "
+                        "instances of such a class");
+        goto fail;
     }
 
     Py_CLEAR(self->future);
 
-    if (tb != NULL) {
-        PyErr_Restore(type, val, tb);
-    }
-    else if (val != NULL) {
-        PyErr_SetObject(type, val);
-    }
-    else {
-        if (PyExceptionClass_Check(type)) {
-            val = PyObject_CallObject(type, NULL);
-            PyErr_SetObject(type, val);
-            Py_DECREF(val);
-        }
-        else {
-            val = type;
-            assert (PyExceptionInstance_Check(val));
-            type = (PyObject*)Py_TYPE(val);
-            assert (PyExceptionClass_Check(type));
-            PyErr_SetObject(type, val);
-        }
-    }
+    PyErr_Restore(type, val, tb);
+
     return FutureIter_iternext(self);
+
+  fail:
+    Py_DECREF(type);
+    Py_XDECREF(val);
+    Py_XDECREF(tb);
+    return NULL;
 }
 
 static PyObject *

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Mon Nov 14 04:08:41 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Mon, 14 Nov 2016 09:08:41 +0000
Subject: [Python-checkins] Daily reference leaks (fcf3ae3e7db4): sum=78
Message-ID: <20161114090841.59751.96244.406FBB81@psf.io>

results for fcf3ae3e7db4 on branch "default"
--------------------------------------------

test_collections leaked [0, 0, 7] memory blocks, sum=7
test_functools leaked [0, 3, 1] memory blocks, sum=4
test_multiprocessing_spawn leaked [0, 44, 0] references, sum=44
test_multiprocessing_spawn leaked [0, 19, 2] memory blocks, sum=21
test_multiprocessing_spawn leaked [0, 2, 0] file descriptors, sum=2


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog3r7GW3', '--timeout', '7200']

From python-checkins at python.org  Mon Nov 14 06:40:58 2016
From: python-checkins at python.org (victor.stinner)
Date: Mon, 14 Nov 2016 11:40:58 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4MDgy?=
 =?utf-8?q?=3A_Add_basic_unit_tests_on_re_enums?=
Message-ID: <20161114114052.13741.27528.1A1A4452@psf.io>

https://hg.python.org/cpython/rev/be66786e95de
changeset:   105103:be66786e95de
branch:      3.6
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Nov 14 12:38:43 2016 +0100
summary:
  Issue #28082: Add basic unit tests on re enums

files:
  Lib/test/test_re.py |  6 ++++++
  1 files changed, 6 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py
--- a/Lib/test/test_re.py
+++ b/Lib/test/test_re.py
@@ -1771,6 +1771,12 @@
         self.checkPatternError(r'(?<>)', 'unknown extension ?<>', 1)
         self.checkPatternError(r'(?', 'unexpected end of pattern', 2)
 
+    def test_enum(self):
+        # Issue #28082: Check that str(flag) returns a human readable string
+        # instead of an integer
+        self.assertIn('ASCII', str(re.A))
+        self.assertIn('DOTALL', str(re.S))
+
 
 class PatternReprTests(unittest.TestCase):
     def check(self, pattern, expected):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 06:40:52 2016
From: python-checkins at python.org (victor.stinner)
Date: Mon, 14 Nov 2016 11:40:52 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NjM3?=
 =?utf-8?q?=3A_Reapply_changeset_223731925d06?=
Message-ID: <20161114114052.60983.72945.061667B6@psf.io>

https://hg.python.org/cpython/rev/5fd69d4a93e0
changeset:   105102:5fd69d4a93e0
branch:      3.6
parent:      105100:3ea121235ede
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Nov 14 12:35:55 2016 +0100
summary:
  Issue #28637: Reapply changeset 223731925d06

"issue28082: use IntFlag for re constants" by Ethan Furman.

The re module is not more used in the site module and so adding "import enum"
to re.py doesn't impact python_startup benchmark anymore.

files:
  Lib/re.py |  33 +++++++++++++++++++++------------
  1 files changed, 21 insertions(+), 12 deletions(-)


diff --git a/Lib/re.py b/Lib/re.py
--- a/Lib/re.py
+++ b/Lib/re.py
@@ -119,6 +119,7 @@
 
 """
 
+import enum
 import sre_compile
 import sre_parse
 import functools
@@ -138,18 +139,26 @@
 
 __version__ = "2.2.1"
 
-# flags
-A = ASCII = sre_compile.SRE_FLAG_ASCII # assume ascii "locale"
-I = IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case
-L = LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale
-U = UNICODE = sre_compile.SRE_FLAG_UNICODE # assume unicode "locale"
-M = MULTILINE = sre_compile.SRE_FLAG_MULTILINE # make anchors look for newline
-S = DOTALL = sre_compile.SRE_FLAG_DOTALL # make dot match newline
-X = VERBOSE = sre_compile.SRE_FLAG_VERBOSE # ignore whitespace and comments
-
-# sre extensions (experimental, don't rely on these)
-T = TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking
-DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation
+class RegexFlag(enum.IntFlag):
+    ASCII = sre_compile.SRE_FLAG_ASCII # assume ascii "locale"
+    IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case
+    LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale
+    UNICODE = sre_compile.SRE_FLAG_UNICODE # assume unicode "locale"
+    MULTILINE = sre_compile.SRE_FLAG_MULTILINE # make anchors look for newline
+    DOTALL = sre_compile.SRE_FLAG_DOTALL # make dot match newline
+    VERBOSE = sre_compile.SRE_FLAG_VERBOSE # ignore whitespace and comments
+    A = ASCII
+    I = IGNORECASE
+    L = LOCALE
+    U = UNICODE
+    M = MULTILINE
+    S = DOTALL
+    X = VERBOSE
+    # sre extensions (experimental, don't rely on these)
+    TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking
+    T = TEMPLATE
+    DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation
+globals().update(RegexFlag.__members__)
 
 # sre exception
 error = sre_compile.error

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 06:40:58 2016
From: python-checkins at python.org (victor.stinner)
Date: Mon, 14 Nov 2016 11:40:58 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42?=
Message-ID: <20161114114052.13919.54853.0C19C171@psf.io>

https://hg.python.org/cpython/rev/3cf248d10bed
changeset:   105104:3cf248d10bed
parent:      105101:71c1970f27b6
parent:      105103:be66786e95de
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Nov 14 12:39:05 2016 +0100
summary:
  Merge 3.6

files:
  Lib/re.py           |  33 +++++++++++++++++++++------------
  Lib/test/test_re.py |   6 ++++++
  2 files changed, 27 insertions(+), 12 deletions(-)


diff --git a/Lib/re.py b/Lib/re.py
--- a/Lib/re.py
+++ b/Lib/re.py
@@ -119,6 +119,7 @@
 
 """
 
+import enum
 import sre_compile
 import sre_parse
 import functools
@@ -138,18 +139,26 @@
 
 __version__ = "2.2.1"
 
-# flags
-A = ASCII = sre_compile.SRE_FLAG_ASCII # assume ascii "locale"
-I = IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case
-L = LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale
-U = UNICODE = sre_compile.SRE_FLAG_UNICODE # assume unicode "locale"
-M = MULTILINE = sre_compile.SRE_FLAG_MULTILINE # make anchors look for newline
-S = DOTALL = sre_compile.SRE_FLAG_DOTALL # make dot match newline
-X = VERBOSE = sre_compile.SRE_FLAG_VERBOSE # ignore whitespace and comments
-
-# sre extensions (experimental, don't rely on these)
-T = TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking
-DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation
+class RegexFlag(enum.IntFlag):
+    ASCII = sre_compile.SRE_FLAG_ASCII # assume ascii "locale"
+    IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case
+    LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale
+    UNICODE = sre_compile.SRE_FLAG_UNICODE # assume unicode "locale"
+    MULTILINE = sre_compile.SRE_FLAG_MULTILINE # make anchors look for newline
+    DOTALL = sre_compile.SRE_FLAG_DOTALL # make dot match newline
+    VERBOSE = sre_compile.SRE_FLAG_VERBOSE # ignore whitespace and comments
+    A = ASCII
+    I = IGNORECASE
+    L = LOCALE
+    U = UNICODE
+    M = MULTILINE
+    S = DOTALL
+    X = VERBOSE
+    # sre extensions (experimental, don't rely on these)
+    TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking
+    T = TEMPLATE
+    DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation
+globals().update(RegexFlag.__members__)
 
 # sre exception
 error = sre_compile.error
diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py
--- a/Lib/test/test_re.py
+++ b/Lib/test/test_re.py
@@ -1771,6 +1771,12 @@
         self.checkPatternError(r'(?<>)', 'unknown extension ?<>', 1)
         self.checkPatternError(r'(?', 'unexpected end of pattern', 2)
 
+    def test_enum(self):
+        # Issue #28082: Check that str(flag) returns a human readable string
+        # instead of an integer
+        self.assertIn('ASCII', str(re.A))
+        self.assertIn('DOTALL', str(re.S))
+
 
 class PatternReprTests(unittest.TestCase):
     def check(self, pattern, expected):

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Mon Nov 14 09:15:09 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Mon, 14 Nov 2016 14:15:09 +0000
Subject: [Python-checkins] UGLY Benchmark Results for Python 2.7 2016-11-14
Message-ID: <82e00758-f54e-44ab-8bc6-76b14c08d87b@irsmsx102.ger.corp.intel.com>

Results for project Python 2.7, build date 2016-11-14 03:48:18 +0000
commit:		828251c2bccf
previous commit:	137c7b92360e
revision date:	2016-11-12 21:34:25 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.14%         -0.46%          4.59%             6.70%
:-)             pybench      0.18%         -0.11%          5.78%             4.55%
:-(            regex_v8      0.64%          0.36%         -2.13%            10.55%
:-)               nbody      0.05%          3.07%          8.21%             1.99%
:-|        json_dump_v2      0.21%         -1.60%          1.36%             9.97%
:-|      normal_startup      0.76%          0.37%         -0.59%             1.94%
:-)             ssbench      0.17%         -0.17%          2.14%             1.96%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-2-7-2016-11-14/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Mon Nov 14 09:19:56 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Mon, 14 Nov 2016 14:19:56 +0000
Subject: [Python-checkins] BAD Benchmark Results for Python Default
 2016-11-14
Message-ID: <05c1f677-3d98-46b9-82f1-d79505adbb1f@irsmsx102.ger.corp.intel.com>

Results for project Python default, build date 2016-11-14 03:01:46 +0000
commit:		817a003ecdc6
previous commit:	59b91b4e9506
revision date:	2016-11-13 23:36:30 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.17%         -2.40%          4.83%            15.92%
:-)             pybench      0.10%         -0.76%          5.41%             5.28%
:-(            regex_v8      2.73%         -1.41%         -2.76%             1.98%
:-(               nbody      0.32%         -2.65%          1.86%             5.29%
:-(        json_dump_v2      0.50%         -1.87%         -4.15%            12.47%
:-|      normal_startup      0.66%         -0.16%          0.42%             7.16%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-default-2016-11-14/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Mon Nov 14 11:16:17 2016
From: python-checkins at python.org (xavier.degaye)
Date: Mon, 14 Nov 2016 16:16:17 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328662=3A_Merge_3=2E6?=
Message-ID: <20161114161615.25510.84397.26726FA9@psf.io>

https://hg.python.org/cpython/rev/c3f7d81d9050
changeset:   105106:c3f7d81d9050
parent:      105104:3cf248d10bed
parent:      105105:1b2b2cb8f962
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Mon Nov 14 17:15:45 2016 +0100
summary:
  Issue #28662: Merge 3.6

files:
  Lib/test/test_dtrace.py     |  2 +-
  Lib/test/test_shutil.py     |  3 ++-
  Lib/test/test_subprocess.py |  5 +++--
  3 files changed, 6 insertions(+), 4 deletions(-)


diff --git a/Lib/test/test_dtrace.py b/Lib/test/test_dtrace.py
--- a/Lib/test/test_dtrace.py
+++ b/Lib/test/test_dtrace.py
@@ -79,7 +79,7 @@
         try:
             output = self.trace(abspath("assert_usable" + self.EXTENSION))
             output = output.strip()
-        except FileNotFoundError as fnfe:
+        except (FileNotFoundError, PermissionError) as fnfe:
             output = str(fnfe)
         if output != "probe: success":
             raise unittest.SkipTest(
diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py
--- a/Lib/test/test_shutil.py
+++ b/Lib/test/test_shutil.py
@@ -1869,7 +1869,8 @@
         """
         try:
             size = subprocess.check_output(['stty', 'size']).decode().split()
-        except (FileNotFoundError, subprocess.CalledProcessError):
+        except (FileNotFoundError, PermissionError,
+                subprocess.CalledProcessError):
             self.skipTest("stty invocation failed")
         expected = (int(size[1]), int(size[0])) # reversed order
 
diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -293,7 +293,8 @@
         # Verify first that the call succeeds without the executable arg.
         pre_args = [sys.executable, "-c"]
         self._assert_python(pre_args)
-        self.assertRaises(FileNotFoundError, self._assert_python, pre_args,
+        self.assertRaises((FileNotFoundError, PermissionError),
+                          self._assert_python, pre_args,
                           executable="doesnotexist")
 
     @unittest.skipIf(mswindows, "executable argument replaces shell")
@@ -2753,7 +2754,7 @@
             self.assertEqual(proc.returncode, 1)
 
     def test_invalid_args(self):
-        with self.assertRaises(FileNotFoundError) as c:
+        with self.assertRaises((FileNotFoundError, PermissionError)) as c:
             with subprocess.Popen(['nonexisting_i_hope'],
                                   stdout=subprocess.PIPE,
                                   stderr=subprocess.PIPE) as proc:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 11:16:17 2016
From: python-checkins at python.org (xavier.degaye)
Date: Mon, 14 Nov 2016 16:16:17 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NjYy?=
 =?utf-8?q?=3A_Catch_PermissionError_in_tests_when_spawning_a_non_existent?=
Message-ID: <20161114161615.14075.43705.9424D8F9@psf.io>

https://hg.python.org/cpython/rev/1b2b2cb8f962
changeset:   105105:1b2b2cb8f962
branch:      3.6
parent:      105103:be66786e95de
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Mon Nov 14 17:14:42 2016 +0100
summary:
  Issue #28662: Catch PermissionError in tests when spawning a non existent program

files:
  Lib/test/test_dtrace.py     |  2 +-
  Lib/test/test_shutil.py     |  3 ++-
  Lib/test/test_subprocess.py |  5 +++--
  3 files changed, 6 insertions(+), 4 deletions(-)


diff --git a/Lib/test/test_dtrace.py b/Lib/test/test_dtrace.py
--- a/Lib/test/test_dtrace.py
+++ b/Lib/test/test_dtrace.py
@@ -79,7 +79,7 @@
         try:
             output = self.trace(abspath("assert_usable" + self.EXTENSION))
             output = output.strip()
-        except FileNotFoundError as fnfe:
+        except (FileNotFoundError, PermissionError) as fnfe:
             output = str(fnfe)
         if output != "probe: success":
             raise unittest.SkipTest(
diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py
--- a/Lib/test/test_shutil.py
+++ b/Lib/test/test_shutil.py
@@ -1869,7 +1869,8 @@
         """
         try:
             size = subprocess.check_output(['stty', 'size']).decode().split()
-        except (FileNotFoundError, subprocess.CalledProcessError):
+        except (FileNotFoundError, PermissionError,
+                subprocess.CalledProcessError):
             self.skipTest("stty invocation failed")
         expected = (int(size[1]), int(size[0])) # reversed order
 
diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -293,7 +293,8 @@
         # Verify first that the call succeeds without the executable arg.
         pre_args = [sys.executable, "-c"]
         self._assert_python(pre_args)
-        self.assertRaises(FileNotFoundError, self._assert_python, pre_args,
+        self.assertRaises((FileNotFoundError, PermissionError),
+                          self._assert_python, pre_args,
                           executable="doesnotexist")
 
     @unittest.skipIf(mswindows, "executable argument replaces shell")
@@ -2753,7 +2754,7 @@
             self.assertEqual(proc.returncode, 1)
 
     def test_invalid_args(self):
-        with self.assertRaises(FileNotFoundError) as c:
+        with self.assertRaises((FileNotFoundError, PermissionError)) as c:
             with subprocess.Popen(['nonexisting_i_hope'],
                                   stdout=subprocess.PIPE,
                                   stderr=subprocess.PIPE) as proc:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 12:31:42 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 14 Nov 2016 17:31:42 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4NTYz?=
 =?utf-8?q?=3A_Make_plural_form_selection_more_lenient_and_accepting?=
Message-ID: <20161114173141.59751.56751.E9847ED2@psf.io>

https://hg.python.org/cpython/rev/6ca91a14a555
changeset:   105108:6ca91a14a555
branch:      2.7
parent:      105097:77cb8fa47138
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 14 19:25:44 2016 +0200
summary:
  Issue #28563: Make plural form selection more lenient and accepting
non-integer numbers.  Django tests depend on this.

files:
  Lib/gettext.py           |  12 ++++++++++--
  Lib/test/test_gettext.py |  16 ++++++++++------
  2 files changed, 20 insertions(+), 8 deletions(-)


diff --git a/Lib/gettext.py b/Lib/gettext.py
--- a/Lib/gettext.py
+++ b/Lib/gettext.py
@@ -158,6 +158,14 @@
 
     return result, nexttok
 
+def _as_int(n):
+    try:
+        i = round(n)
+    except TypeError:
+        raise TypeError('Plural value must be an integer, got %s' %
+                        (n.__class__.__name__,))
+    return n
+
 def c2py(plural):
     """Gets a C expression as used in PO files for plural forms and returns a
     Python function that implements an equivalent expression.
@@ -181,11 +189,11 @@
             elif c == ')':
                 depth -= 1
 
-        ns = {}
+        ns = {'_as_int': _as_int}
         exec('''if 1:
             def func(n):
                 if not isinstance(n, int):
-                    raise ValueError('Plural value must be an integer.')
+                    n = _as_int(n)
                 return int(%s)
             ''' % result, ns)
         return ns['func']
diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py
--- a/Lib/test/test_gettext.py
+++ b/Lib/test/test_gettext.py
@@ -372,12 +372,16 @@
         self.assertRaises(ZeroDivisionError, f, 0)
 
     def test_plural_number(self):
-        f = gettext.c2py('1')
-        self.assertEqual(f(1), 1)
-        self.assertRaises(ValueError, f, 1.0)
-        self.assertRaises(ValueError, f, '1')
-        self.assertRaises(ValueError, f, [])
-        self.assertRaises(ValueError, f, object())
+        f = gettext.c2py('n != 1')
+        self.assertEqual(f(1), 0)
+        self.assertEqual(f(2), 1)
+        self.assertEqual(f(1.0), 0)
+        self.assertEqual(f(2.0), 1)
+        self.assertEqual(f(1.1), 1)
+        self.assertRaises(TypeError, f, '2')
+        self.assertRaises(TypeError, f, b'2')
+        self.assertRaises(TypeError, f, [])
+        self.assertRaises(TypeError, f, object())
 
 
 class GNUTranslationParsingTest(GettextBaseTest):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 12:31:42 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 14 Nov 2016 17:31:42 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?=
 =?utf-8?q?_Issue_=2328563=3A_Make_plural_form_selection_more_lenient_and_?=
 =?utf-8?q?accepting?=
Message-ID: <20161114173142.61667.5200.8F9E5310@psf.io>

https://hg.python.org/cpython/rev/f78a05cda5aa
changeset:   105109:f78a05cda5aa
branch:      3.4
parent:      105069:bfd0da08438f
parent:      105107:5b6cde995b3a
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 14 19:25:56 2016 +0200
summary:
  Issue #28563: Make plural form selection more lenient and accepting
non-integer numbers.  Django tests depend on this.

files:
  Lib/gettext.py           |  12 ++++++++++--
  Lib/test/test_gettext.py |  16 ++++++++++------
  2 files changed, 20 insertions(+), 8 deletions(-)


diff --git a/Lib/gettext.py b/Lib/gettext.py
--- a/Lib/gettext.py
+++ b/Lib/gettext.py
@@ -158,6 +158,14 @@
 
     return result, nexttok
 
+def _as_int(n):
+    try:
+        i = round(n)
+    except TypeError:
+        raise TypeError('Plural value must be an integer, got %s' %
+                        (n.__class__.__name__,)) from None
+    return n
+
 def c2py(plural):
     """Gets a C expression as used in PO files for plural forms and returns a
     Python function that implements an equivalent expression.
@@ -181,11 +189,11 @@
             elif c == ')':
                 depth -= 1
 
-        ns = {}
+        ns = {'_as_int': _as_int}
         exec('''if True:
             def func(n):
                 if not isinstance(n, int):
-                    raise ValueError('Plural value must be an integer.')
+                    n = _as_int(n)
                 return int(%s)
             ''' % result, ns)
         return ns['func']
diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py
--- a/Lib/test/test_gettext.py
+++ b/Lib/test/test_gettext.py
@@ -371,12 +371,16 @@
         self.assertRaises(ZeroDivisionError, f, 0)
 
     def test_plural_number(self):
-        f = gettext.c2py('1')
-        self.assertEqual(f(1), 1)
-        self.assertRaises(ValueError, f, 1.0)
-        self.assertRaises(ValueError, f, '1')
-        self.assertRaises(ValueError, f, [])
-        self.assertRaises(ValueError, f, object())
+        f = gettext.c2py('n != 1')
+        self.assertEqual(f(1), 0)
+        self.assertEqual(f(2), 1)
+        self.assertEqual(f(1.0), 0)
+        self.assertEqual(f(2.0), 1)
+        self.assertEqual(f(1.1), 1)
+        self.assertRaises(TypeError, f, '2')
+        self.assertRaises(TypeError, f, b'2')
+        self.assertRaises(TypeError, f, [])
+        self.assertRaises(TypeError, f, object())
 
 
 class GNUTranslationParsingTest(GettextBaseTest):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 12:31:41 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 14 Nov 2016 17:31:41 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzI4NTYz?=
 =?utf-8?q?=3A_Make_plural_form_selection_more_lenient_and_accepting?=
Message-ID: <20161114173141.61556.24323.D92795F0@psf.io>

https://hg.python.org/cpython/rev/5b6cde995b3a
changeset:   105107:5b6cde995b3a
branch:      3.3
parent:      105068:9bf1ca6ce1fe
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 14 19:22:12 2016 +0200
summary:
  Issue #28563: Make plural form selection more lenient and accepting
non-integer numbers.  Django tests depend on this.

files:
  Lib/gettext.py           |  12 ++++++++++--
  Lib/test/test_gettext.py |  16 ++++++++++------
  2 files changed, 20 insertions(+), 8 deletions(-)


diff --git a/Lib/gettext.py b/Lib/gettext.py
--- a/Lib/gettext.py
+++ b/Lib/gettext.py
@@ -156,6 +156,14 @@
 
     return result, nexttok
 
+def _as_int(n):
+    try:
+        i = round(n)
+    except TypeError:
+        raise TypeError('Plural value must be an integer, got %s' %
+                        (n.__class__.__name__,)) from None
+    return n
+
 def c2py(plural):
     """Gets a C expression as used in PO files for plural forms and returns a
     Python function that implements an equivalent expression.
@@ -179,11 +187,11 @@
             elif c == ')':
                 depth -= 1
 
-        ns = {}
+        ns = {'_as_int': _as_int}
         exec('''if True:
             def func(n):
                 if not isinstance(n, int):
-                    raise ValueError('Plural value must be an integer.')
+                    n = _as_int(n)
                 return int(%s)
             ''' % result, ns)
         return ns['func']
diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py
--- a/Lib/test/test_gettext.py
+++ b/Lib/test/test_gettext.py
@@ -365,12 +365,16 @@
         self.assertRaises(ZeroDivisionError, f, 0)
 
     def test_plural_number(self):
-        f = gettext.c2py('1')
-        self.assertEqual(f(1), 1)
-        self.assertRaises(ValueError, f, 1.0)
-        self.assertRaises(ValueError, f, '1')
-        self.assertRaises(ValueError, f, [])
-        self.assertRaises(ValueError, f, object())
+        f = gettext.c2py('n != 1')
+        self.assertEqual(f(1), 0)
+        self.assertEqual(f(2), 1)
+        self.assertEqual(f(1.0), 0)
+        self.assertEqual(f(2.0), 1)
+        self.assertEqual(f(1.1), 1)
+        self.assertRaises(TypeError, f, '2')
+        self.assertRaises(TypeError, f, b'2')
+        self.assertRaises(TypeError, f, [])
+        self.assertRaises(TypeError, f, object())
 
 
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 12:31:42 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 14 Nov 2016 17:31:42 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?=
 =?utf-8?q?_Issue_=2328563=3A_Make_plural_form_selection_more_lenient_and_?=
 =?utf-8?q?accepting?=
Message-ID: <20161114173142.59692.34792.71F22295@psf.io>

https://hg.python.org/cpython/rev/6a2754055ff8
changeset:   105110:6a2754055ff8
branch:      3.5
parent:      105096:7598e65a924e
parent:      105109:f78a05cda5aa
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 14 19:29:59 2016 +0200
summary:
  Issue #28563: Make plural form selection more lenient and accepting
non-integer numbers.  Django tests depend on this.

files:
  Lib/gettext.py           |  12 ++++++++++--
  Lib/test/test_gettext.py |  16 ++++++++++------
  2 files changed, 20 insertions(+), 8 deletions(-)


diff --git a/Lib/gettext.py b/Lib/gettext.py
--- a/Lib/gettext.py
+++ b/Lib/gettext.py
@@ -158,6 +158,14 @@
 
     return result, nexttok
 
+def _as_int(n):
+    try:
+        i = round(n)
+    except TypeError:
+        raise TypeError('Plural value must be an integer, got %s' %
+                        (n.__class__.__name__,)) from None
+    return n
+
 def c2py(plural):
     """Gets a C expression as used in PO files for plural forms and returns a
     Python function that implements an equivalent expression.
@@ -181,11 +189,11 @@
             elif c == ')':
                 depth -= 1
 
-        ns = {}
+        ns = {'_as_int': _as_int}
         exec('''if True:
             def func(n):
                 if not isinstance(n, int):
-                    raise ValueError('Plural value must be an integer.')
+                    n = _as_int(n)
                 return int(%s)
             ''' % result, ns)
         return ns['func']
diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py
--- a/Lib/test/test_gettext.py
+++ b/Lib/test/test_gettext.py
@@ -441,12 +441,16 @@
         self.assertRaises(ZeroDivisionError, f, 0)
 
     def test_plural_number(self):
-        f = gettext.c2py('1')
-        self.assertEqual(f(1), 1)
-        self.assertRaises(ValueError, f, 1.0)
-        self.assertRaises(ValueError, f, '1')
-        self.assertRaises(ValueError, f, [])
-        self.assertRaises(ValueError, f, object())
+        f = gettext.c2py('n != 1')
+        self.assertEqual(f(1), 0)
+        self.assertEqual(f(2), 1)
+        self.assertEqual(f(1.0), 0)
+        self.assertEqual(f(2.0), 1)
+        self.assertEqual(f(1.1), 1)
+        self.assertRaises(TypeError, f, '2')
+        self.assertRaises(TypeError, f, b'2')
+        self.assertRaises(TypeError, f, [])
+        self.assertRaises(TypeError, f, object())
 
 
 class GNUTranslationParsingTest(GettextBaseTest):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 12:31:42 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 14 Nov 2016 17:31:42 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328563=3A_Make_plural_form_selection_more_lenient_and_?=
 =?utf-8?q?accepting?=
Message-ID: <20161114173142.5348.77002.8A1C5245@psf.io>

https://hg.python.org/cpython/rev/4c201c65ce5d
changeset:   105111:4c201c65ce5d
branch:      3.6
parent:      105105:1b2b2cb8f962
parent:      105110:6a2754055ff8
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 14 19:30:36 2016 +0200
summary:
  Issue #28563: Make plural form selection more lenient and accepting
non-integer numbers.  Django tests depend on this.

files:
  Lib/gettext.py           |  12 ++++++++++--
  Lib/test/test_gettext.py |  16 ++++++++++------
  2 files changed, 20 insertions(+), 8 deletions(-)


diff --git a/Lib/gettext.py b/Lib/gettext.py
--- a/Lib/gettext.py
+++ b/Lib/gettext.py
@@ -158,6 +158,14 @@
 
     return result, nexttok
 
+def _as_int(n):
+    try:
+        i = round(n)
+    except TypeError:
+        raise TypeError('Plural value must be an integer, got %s' %
+                        (n.__class__.__name__,)) from None
+    return n
+
 def c2py(plural):
     """Gets a C expression as used in PO files for plural forms and returns a
     Python function that implements an equivalent expression.
@@ -181,11 +189,11 @@
             elif c == ')':
                 depth -= 1
 
-        ns = {}
+        ns = {'_as_int': _as_int}
         exec('''if True:
             def func(n):
                 if not isinstance(n, int):
-                    raise ValueError('Plural value must be an integer.')
+                    n = _as_int(n)
                 return int(%s)
             ''' % result, ns)
         return ns['func']
diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py
--- a/Lib/test/test_gettext.py
+++ b/Lib/test/test_gettext.py
@@ -440,12 +440,16 @@
         self.assertRaises(ZeroDivisionError, f, 0)
 
     def test_plural_number(self):
-        f = gettext.c2py('1')
-        self.assertEqual(f(1), 1)
-        self.assertRaises(ValueError, f, 1.0)
-        self.assertRaises(ValueError, f, '1')
-        self.assertRaises(ValueError, f, [])
-        self.assertRaises(ValueError, f, object())
+        f = gettext.c2py('n != 1')
+        self.assertEqual(f(1), 0)
+        self.assertEqual(f(2), 1)
+        self.assertEqual(f(1.0), 0)
+        self.assertEqual(f(2.0), 1)
+        self.assertEqual(f(1.1), 1)
+        self.assertRaises(TypeError, f, '2')
+        self.assertRaises(TypeError, f, b'2')
+        self.assertRaises(TypeError, f, [])
+        self.assertRaises(TypeError, f, object())
 
 
 class GNUTranslationParsingTest(GettextBaseTest):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 12:31:42 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 14 Nov 2016 17:31:42 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328563=3A_Make_plural_form_selection_more_lenien?=
 =?utf-8?q?t_and_accepting?=
Message-ID: <20161114173142.5016.84903.AB939405@psf.io>

https://hg.python.org/cpython/rev/d0efb8532589
changeset:   105112:d0efb8532589
parent:      105106:c3f7d81d9050
parent:      105111:4c201c65ce5d
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 14 19:31:04 2016 +0200
summary:
  Issue #28563: Make plural form selection more lenient and accepting
non-integer numbers.  Django tests depend on this.

files:
  Lib/gettext.py           |  12 ++++++++++--
  Lib/test/test_gettext.py |  16 ++++++++++------
  2 files changed, 20 insertions(+), 8 deletions(-)


diff --git a/Lib/gettext.py b/Lib/gettext.py
--- a/Lib/gettext.py
+++ b/Lib/gettext.py
@@ -158,6 +158,14 @@
 
     return result, nexttok
 
+def _as_int(n):
+    try:
+        i = round(n)
+    except TypeError:
+        raise TypeError('Plural value must be an integer, got %s' %
+                        (n.__class__.__name__,)) from None
+    return n
+
 def c2py(plural):
     """Gets a C expression as used in PO files for plural forms and returns a
     Python function that implements an equivalent expression.
@@ -181,11 +189,11 @@
             elif c == ')':
                 depth -= 1
 
-        ns = {}
+        ns = {'_as_int': _as_int}
         exec('''if True:
             def func(n):
                 if not isinstance(n, int):
-                    raise ValueError('Plural value must be an integer.')
+                    n = _as_int(n)
                 return int(%s)
             ''' % result, ns)
         return ns['func']
diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py
--- a/Lib/test/test_gettext.py
+++ b/Lib/test/test_gettext.py
@@ -440,12 +440,16 @@
         self.assertRaises(ZeroDivisionError, f, 0)
 
     def test_plural_number(self):
-        f = gettext.c2py('1')
-        self.assertEqual(f(1), 1)
-        self.assertRaises(ValueError, f, 1.0)
-        self.assertRaises(ValueError, f, '1')
-        self.assertRaises(ValueError, f, [])
-        self.assertRaises(ValueError, f, object())
+        f = gettext.c2py('n != 1')
+        self.assertEqual(f(1), 0)
+        self.assertEqual(f(2), 1)
+        self.assertEqual(f(1.0), 0)
+        self.assertEqual(f(2.0), 1)
+        self.assertEqual(f(1.1), 1)
+        self.assertRaises(TypeError, f, '2')
+        self.assertRaises(TypeError, f, b'2')
+        self.assertRaises(TypeError, f, [])
+        self.assertRaises(TypeError, f, object())
 
 
 class GNUTranslationParsingTest(GettextBaseTest):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 14:49:41 2016
From: python-checkins at python.org (yury.selivanov)
Date: Mon, 14 Nov 2016 19:49:41 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NjM1?=
 =?utf-8?q?=3A_what=27s_new_in_3=2E6=3A_add_a_few_more_notes_on_typing?=
Message-ID: <20161114194941.61909.14182.3CFF8FB2@psf.io>

https://hg.python.org/cpython/rev/993215342a95
changeset:   105113:993215342a95
branch:      3.6
parent:      105111:4c201c65ce5d
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 14 14:49:18 2016 -0500
summary:
  Issue #28635: what's new in 3.6: add a few more notes on typing

Per suggestions by Ivan Levkivskyi.
Patch by Elvis Pranskevichus.

files:
  Doc/whatsnew/3.6.rst |  30 ++++++++++++++++++++++++++----
  1 files changed, 26 insertions(+), 4 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -295,7 +295,7 @@
 possible to define *asynchronous generators*::
 
     async def ticker(delay, to):
-        """Yield numbers from 0 to `to` every `delay` seconds."""
+        """Yield numbers from 0 to *to* every *delay* seconds."""
         for i in range(to):
             yield i
             await asyncio.sleep(delay)
@@ -490,7 +490,7 @@
     07:00:00 UTC = 02:00:00 EST 0
 
 The values of the :attr:`fold <datetime.datetime.fold>` attribute have the
-value `0`  all instances except those that represent the second
+value ``0`` for all instances except those that represent the second
 (chronologically) moment in time in an ambiguous case.
 
 .. seealso::
@@ -741,6 +741,12 @@
   before the first use of the affected name in the same scope.
   Previously this was a ``SyntaxWarning``.
 
+* It is now possible to set a :ref:`special method <specialnames>` to
+  ``None`` to indicate that the corresponding operation is not available.
+  For example, if a class sets :meth:`__iter__` to ``None``, the class
+  is not iterable.
+  (Contributed by Andrew Barnert and Ivan Levkivskyi in :issue:`25958`.)
+
 * Long sequences of repeated traceback lines are now abbreviated as
   ``"[Previous line repeated {count} more times]"`` (see
   :ref:`whatsnew36-traceback` for an example).
@@ -898,8 +904,13 @@
 collections
 -----------
 
-The new :class:`~collections.Collection` abstract base class has been
+The new :class:`~collections.abc.Collection` abstract base class has been
 added to represent sized iterable container classes.
+(Contributed by Ivan Levkivskyi, docs by Neil Girdhar in :issue:`27598`.)
+
+The new :class:`~collections.abc.Reversible` abstract base class represents
+iterable classes that also provide the :meth:`__reversed__`.
+(Contributed by Ivan Levkivskyi in :issue:`25987`.)
 
 The :func:`~collections.namedtuple` function now accepts an optional
 keyword argument *module*, which, when specified, is used for
@@ -1509,6 +1520,12 @@
 in Python 3.5, all changes introduced in Python 3.6 have also been
 backported to Python 3.5.x.
 
+The :mod:`typing` module has a much improved support for generic type
+aliases.  For example ``Dict[str, Tuple[S, T]]`` is now a valid
+type annotation.
+(Contributed by Guido van Rossum in `Github #195
+<https://github.com/python/typing/pull/195>`_.)
+
 The :class:`typing.ContextManager` class has been added for
 representing :class:`contextlib.AbstractContextManager`.
 (Contributed by Brett Cannon in :issue:`25609`.)
@@ -1692,12 +1709,17 @@
 * The :class:`Task <asyncio.tasks.Task>` now has an optimized
   C implementation. (Contributed by Yury Selivanov in :issue:`28544`.)
 
+* Various implementation improvements in the :mod:`typing` module
+  (such as caching of generic types) allow up to 30 times performance
+  improvements and reduced memory footprint.
+
 * The ASCII decoder is now up to 60 times as fast for error handlers
   ``surrogateescape``, ``ignore`` and ``replace`` (Contributed
   by Victor Stinner in :issue:`24870`).
 
 * The ASCII and the Latin1 encoders are now up to 3 times as fast for the
-  error handler ``surrogateescape`` (Contributed by Victor Stinner in :issue:`25227`).
+  error handler ``surrogateescape``
+  (Contributed by Victor Stinner in :issue:`25227`).
 
 * The UTF-8 encoder is now up to 75 times as fast for error handlers
   ``ignore``, ``replace``, ``surrogateescape``, ``surrogatepass`` (Contributed

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 14:49:41 2016
From: python-checkins at python.org (yury.selivanov)
Date: Mon, 14 Nov 2016 19:49:41 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg2MzUp?=
Message-ID: <20161114194941.59751.50327.74602052@psf.io>

https://hg.python.org/cpython/rev/c3b4eea73615
changeset:   105114:c3b4eea73615
parent:      105112:d0efb8532589
parent:      105113:993215342a95
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 14 14:49:38 2016 -0500
summary:
  Merge 3.6 (issue #28635)

files:
  Doc/whatsnew/3.6.rst |  30 ++++++++++++++++++++++++++----
  1 files changed, 26 insertions(+), 4 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -295,7 +295,7 @@
 possible to define *asynchronous generators*::
 
     async def ticker(delay, to):
-        """Yield numbers from 0 to `to` every `delay` seconds."""
+        """Yield numbers from 0 to *to* every *delay* seconds."""
         for i in range(to):
             yield i
             await asyncio.sleep(delay)
@@ -490,7 +490,7 @@
     07:00:00 UTC = 02:00:00 EST 0
 
 The values of the :attr:`fold <datetime.datetime.fold>` attribute have the
-value `0`  all instances except those that represent the second
+value ``0`` for all instances except those that represent the second
 (chronologically) moment in time in an ambiguous case.
 
 .. seealso::
@@ -741,6 +741,12 @@
   before the first use of the affected name in the same scope.
   Previously this was a ``SyntaxWarning``.
 
+* It is now possible to set a :ref:`special method <specialnames>` to
+  ``None`` to indicate that the corresponding operation is not available.
+  For example, if a class sets :meth:`__iter__` to ``None``, the class
+  is not iterable.
+  (Contributed by Andrew Barnert and Ivan Levkivskyi in :issue:`25958`.)
+
 * Long sequences of repeated traceback lines are now abbreviated as
   ``"[Previous line repeated {count} more times]"`` (see
   :ref:`whatsnew36-traceback` for an example).
@@ -898,8 +904,13 @@
 collections
 -----------
 
-The new :class:`~collections.Collection` abstract base class has been
+The new :class:`~collections.abc.Collection` abstract base class has been
 added to represent sized iterable container classes.
+(Contributed by Ivan Levkivskyi, docs by Neil Girdhar in :issue:`27598`.)
+
+The new :class:`~collections.abc.Reversible` abstract base class represents
+iterable classes that also provide the :meth:`__reversed__`.
+(Contributed by Ivan Levkivskyi in :issue:`25987`.)
 
 The :func:`~collections.namedtuple` function now accepts an optional
 keyword argument *module*, which, when specified, is used for
@@ -1509,6 +1520,12 @@
 in Python 3.5, all changes introduced in Python 3.6 have also been
 backported to Python 3.5.x.
 
+The :mod:`typing` module has a much improved support for generic type
+aliases.  For example ``Dict[str, Tuple[S, T]]`` is now a valid
+type annotation.
+(Contributed by Guido van Rossum in `Github #195
+<https://github.com/python/typing/pull/195>`_.)
+
 The :class:`typing.ContextManager` class has been added for
 representing :class:`contextlib.AbstractContextManager`.
 (Contributed by Brett Cannon in :issue:`25609`.)
@@ -1692,12 +1709,17 @@
 * The :class:`Task <asyncio.tasks.Task>` now has an optimized
   C implementation. (Contributed by Yury Selivanov in :issue:`28544`.)
 
+* Various implementation improvements in the :mod:`typing` module
+  (such as caching of generic types) allow up to 30 times performance
+  improvements and reduced memory footprint.
+
 * The ASCII decoder is now up to 60 times as fast for error handlers
   ``surrogateescape``, ``ignore`` and ``replace`` (Contributed
   by Victor Stinner in :issue:`24870`).
 
 * The ASCII and the Latin1 encoders are now up to 3 times as fast for the
-  error handler ``surrogateescape`` (Contributed by Victor Stinner in :issue:`25227`).
+  error handler ``surrogateescape``
+  (Contributed by Victor Stinner in :issue:`25227`).
 
 * The UTF-8 encoder is now up to 75 times as fast for error handlers
   ``ignore``, ``replace``, ``surrogateescape``, ``surrogatepass`` (Contributed

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 19:17:09 2016
From: python-checkins at python.org (steve.dower)
Date: Tue, 15 Nov 2016 00:17:09 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fixes_incorrect_merge?=
Message-ID: <20161115001709.59885.80265.BDCCB2F2@psf.io>

https://hg.python.org/cpython/rev/6f7a3a1a0a7d
changeset:   105117:6f7a3a1a0a7d
user:        Steve Dower <steve.dower at microsoft.com>
date:        Mon Nov 14 16:16:44 2016 -0800
summary:
  Fixes incorrect merge

files:
  Tools/msi/buildrelease.bat |  5 -----
  1 files changed, 0 insertions(+), 5 deletions(-)


diff --git a/Tools/msi/buildrelease.bat b/Tools/msi/buildrelease.bat
--- a/Tools/msi/buildrelease.bat
+++ b/Tools/msi/buildrelease.bat
@@ -141,11 +141,6 @@
 )
 
 if not "%SKIPBUILD%" EQU "1" (
-    @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -d -t %TARGET% %CERTOPTS%
-    @if errorlevel 1 exit /B
-    @rem build.bat turns echo back on, so we disable it again
-    @echo off
-
     if "%PGO%" EQU "" (
         set PGOOPTS=
     ) else (

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 19:17:09 2016
From: python-checkins at python.org (steve.dower)
Date: Tue, 15 Nov 2016 00:17:09 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328573=3A_Avoid_setting_up_env_too_many_times_du?=
 =?utf-8?q?ring_build?=
Message-ID: <20161115001709.61182.88598.0073335A@psf.io>

https://hg.python.org/cpython/rev/ae8f525cef2a
changeset:   105116:ae8f525cef2a
parent:      105114:c3b4eea73615
parent:      105115:25cb7df5b19d
user:        Steve Dower <steve.dower at microsoft.com>
date:        Mon Nov 14 16:15:39 2016 -0800
summary:
  Issue #28573: Avoid setting up env too many times during build

files:
  PCbuild/build.bat          |   2 +-
  Tools/msi/buildrelease.bat |  37 ++++++-------------------
  2 files changed, 10 insertions(+), 29 deletions(-)


diff --git a/PCbuild/build.bat b/PCbuild/build.bat
--- a/PCbuild/build.bat
+++ b/PCbuild/build.bat
@@ -120,9 +120,9 @@
     @echo off
     call :Kill
     set conf=PGUpdate
+    set target=Build
 )
 goto Build
-
 :Kill
 echo on
 msbuild "%dir%\pythoncore.vcxproj" /t:KillPython %verbose%^
diff --git a/Tools/msi/buildrelease.bat b/Tools/msi/buildrelease.bat
--- a/Tools/msi/buildrelease.bat
+++ b/Tools/msi/buildrelease.bat
@@ -35,7 +35,7 @@
 set BUILDX64=
 set TARGET=Rebuild
 set TESTTARGETDIR=
-set PGO=default
+set PGO=-m test -q --pgo
 set BUILDNUGET=1
 set BUILDZIP=1
 
@@ -109,14 +109,12 @@
 @echo off
 
 if "%1" EQU "x86" (
-    call "%PCBUILD%env.bat" x86
     set PGO=
     set BUILD=%PCBUILD%win32\
     set BUILD_PLAT=Win32
     set OUTDIR_PLAT=win32
     set OBJDIR_PLAT=x86
 ) else (
-    call "%PCBUILD%env.bat" amd64
     set BUILD=%PCBUILD%amd64\
     set PGO=%~2
     set BUILD_PLAT=x64
@@ -149,29 +147,17 @@
     @echo off
 
     if "%PGO%" EQU "" (
-        @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %CERTOPTS%
+        set PGOOPTS=
     ) else (
-        @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -c PGInstrument -t %TARGET% %CERTOPTS%
-        @if errorlevel 1 exit /B
-
-        @del "%BUILD%*.pgc"
-        if "%PGO%" EQU "default" (
-            "%BUILD%python.exe" -m test -q --pgo
-        ) else if "%PGO%" EQU "default2" (
-            "%BUILD%python.exe" -m test -r -q --pgo
-            "%BUILD%python.exe" -m test -r -q --pgo
-        ) else if "%PGO%" EQU "default10" (
-            for /L %%i in (0, 1, 9) do "%BUILD%python.exe" -m test -q -r --pgo
-        ) else (
-            "%BUILD%python.exe" %PGO%
-        )
-
-        @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -c PGUpdate -t Build %CERTOPTS%
+        set PGOOPTS=--pgo --pgojob "%PGO%"
     )
+    @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% %PGOOPTS%
     @if errorlevel 1 exit /B
+    @rem build.bat turns echo back on, so we disable it again
     @echo off
 )
 
+call "%PCBUILD%env.bat"
 if "%OUTDIR_PLAT%" EQU "win32" (
     msbuild "%D%launcher\launcher.wixproj" /p:Platform=x86 %CERTOPTS% /p:ReleaseUri=%RELEASE_URI%
     if errorlevel 1 exit /B
@@ -220,10 +206,9 @@
 echo    --skip-build (-B)   Do not build Python (just do the installers)
 echo    --skip-doc (-D)     Do not build documentation
 echo    --pgo               Specify PGO command for x64 installers
-echo    --skip-pgo          Build x64 installers using PGO
+echo    --skip-pgo          Build x64 installers without using PGO
 echo    --skip-nuget        Do not build Nuget packages
 echo    --skip-zip          Do not build embeddable package
-echo    --pgo               Build x64 installers using PGO
 echo    --download          Specify the full download URL for MSIs
 echo    --test              Specify the test directory to run the installer tests
 echo    -h                  Display this help information
@@ -231,12 +216,8 @@
 echo If no architecture is specified, all architectures will be built.
 echo If --test is not specified, the installer tests are not run.
 echo.
-echo For the --pgo option, any Python command line can be used as well as the
-echo following shortcuts:
-echo     Shortcut        Description
-echo     default         Test suite with --pgo
-echo     default2        2x test suite with --pgo and randomized test order
-echo     default10       10x test suite with --pgo and randomized test order
+echo For the --pgo option, any Python command line can be used, or 'default' to
+echo use the default task (-m test --pgo).
 echo.
 echo The following substitutions will be applied to the download URL:
 echo     Variable        Description         Example

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 19:17:09 2016
From: python-checkins at python.org (steve.dower)
Date: Tue, 15 Nov 2016 00:17:09 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NTcz?=
 =?utf-8?q?=3A_Avoid_setting_up_env_too_many_times_during_build?=
Message-ID: <20161115001709.60974.76716.91869117@psf.io>

https://hg.python.org/cpython/rev/25cb7df5b19d
changeset:   105115:25cb7df5b19d
branch:      3.6
parent:      105113:993215342a95
user:        Steve Dower <steve.dower at microsoft.com>
date:        Mon Nov 14 16:13:56 2016 -0800
summary:
  Issue #28573: Avoid setting up env too many times during build

files:
  PCbuild/build.bat          |   2 +-
  Tools/msi/buildrelease.bat |  49 +++++--------------------
  2 files changed, 12 insertions(+), 39 deletions(-)


diff --git a/PCbuild/build.bat b/PCbuild/build.bat
--- a/PCbuild/build.bat
+++ b/PCbuild/build.bat
@@ -120,9 +120,9 @@
     @echo off
     call :Kill
     set conf=PGUpdate
+    set target=Build
 )
 goto Build
-
 :Kill
 echo on
 msbuild "%dir%\pythoncore.vcxproj" /t:KillPython %verbose%^
diff --git a/Tools/msi/buildrelease.bat b/Tools/msi/buildrelease.bat
--- a/Tools/msi/buildrelease.bat
+++ b/Tools/msi/buildrelease.bat
@@ -35,7 +35,7 @@
 set BUILDX64=
 set TARGET=Rebuild
 set TESTTARGETDIR=
-set PGO=default
+set PGO=-m test -q --pgo
 set BUILDNUGET=1
 set BUILDZIP=1
 
@@ -109,14 +109,12 @@
 @echo off
 
 if "%1" EQU "x86" (
-    call "%PCBUILD%env.bat" x86
     set PGO=
     set BUILD=%PCBUILD%win32\
     set BUILD_PLAT=Win32
     set OUTDIR_PLAT=win32
     set OBJDIR_PLAT=x86
 ) else (
-    call "%PCBUILD%env.bat" amd64
     set BUILD=%PCBUILD%amd64\
     set PGO=%~2
     set BUILD_PLAT=x64
@@ -143,37 +141,18 @@
 )
 
 if not "%SKIPBUILD%" EQU "1" (
-    @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -d -t %TARGET% %CERTOPTS%
+    if "%PGO%" EQU "" (
+        set PGOOPTS=
+    ) else (
+        set PGOOPTS=--pgo --pgojob "%PGO%"
+    )
+    @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% %PGOOPTS%
     @if errorlevel 1 exit /B
     @rem build.bat turns echo back on, so we disable it again
     @echo off
-    
-    if "%PGO%" EQU "" (
-        @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %CERTOPTS%
-    ) else (
-        @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -c PGInstrument -t %TARGET% %CERTOPTS%
-        @if errorlevel 1 exit /B
-        
-        @del "%BUILD%*.pgc"
-        if "%PGO%" EQU "default" (
-            "%BUILD%python.exe" -m test -q --pgo
-        ) else if "%PGO%" EQU "default2" (
-            "%BUILD%python.exe" -m test -r -q --pgo
-            "%BUILD%python.exe" -m test -r -q --pgo
-        ) else if "%PGO%" EQU "default10" (
-            for /L %%i in (0, 1, 9) do "%BUILD%python.exe" -m test -q -r --pgo
-        ) else if "%PGO%" EQU "pybench" (
-            "%BUILD%python.exe" "%PCBUILD%..\Tools\pybench\pybench.py"
-        ) else (
-            "%BUILD%python.exe" %PGO%
-        )
-        
-        @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -c PGUpdate -t Build %CERTOPTS%
-    )
-    @if errorlevel 1 exit /B
-    @echo off
 )
 
+call "%PCBUILD%env.bat"
 if "%OUTDIR_PLAT%" EQU "win32" (
     msbuild "%D%launcher\launcher.wixproj" /p:Platform=x86 %CERTOPTS% /p:ReleaseUri=%RELEASE_URI%
     if errorlevel 1 exit /B
@@ -222,10 +201,9 @@
 echo    --skip-build (-B)   Do not build Python (just do the installers)
 echo    --skip-doc (-D)     Do not build documentation
 echo    --pgo               Specify PGO command for x64 installers
-echo    --skip-pgo          Build x64 installers using PGO
+echo    --skip-pgo          Build x64 installers without using PGO
 echo    --skip-nuget        Do not build Nuget packages
 echo    --skip-zip          Do not build embeddable package
-echo    --pgo               Build x64 installers using PGO
 echo    --download          Specify the full download URL for MSIs
 echo    --test              Specify the test directory to run the installer tests
 echo    -h                  Display this help information
@@ -233,13 +211,8 @@
 echo If no architecture is specified, all architectures will be built.
 echo If --test is not specified, the installer tests are not run.
 echo.
-echo For the --pgo option, any Python command line can be used as well as the
-echo following shortcuts:
-echo     Shortcut        Description
-echo     default         Test suite with --pgo
-echo     default2        2x test suite with --pgo and randomized test order
-echo     default10       10x test suite with --pgo and randomized test order
-echo     pybench         pybench script
+echo For the --pgo option, any Python command line can be used, or 'default' to
+echo use the default task (-m test --pgo).
 echo.
 echo The following substitutions will be applied to the download URL:
 echo     Variable        Description         Example

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 20:52:00 2016
From: python-checkins at python.org (steve.dower)
Date: Tue, 15 Nov 2016 01:52:00 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NTcz?=
 =?utf-8?q?=3A_Fixes_issue_with_nested_if_blocks?=
Message-ID: <20161115015200.60852.37526.A3F3CFEB@psf.io>

https://hg.python.org/cpython/rev/d997e64130bd
changeset:   105118:d997e64130bd
branch:      3.6
parent:      105115:25cb7df5b19d
user:        Steve Dower <steve.dower at microsoft.com>
date:        Mon Nov 14 17:51:42 2016 -0800
summary:
  Issue #28573: Fixes issue with nested if blocks

files:
  Tools/msi/buildrelease.bat |  12 ++++++------
  1 files changed, 6 insertions(+), 6 deletions(-)


diff --git a/Tools/msi/buildrelease.bat b/Tools/msi/buildrelease.bat
--- a/Tools/msi/buildrelease.bat
+++ b/Tools/msi/buildrelease.bat
@@ -139,13 +139,13 @@
 ) else (
     set CERTOPTS=
 )
-
+if not "%PGO%" EQU "" (
+    set PGOOPTS=--pgo-job "%PGO%"
+) else (
+    set PGOOPTS=
+)
 if not "%SKIPBUILD%" EQU "1" (
-    if "%PGO%" EQU "" (
-        set PGOOPTS=
-    ) else (
-        set PGOOPTS=--pgo --pgojob "%PGO%"
-    )
+    @echo call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% %PGOOPTS%
     @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% %PGOOPTS%
     @if errorlevel 1 exit /B
     @rem build.bat turns echo back on, so we disable it again

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 14 20:52:00 2016
From: python-checkins at python.org (steve.dower)
Date: Tue, 15 Nov 2016 01:52:00 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328573=3A_Fixes_issue_with_nested_if_blocks?=
Message-ID: <20161115015200.4823.72434.16BD981F@psf.io>

https://hg.python.org/cpython/rev/35f510158490
changeset:   105119:35f510158490
parent:      105117:6f7a3a1a0a7d
parent:      105118:d997e64130bd
user:        Steve Dower <steve.dower at microsoft.com>
date:        Mon Nov 14 17:51:51 2016 -0800
summary:
  Issue #28573: Fixes issue with nested if blocks

files:
  Tools/msi/buildrelease.bat |  12 ++++++------
  1 files changed, 6 insertions(+), 6 deletions(-)


diff --git a/Tools/msi/buildrelease.bat b/Tools/msi/buildrelease.bat
--- a/Tools/msi/buildrelease.bat
+++ b/Tools/msi/buildrelease.bat
@@ -139,13 +139,13 @@
 ) else (
     set CERTOPTS=
 )
-
+if not "%PGO%" EQU "" (
+    set PGOOPTS=--pgo-job "%PGO%"
+) else (
+    set PGOOPTS=
+)
 if not "%SKIPBUILD%" EQU "1" (
-    if "%PGO%" EQU "" (
-        set PGOOPTS=
-    ) else (
-        set PGOOPTS=--pgo --pgojob "%PGO%"
-    )
+    @echo call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% %PGOOPTS%
     @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% %PGOOPTS%
     @if errorlevel 1 exit /B
     @rem build.bat turns echo back on, so we disable it again

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 03:12:59 2016
From: python-checkins at python.org (victor.stinner)
Date: Tue, 15 Nov 2016 08:12:59 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42?=
Message-ID: <20161115081259.61578.97029.2CC29268@psf.io>

https://hg.python.org/cpython/rev/ac93d188ebd6
changeset:   105121:ac93d188ebd6
parent:      105119:35f510158490
parent:      105120:aa52ea2a7731
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 15 09:12:36 2016 +0100
summary:
  Merge 3.6

files:
  Lib/test/test_string_literals.py |  20 ++++++++++++++++++++
  Python/ast.c                     |   8 +++++++-
  2 files changed, 27 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_string_literals.py b/Lib/test/test_string_literals.py
--- a/Lib/test/test_string_literals.py
+++ b/Lib/test/test_string_literals.py
@@ -111,6 +111,7 @@
                 continue
             with self.assertWarns(DeprecationWarning):
                 self.assertEqual(eval(r"'\%c'" % b), '\\' + chr(b))
+
         with warnings.catch_warnings(record=True) as w:
             warnings.simplefilter('always', category=DeprecationWarning)
             eval("'''\n\\z'''")
@@ -118,6 +119,15 @@
         self.assertEqual(w[0].filename, '<string>')
         self.assertEqual(w[0].lineno, 2)
 
+        with warnings.catch_warnings(record=True) as w:
+            warnings.simplefilter('error', category=DeprecationWarning)
+            with self.assertRaises(SyntaxError) as cm:
+                eval("'''\n\\z'''")
+            exc = cm.exception
+        self.assertEqual(w, [])
+        self.assertEqual(exc.filename, '<string>')
+        self.assertEqual(exc.lineno, 2)
+
     def test_eval_str_raw(self):
         self.assertEqual(eval(""" r'x' """), 'x')
         self.assertEqual(eval(r""" r'\x01' """), '\\' + 'x01')
@@ -150,6 +160,7 @@
                 continue
             with self.assertWarns(DeprecationWarning):
                 self.assertEqual(eval(r"b'\%c'" % b), b'\\' + bytes([b]))
+
         with warnings.catch_warnings(record=True) as w:
             warnings.simplefilter('always', category=DeprecationWarning)
             eval("b'''\n\\z'''")
@@ -157,6 +168,15 @@
         self.assertEqual(w[0].filename, '<string>')
         self.assertEqual(w[0].lineno, 2)
 
+        with warnings.catch_warnings(record=True) as w:
+            warnings.simplefilter('error', category=DeprecationWarning)
+            with self.assertRaises(SyntaxError) as cm:
+                eval("b'''\n\\z'''")
+            exc = cm.exception
+        self.assertEqual(w, [])
+        self.assertEqual(exc.filename, '<string>')
+        self.assertEqual(exc.lineno, 2)
+
     def test_eval_bytes_raw(self):
         self.assertEqual(eval(""" br'x' """), b'x')
         self.assertEqual(eval(""" rb'x' """), b'x')
diff --git a/Python/ast.c b/Python/ast.c
--- a/Python/ast.c
+++ b/Python/ast.c
@@ -4129,7 +4129,13 @@
                                    NULL, NULL) < 0 &&
         PyErr_ExceptionMatches(PyExc_DeprecationWarning))
     {
-        const char *s = PyUnicode_AsUTF8(msg);
+        const char *s;
+
+        /* Replace the DeprecationWarning exception with a SyntaxError
+           to get a more accurate error report */
+        PyErr_Clear();
+
+        s = PyUnicode_AsUTF8(msg);
         if (s != NULL) {
             ast_error(c, n, s);
         }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 03:13:02 2016
From: python-checkins at python.org (victor.stinner)
Date: Tue, 15 Nov 2016 08:13:02 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogRml4IHdhcm5faW52?=
 =?utf-8?q?alid=5Fescape=5Fsequence=28=29?=
Message-ID: <20161115081259.14262.46450.7E91A0F2@psf.io>

https://hg.python.org/cpython/rev/aa52ea2a7731
changeset:   105120:aa52ea2a7731
branch:      3.6
parent:      105118:d997e64130bd
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 15 09:12:10 2016 +0100
summary:
  Fix warn_invalid_escape_sequence()

Issue #28691: Fix warn_invalid_escape_sequence(): handle correctly
DeprecationWarning raised as an exception. First clear the current exception to
replace the DeprecationWarning exception with a SyntaxError exception.

Unit test written by Serhiy Storchaka.

files:
  Lib/test/test_string_literals.py |  20 ++++++++++++++++++++
  Python/ast.c                     |   8 +++++++-
  2 files changed, 27 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_string_literals.py b/Lib/test/test_string_literals.py
--- a/Lib/test/test_string_literals.py
+++ b/Lib/test/test_string_literals.py
@@ -111,6 +111,7 @@
                 continue
             with self.assertWarns(DeprecationWarning):
                 self.assertEqual(eval(r"'\%c'" % b), '\\' + chr(b))
+
         with warnings.catch_warnings(record=True) as w:
             warnings.simplefilter('always', category=DeprecationWarning)
             eval("'''\n\\z'''")
@@ -118,6 +119,15 @@
         self.assertEqual(w[0].filename, '<string>')
         self.assertEqual(w[0].lineno, 2)
 
+        with warnings.catch_warnings(record=True) as w:
+            warnings.simplefilter('error', category=DeprecationWarning)
+            with self.assertRaises(SyntaxError) as cm:
+                eval("'''\n\\z'''")
+            exc = cm.exception
+        self.assertEqual(w, [])
+        self.assertEqual(exc.filename, '<string>')
+        self.assertEqual(exc.lineno, 2)
+
     def test_eval_str_raw(self):
         self.assertEqual(eval(""" r'x' """), 'x')
         self.assertEqual(eval(r""" r'\x01' """), '\\' + 'x01')
@@ -150,6 +160,7 @@
                 continue
             with self.assertWarns(DeprecationWarning):
                 self.assertEqual(eval(r"b'\%c'" % b), b'\\' + bytes([b]))
+
         with warnings.catch_warnings(record=True) as w:
             warnings.simplefilter('always', category=DeprecationWarning)
             eval("b'''\n\\z'''")
@@ -157,6 +168,15 @@
         self.assertEqual(w[0].filename, '<string>')
         self.assertEqual(w[0].lineno, 2)
 
+        with warnings.catch_warnings(record=True) as w:
+            warnings.simplefilter('error', category=DeprecationWarning)
+            with self.assertRaises(SyntaxError) as cm:
+                eval("b'''\n\\z'''")
+            exc = cm.exception
+        self.assertEqual(w, [])
+        self.assertEqual(exc.filename, '<string>')
+        self.assertEqual(exc.lineno, 2)
+
     def test_eval_bytes_raw(self):
         self.assertEqual(eval(""" br'x' """), b'x')
         self.assertEqual(eval(""" rb'x' """), b'x')
diff --git a/Python/ast.c b/Python/ast.c
--- a/Python/ast.c
+++ b/Python/ast.c
@@ -4129,7 +4129,13 @@
                                    NULL, NULL) < 0 &&
         PyErr_ExceptionMatches(PyExc_DeprecationWarning))
     {
-        const char *s = PyUnicode_AsUTF8(msg);
+        const char *s;
+
+        /* Replace the DeprecationWarning exception with a SyntaxError
+           to get a more accurate error report */
+        PyErr_Clear();
+
+        s = PyUnicode_AsUTF8(msg);
         if (s != NULL) {
             ast_error(c, n, s);
         }

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Tue Nov 15 04:09:06 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Tue, 15 Nov 2016 09:09:06 +0000
Subject: [Python-checkins] Daily reference leaks (35f510158490): sum=-2
Message-ID: <20161115090906.8037.55279.74276FD0@psf.io>

results for 35f510158490 on branch "default"
--------------------------------------------

test_collections leaked [-7, 8, -7] memory blocks, sum=-6
test_functools leaked [0, 3, 1] memory blocks, sum=4


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/refloghBwvsR', '--timeout', '7200']

From python-checkins at python.org  Tue Nov 15 09:15:25 2016
From: python-checkins at python.org (victor.stinner)
Date: Tue, 15 Nov 2016 14:15:25 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328618=3A_Mark_dic?=
 =?utf-8?q?t_lookup_functions_as_hot?=
Message-ID: <20161115141525.13668.56163.72A5B54A@psf.io>

https://hg.python.org/cpython/rev/cfc956f13ce2
changeset:   105122:cfc956f13ce2
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 15 15:13:40 2016 +0100
summary:
  Issue #28618: Mark dict lookup functions as hot

It's common to see these functions in the top 3 of "perf report".

files:
  Include/pyport.h     |  4 ++--
  Objects/dictobject.c |  8 ++++----
  2 files changed, 6 insertions(+), 6 deletions(-)


diff --git a/Include/pyport.h b/Include/pyport.h
--- a/Include/pyport.h
+++ b/Include/pyport.h
@@ -498,7 +498,7 @@
 #endif
 
 
-/* Py_HOT_FUNCTION
+/* _Py_HOT_FUNCTION
  * The hot attribute on a function is used to inform the compiler that the
  * function is a hot spot of the compiled program. The function is optimized
  * more aggressively and on many target it is placed into special subsection of
@@ -506,7 +506,7 @@
  * locality.
  *
  * Usage:
- *    int Py_HOT_FUNCTION x() { return 3; }
+ *    int _Py_HOT_FUNCTION x() { return 3; }
  *
  * Issue #28618: This attribute must not be abused, otherwise it can have a
  * negative effect on performance. Only the functions were Python spend most of
diff --git a/Objects/dictobject.c b/Objects/dictobject.c
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -683,7 +683,7 @@
 For both, when the key isn't found a DKIX_EMPTY is returned. hashpos returns
 where the key index should be inserted.
 */
-static Py_ssize_t
+static Py_ssize_t _Py_HOT_FUNCTION
 lookdict(PyDictObject *mp, PyObject *key,
          Py_hash_t hash, PyObject ***value_addr, Py_ssize_t *hashpos)
 {
@@ -798,7 +798,7 @@
 }
 
 /* Specialized version for string-only keys */
-static Py_ssize_t
+static Py_ssize_t _Py_HOT_FUNCTION
 lookdict_unicode(PyDictObject *mp, PyObject *key,
                  Py_hash_t hash, PyObject ***value_addr, Py_ssize_t *hashpos)
 {
@@ -873,7 +873,7 @@
 
 /* Faster version of lookdict_unicode when it is known that no <dummy> keys
  * will be present. */
-static Py_ssize_t
+static Py_ssize_t _Py_HOT_FUNCTION
 lookdict_unicode_nodummy(PyDictObject *mp, PyObject *key,
                          Py_hash_t hash, PyObject ***value_addr,
                          Py_ssize_t *hashpos)
@@ -941,7 +941,7 @@
  * Split tables only contain unicode keys and no dummy keys,
  * so algorithm is the same as lookdict_unicode_nodummy.
  */
-static Py_ssize_t
+static Py_ssize_t _Py_HOT_FUNCTION
 lookdict_split(PyDictObject *mp, PyObject *key,
                Py_hash_t hash, PyObject ***value_addr, Py_ssize_t *hashpos)
 {

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Tue Nov 15 09:37:58 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Tue, 15 Nov 2016 14:37:58 +0000
Subject: [Python-checkins] UGLY Benchmark Results for Python 2.7 2016-11-15
Message-ID: <c65ced8c-94cf-46c7-bbc9-fb6399c23aaa@irsmsx102.ger.corp.intel.com>

Results for project Python 2.7, build date 2016-11-15 03:47:55 +0000
commit:		6ca91a14a555
previous commit:	828251c2bccf
revision date:	2016-11-14 17:25:44 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.17%         -1.04%          3.59%             8.59%
:-)             pybench      0.18%         -0.03%          5.75%             3.77%
:-(            regex_v8      0.57%         -0.07%         -2.20%            10.50%
:-)               nbody      0.10%          0.23%          8.42%            -0.86%
:-)        json_dump_v2      0.35%          2.14%          3.47%             9.82%
:-|      normal_startup      1.07%          0.21%         -0.38%             2.91%
:-)             ssbench      0.16%          0.31%          2.45%             1.66%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-2-7-2016-11-15/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Tue Nov 15 09:42:07 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Tue, 15 Nov 2016 14:42:07 +0000
Subject: [Python-checkins] BAD Benchmark Results for Python Default
 2016-11-15
Message-ID: <25d9dd76-780b-473a-abf4-d62fe939a02c@irsmsx102.ger.corp.intel.com>

Results for project Python default, build date 2016-11-15 03:02:00 +0000
commit:		35f510158490
previous commit:	817a003ecdc6
revision date:	2016-11-15 01:51:51 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.14%         -1.47%          3.43%            17.77%
:-)             pybench      0.92%         -0.23%          5.19%             4.96%
:-(            regex_v8      3.77%         -0.92%         -3.70%             3.16%
:-|               nbody      0.15%         -0.46%          1.42%             5.69%
:-(        json_dump_v2      0.30%         -0.23%         -4.40%            10.73%
:-|      normal_startup      1.03%          0.38%          0.44%             6.91%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-default-2016-11-15/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Tue Nov 15 11:06:53 2016
From: python-checkins at python.org (xavier.degaye)
Date: Tue, 15 Nov 2016 16:06:53 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_28668=3A_Merge_3=2E6?=
Message-ID: <20161115160653.5153.47386.F98AE1F6@psf.io>

https://hg.python.org/cpython/rev/1f0b0ecf7dc1
changeset:   105125:1f0b0ecf7dc1
parent:      105122:cfc956f13ce2
parent:      105124:e5404ba1b19e
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Tue Nov 15 17:04:53 2016 +0100
summary:
  Issue 28668: Merge 3.6

files:
  Lib/test/support/__init__.py |  18 +++++++++++++++++-
  Lib/test/test_logging.py     |   2 ++
  2 files changed, 19 insertions(+), 1 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -90,7 +90,7 @@
     "bigmemtest", "bigaddrspacetest", "cpython_only", "get_attribute",
     "requires_IEEE_754", "skip_unless_xattr", "requires_zlib",
     "anticipate_failure", "load_package_tests", "detect_api_mismatch",
-    "check__all__", "requires_android_level",
+    "check__all__", "requires_android_level", "requires_multiprocessing_queue",
     # sys
     "is_jython", "is_android", "check_impl_detail", "unix_shell",
     # network
@@ -1752,6 +1752,22 @@
         msg = msg.format(' or '.join(guardnames))
     return unittest.skip(msg)
 
+_have_mp_queue = None
+def requires_multiprocessing_queue(test):
+    """Skip decorator for tests that use multiprocessing.Queue."""
+    global _have_mp_queue
+    if _have_mp_queue is None:
+        import multiprocessing
+        # Without a functioning shared semaphore implementation attempts to
+        # instantiate a Queue will result in an ImportError (issue #3770).
+        try:
+            multiprocessing.Queue()
+            _have_mp_queue = True
+        except ImportError:
+            _have_mp_queue = False
+    msg = "requires a functioning shared semaphore implementation"
+    return test if _have_mp_queue else unittest.skip(msg)(test)
+
 def _parse_guards(guards):
     # Returns a tuple ({platform_name: run_me}, default_value)
     if not guards:
diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py
--- a/Lib/test/test_logging.py
+++ b/Lib/test/test_logging.py
@@ -3124,6 +3124,7 @@
             self.assertEqual(mock_handle.call_count, 5 * self.repeat,
                              'correct number of handled log messages')
 
+        @support.requires_multiprocessing_queue
         @patch.object(logging.handlers.QueueListener, 'handle')
         def test_handle_called_with_mp_queue(self, mock_handle):
             for i in range(self.repeat):
@@ -3140,6 +3141,7 @@
             except queue.Empty:
                 return []
 
+        @support.requires_multiprocessing_queue
         def test_no_messages_in_queue_after_stop(self):
             """
             Five messages are logged then the QueueListener is stopped. This

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 11:06:53 2016
From: python-checkins at python.org (xavier.degaye)
Date: Tue, 15 Nov 2016 16:06:53 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgMjg2Njg6?=
 =?utf-8?q?_Skip_tests_where_instanciation_of_multiprocessing=2EQueue?=
Message-ID: <20161115160652.4796.95919.7D4E581F@psf.io>

https://hg.python.org/cpython/rev/a377e6987821
changeset:   105123:a377e6987821
branch:      3.5
parent:      105110:6a2754055ff8
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Tue Nov 15 16:57:52 2016 +0100
summary:
  Issue 28668: Skip tests where instanciation of multiprocessing.Queue
would raise ImportError

files:
  Lib/test/support/__init__.py |  17 +++++++++++++++++
  Lib/test/test_logging.py     |   2 ++
  2 files changed, 19 insertions(+), 0 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -89,6 +89,7 @@
     "bigmemtest", "bigaddrspacetest", "cpython_only", "get_attribute",
     "requires_IEEE_754", "skip_unless_xattr", "requires_zlib",
     "anticipate_failure", "load_package_tests", "detect_api_mismatch",
+     "requires_multiprocessing_queue",
     # sys
     "is_jython", "check_impl_detail",
     # network
@@ -1731,6 +1732,22 @@
         msg = msg.format(' or '.join(guardnames))
     return unittest.skip(msg)
 
+_have_mp_queue = None
+def requires_multiprocessing_queue(test):
+    """Skip decorator for tests that use multiprocessing.Queue."""
+    global _have_mp_queue
+    if _have_mp_queue is None:
+        import multiprocessing
+        # Without a functioning shared semaphore implementation attempts to
+        # instantiate a Queue will result in an ImportError (issue #3770).
+        try:
+            multiprocessing.Queue()
+            _have_mp_queue = True
+        except ImportError:
+            _have_mp_queue = False
+    msg = "requires a functioning shared semaphore implementation"
+    return test if _have_mp_queue else unittest.skip(msg)(test)
+
 def _parse_guards(guards):
     # Returns a tuple ({platform_name: run_me}, default_value)
     if not guards:
diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py
--- a/Lib/test/test_logging.py
+++ b/Lib/test/test_logging.py
@@ -3066,6 +3066,7 @@
             self.assertEqual(mock_handle.call_count, 5 * self.repeat,
                              'correct number of handled log messages')
 
+        @support.requires_multiprocessing_queue
         @patch.object(logging.handlers.QueueListener, 'handle')
         def test_handle_called_with_mp_queue(self, mock_handle):
             for i in range(self.repeat):
@@ -3082,6 +3083,7 @@
             except queue.Empty:
                 return []
 
+        @support.requires_multiprocessing_queue
         def test_no_messages_in_queue_after_stop(self):
             """
             Five messages are logged then the QueueListener is stopped. This

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 11:10:25 2016
From: python-checkins at python.org (xavier.degaye)
Date: Tue, 15 Nov 2016 16:10:25 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_28668=3A_Merge_3=2E5?=
Message-ID: <20161115160653.25104.92197.10F82E13@psf.io>

https://hg.python.org/cpython/rev/e5404ba1b19e
changeset:   105124:e5404ba1b19e
branch:      3.6
parent:      105120:aa52ea2a7731
parent:      105123:a377e6987821
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Tue Nov 15 17:03:12 2016 +0100
summary:
  Issue 28668: Merge 3.5

files:
  Lib/test/support/__init__.py |  18 +++++++++++++++++-
  Lib/test/test_logging.py     |   2 ++
  2 files changed, 19 insertions(+), 1 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -90,7 +90,7 @@
     "bigmemtest", "bigaddrspacetest", "cpython_only", "get_attribute",
     "requires_IEEE_754", "skip_unless_xattr", "requires_zlib",
     "anticipate_failure", "load_package_tests", "detect_api_mismatch",
-    "check__all__", "requires_android_level",
+    "check__all__", "requires_android_level", "requires_multiprocessing_queue",
     # sys
     "is_jython", "is_android", "check_impl_detail", "unix_shell",
     # network
@@ -1752,6 +1752,22 @@
         msg = msg.format(' or '.join(guardnames))
     return unittest.skip(msg)
 
+_have_mp_queue = None
+def requires_multiprocessing_queue(test):
+    """Skip decorator for tests that use multiprocessing.Queue."""
+    global _have_mp_queue
+    if _have_mp_queue is None:
+        import multiprocessing
+        # Without a functioning shared semaphore implementation attempts to
+        # instantiate a Queue will result in an ImportError (issue #3770).
+        try:
+            multiprocessing.Queue()
+            _have_mp_queue = True
+        except ImportError:
+            _have_mp_queue = False
+    msg = "requires a functioning shared semaphore implementation"
+    return test if _have_mp_queue else unittest.skip(msg)(test)
+
 def _parse_guards(guards):
     # Returns a tuple ({platform_name: run_me}, default_value)
     if not guards:
diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py
--- a/Lib/test/test_logging.py
+++ b/Lib/test/test_logging.py
@@ -3124,6 +3124,7 @@
             self.assertEqual(mock_handle.call_count, 5 * self.repeat,
                              'correct number of handled log messages')
 
+        @support.requires_multiprocessing_queue
         @patch.object(logging.handlers.QueueListener, 'handle')
         def test_handle_called_with_mp_queue(self, mock_handle):
             for i in range(self.repeat):
@@ -3140,6 +3141,7 @@
             except queue.Empty:
                 return []
 
+        @support.requires_multiprocessing_queue
         def test_no_messages_in_queue_after_stop(self):
             """
             Five messages are logged then the QueueListener is stopped. This

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 11:26:46 2016
From: python-checkins at python.org (xavier.degaye)
Date: Tue, 15 Nov 2016 16:26:46 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2326929=3A_Merge_3=2E6?=
Message-ID: <20161115162646.59784.46584.1026E64B@psf.io>

https://hg.python.org/cpython/rev/91e0cf7f8e30
changeset:   105127:91e0cf7f8e30
parent:      105125:1f0b0ecf7dc1
parent:      105126:3c6e5f83d235
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Tue Nov 15 17:25:54 2016 +0100
summary:
  Issue #26929: Merge 3.6

files:
  Lib/test/test_strptime.py |  13 +++++++++++++
  1 files changed, 13 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_strptime.py b/Lib/test/test_strptime.py
--- a/Lib/test/test_strptime.py
+++ b/Lib/test/test_strptime.py
@@ -472,11 +472,24 @@
                         "Calculation of day of the week failed;"
                          "%s != %s" % (result.tm_wday, self.time_tuple.tm_wday))
 
+    if support.is_android:
+        # Issue #26929: strftime() on Android incorrectly formats %V or %G for
+        # the last or the first incomplete week in a year.
+        _ymd_excluded = ((1905, 1, 1), (1906, 12, 31), (2008, 12, 29),
+                        (1917, 12, 31))
+        _formats_excluded = ('%G %V',)
+    else:
+        _ymd_excluded = ()
+        _formats_excluded = ()
+
     def test_week_of_year_and_day_of_week_calculation(self):
         # Should be able to infer date if given year, week of year (%U or %W)
         # and day of the week
         def test_helper(ymd_tuple, test_reason):
             for year_week_format in ('%Y %W', '%Y %U', '%G %V'):
+                if (year_week_format in self._formats_excluded and
+                        ymd_tuple in self._ymd_excluded):
+                    return
                 for weekday_format in ('%w', '%u', '%a', '%A'):
                     format_string = year_week_format + ' ' + weekday_format
                     with self.subTest(test_reason,

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 11:26:49 2016
From: python-checkins at python.org (xavier.degaye)
Date: Tue, 15 Nov 2016 16:26:49 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI2OTI5?=
 =?utf-8?q?=3A_Skip_some_test=5Fstrptime_tests_failing_on_Android_that?=
Message-ID: <20161115162646.8037.30453.433BA56B@psf.io>

https://hg.python.org/cpython/rev/3c6e5f83d235
changeset:   105126:3c6e5f83d235
branch:      3.6
parent:      105124:e5404ba1b19e
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Tue Nov 15 17:24:42 2016 +0100
summary:
  Issue #26929: Skip some test_strptime tests failing on Android that
incorrectly formats %V or %G for the last or the first
incomplete week in a year

files:
  Lib/test/test_strptime.py |  13 +++++++++++++
  1 files changed, 13 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_strptime.py b/Lib/test/test_strptime.py
--- a/Lib/test/test_strptime.py
+++ b/Lib/test/test_strptime.py
@@ -472,11 +472,24 @@
                         "Calculation of day of the week failed;"
                          "%s != %s" % (result.tm_wday, self.time_tuple.tm_wday))
 
+    if support.is_android:
+        # Issue #26929: strftime() on Android incorrectly formats %V or %G for
+        # the last or the first incomplete week in a year.
+        _ymd_excluded = ((1905, 1, 1), (1906, 12, 31), (2008, 12, 29),
+                        (1917, 12, 31))
+        _formats_excluded = ('%G %V',)
+    else:
+        _ymd_excluded = ()
+        _formats_excluded = ()
+
     def test_week_of_year_and_day_of_week_calculation(self):
         # Should be able to infer date if given year, week of year (%U or %W)
         # and day of the week
         def test_helper(ymd_tuple, test_reason):
             for year_week_format in ('%Y %W', '%Y %U', '%G %V'):
+                if (year_week_format in self._formats_excluded and
+                        ymd_tuple in self._ymd_excluded):
+                    return
                 for weekday_format in ('%w', '%u', '%a', '%A'):
                     format_string = year_week_format + ' ' + weekday_format
                     with self.subTest(test_reason,

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 12:49:36 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Tue, 15 Nov 2016 17:49:36 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NTU2?=
 =?utf-8?q?=3A_Allow_keyword_syntax_for_NamedTuple_=28Ivan_Levkivskyi=29_?=
 =?utf-8?q?=28upstream?=
Message-ID: <20161115174936.5016.17784.929A6294@psf.io>

https://hg.python.org/cpython/rev/da2ac103d326
changeset:   105128:da2ac103d326
branch:      3.5
parent:      105123:a377e6987821
user:        Guido van Rossum <guido at python.org>
date:        Tue Nov 15 09:48:06 2016 -0800
summary:
  Issue #28556: Allow keyword syntax for NamedTuple (Ivan Levkivskyi) (upstream #321)

files:
  Lib/test/test_typing.py |   14 +++
  Lib/typing.py           |  100 ++++++++++++++-------------
  2 files changed, 65 insertions(+), 49 deletions(-)


diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -1865,6 +1865,20 @@
         self.assertEqual(CoolEmployee._fields, ('name', 'cool'))
         self.assertEqual(CoolEmployee._field_types, dict(name=str, cool=int))
 
+    @skipUnless(PY36, 'Python 3.6 required')
+    def test_namedtuple_keyword_usage(self):
+        LocalEmployee = NamedTuple("LocalEmployee", name=str, age=int)
+        nick = LocalEmployee('Nick', 25)
+        self.assertIsInstance(nick, tuple)
+        self.assertEqual(nick.name, 'Nick')
+        self.assertEqual(LocalEmployee.__name__, 'LocalEmployee')
+        self.assertEqual(LocalEmployee._fields, ('name', 'age'))
+        self.assertEqual(LocalEmployee._field_types, dict(name=str, age=int))
+        with self.assertRaises(TypeError):
+            NamedTuple('Name', [('x', int)], y=str)
+        with self.assertRaises(TypeError):
+            NamedTuple('Name', x=1, y='a')
+
     def test_pickle(self):
         global Emp  # pickle wants to reference the class by name
         Emp = NamedTuple('Emp', [('name', str), ('id', int)])
diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -1875,6 +1875,8 @@
 
 
 def _make_nmtuple(name, types):
+    msg = "NamedTuple('Name', [(f0, t0), (f1, t1), ...]); each t must be a type"
+    types = [(n, _type_check(t, msg)) for n, t in types]
     nm_tpl = collections.namedtuple(name, [n for n, t in types])
     nm_tpl._field_types = dict(types)
     try:
@@ -1884,55 +1886,55 @@
     return nm_tpl
 
 
-if sys.version_info[:2] >= (3, 6):
-    class NamedTupleMeta(type):
-
-        def __new__(cls, typename, bases, ns, *, _root=False):
-            if _root:
-                return super().__new__(cls, typename, bases, ns)
-            types = ns.get('__annotations__', {})
-            return _make_nmtuple(typename, types.items())
-
-    class NamedTuple(metaclass=NamedTupleMeta, _root=True):
-        """Typed version of namedtuple.
-
-        Usage::
-
-            class Employee(NamedTuple):
-                name: str
-                id: int
-
-        This is equivalent to::
-
-            Employee = collections.namedtuple('Employee', ['name', 'id'])
-
-        The resulting class has one extra attribute: _field_types,
-        giving a dict mapping field names to types.  (The field names
-        are in the _fields attribute, which is part of the namedtuple
-        API.) Backward-compatible usage::
-
-            Employee = NamedTuple('Employee', [('name', str), ('id', int)])
-        """
-
-        def __new__(self, typename, fields):
-            return _make_nmtuple(typename, fields)
-else:
-    def NamedTuple(typename, fields):
-        """Typed version of namedtuple.
-
-        Usage::
-
-            Employee = typing.NamedTuple('Employee', [('name', str), 'id', int)])
-
-        This is equivalent to::
-
-            Employee = collections.namedtuple('Employee', ['name', 'id'])
-
-        The resulting class has one extra attribute: _field_types,
-        giving a dict mapping field names to types.  (The field names
-        are in the _fields attribute, which is part of the namedtuple
-        API.)
-        """
+_PY36 = sys.version_info[:2] >= (3, 6)
+
+
+class NamedTupleMeta(type):
+
+    def __new__(cls, typename, bases, ns):
+        if ns.get('_root', False):
+            return super().__new__(cls, typename, bases, ns)
+        if not _PY36:
+            raise TypeError("Class syntax for NamedTuple is only supported"
+                            " in Python 3.6+")
+        types = ns.get('__annotations__', {})
+        return _make_nmtuple(typename, types.items())
+
+class NamedTuple(metaclass=NamedTupleMeta):
+    """Typed version of namedtuple.
+
+    Usage in Python versions >= 3.6::
+
+        class Employee(NamedTuple):
+            name: str
+            id: int
+
+    This is equivalent to::
+
+        Employee = collections.namedtuple('Employee', ['name', 'id'])
+
+    The resulting class has one extra attribute: _field_types,
+    giving a dict mapping field names to types.  (The field names
+    are in the _fields attribute, which is part of the namedtuple
+    API.) Alternative equivalent keyword syntax is also accepted::
+
+        Employee = NamedTuple('Employee', name=str, id=int)
+
+    In Python versions <= 3.5 use::
+
+        Employee = NamedTuple('Employee', [('name', str), ('id', int)])
+    """
+    _root = True
+
+    def __new__(self, typename, fields=None, **kwargs):
+        if kwargs and not _PY36:
+            raise TypeError("Keyword syntax for NamedTuple is only supported"
+                            " in Python 3.6+")
+        if fields is None:
+            fields = kwargs.items()
+        elif kwargs:
+            raise TypeError("Either list of fields or keywords"
+                            " can be provided to NamedTuple, not both")
         return _make_nmtuple(typename, fields)
 
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 12:49:36 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Tue, 15 Nov 2016 17:49:36 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328556=3A_Allow_keyword_syntax_for_NamedTuple_?=
 =?utf-8?q?=28Ivan_Levkivskyi=29_=28upstream?=
Message-ID: <20161115174936.5697.5299.E60A892B@psf.io>

https://hg.python.org/cpython/rev/a3de2d0f49ea
changeset:   105130:a3de2d0f49ea
parent:      105127:91e0cf7f8e30
parent:      105129:38ec88a4e282
user:        Guido van Rossum <guido at python.org>
date:        Tue Nov 15 09:48:12 2016 -0800
summary:
  Issue #28556: Allow keyword syntax for NamedTuple (Ivan Levkivskyi) (upstream #321) (3.6->3.7)

files:
  Lib/test/test_typing.py |   14 +++
  Lib/typing.py           |  100 ++++++++++++++-------------
  2 files changed, 65 insertions(+), 49 deletions(-)


diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -1865,6 +1865,20 @@
         self.assertEqual(CoolEmployee._fields, ('name', 'cool'))
         self.assertEqual(CoolEmployee._field_types, dict(name=str, cool=int))
 
+    @skipUnless(PY36, 'Python 3.6 required')
+    def test_namedtuple_keyword_usage(self):
+        LocalEmployee = NamedTuple("LocalEmployee", name=str, age=int)
+        nick = LocalEmployee('Nick', 25)
+        self.assertIsInstance(nick, tuple)
+        self.assertEqual(nick.name, 'Nick')
+        self.assertEqual(LocalEmployee.__name__, 'LocalEmployee')
+        self.assertEqual(LocalEmployee._fields, ('name', 'age'))
+        self.assertEqual(LocalEmployee._field_types, dict(name=str, age=int))
+        with self.assertRaises(TypeError):
+            NamedTuple('Name', [('x', int)], y=str)
+        with self.assertRaises(TypeError):
+            NamedTuple('Name', x=1, y='a')
+
     def test_pickle(self):
         global Emp  # pickle wants to reference the class by name
         Emp = NamedTuple('Emp', [('name', str), ('id', int)])
diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -1875,6 +1875,8 @@
 
 
 def _make_nmtuple(name, types):
+    msg = "NamedTuple('Name', [(f0, t0), (f1, t1), ...]); each t must be a type"
+    types = [(n, _type_check(t, msg)) for n, t in types]
     nm_tpl = collections.namedtuple(name, [n for n, t in types])
     nm_tpl._field_types = dict(types)
     try:
@@ -1884,55 +1886,55 @@
     return nm_tpl
 
 
-if sys.version_info[:2] >= (3, 6):
-    class NamedTupleMeta(type):
-
-        def __new__(cls, typename, bases, ns, *, _root=False):
-            if _root:
-                return super().__new__(cls, typename, bases, ns)
-            types = ns.get('__annotations__', {})
-            return _make_nmtuple(typename, types.items())
-
-    class NamedTuple(metaclass=NamedTupleMeta, _root=True):
-        """Typed version of namedtuple.
-
-        Usage::
-
-            class Employee(NamedTuple):
-                name: str
-                id: int
-
-        This is equivalent to::
-
-            Employee = collections.namedtuple('Employee', ['name', 'id'])
-
-        The resulting class has one extra attribute: _field_types,
-        giving a dict mapping field names to types.  (The field names
-        are in the _fields attribute, which is part of the namedtuple
-        API.) Backward-compatible usage::
-
-            Employee = NamedTuple('Employee', [('name', str), ('id', int)])
-        """
-
-        def __new__(self, typename, fields):
-            return _make_nmtuple(typename, fields)
-else:
-    def NamedTuple(typename, fields):
-        """Typed version of namedtuple.
-
-        Usage::
-
-            Employee = typing.NamedTuple('Employee', [('name', str), 'id', int)])
-
-        This is equivalent to::
-
-            Employee = collections.namedtuple('Employee', ['name', 'id'])
-
-        The resulting class has one extra attribute: _field_types,
-        giving a dict mapping field names to types.  (The field names
-        are in the _fields attribute, which is part of the namedtuple
-        API.)
-        """
+_PY36 = sys.version_info[:2] >= (3, 6)
+
+
+class NamedTupleMeta(type):
+
+    def __new__(cls, typename, bases, ns):
+        if ns.get('_root', False):
+            return super().__new__(cls, typename, bases, ns)
+        if not _PY36:
+            raise TypeError("Class syntax for NamedTuple is only supported"
+                            " in Python 3.6+")
+        types = ns.get('__annotations__', {})
+        return _make_nmtuple(typename, types.items())
+
+class NamedTuple(metaclass=NamedTupleMeta):
+    """Typed version of namedtuple.
+
+    Usage in Python versions >= 3.6::
+
+        class Employee(NamedTuple):
+            name: str
+            id: int
+
+    This is equivalent to::
+
+        Employee = collections.namedtuple('Employee', ['name', 'id'])
+
+    The resulting class has one extra attribute: _field_types,
+    giving a dict mapping field names to types.  (The field names
+    are in the _fields attribute, which is part of the namedtuple
+    API.) Alternative equivalent keyword syntax is also accepted::
+
+        Employee = NamedTuple('Employee', name=str, id=int)
+
+    In Python versions <= 3.5 use::
+
+        Employee = NamedTuple('Employee', [('name', str), ('id', int)])
+    """
+    _root = True
+
+    def __new__(self, typename, fields=None, **kwargs):
+        if kwargs and not _PY36:
+            raise TypeError("Keyword syntax for NamedTuple is only supported"
+                            " in Python 3.6+")
+        if fields is None:
+            fields = kwargs.items()
+        elif kwargs:
+            raise TypeError("Either list of fields or keywords"
+                            " can be provided to NamedTuple, not both")
         return _make_nmtuple(typename, fields)
 
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 12:49:36 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Tue, 15 Nov 2016 17:49:36 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328556=3A_Allow_keyword_syntax_for_NamedTuple_=28Ivan_?=
 =?utf-8?q?Levkivskyi=29_=28upstream?=
Message-ID: <20161115174936.5697.81546.556C2EF9@psf.io>

https://hg.python.org/cpython/rev/38ec88a4e282
changeset:   105129:38ec88a4e282
branch:      3.6
parent:      105126:3c6e5f83d235
parent:      105128:da2ac103d326
user:        Guido van Rossum <guido at python.org>
date:        Tue Nov 15 09:48:09 2016 -0800
summary:
  Issue #28556: Allow keyword syntax for NamedTuple (Ivan Levkivskyi) (upstream #321) (3.5->3.6)

files:
  Lib/test/test_typing.py |   14 +++
  Lib/typing.py           |  100 ++++++++++++++-------------
  2 files changed, 65 insertions(+), 49 deletions(-)


diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -1865,6 +1865,20 @@
         self.assertEqual(CoolEmployee._fields, ('name', 'cool'))
         self.assertEqual(CoolEmployee._field_types, dict(name=str, cool=int))
 
+    @skipUnless(PY36, 'Python 3.6 required')
+    def test_namedtuple_keyword_usage(self):
+        LocalEmployee = NamedTuple("LocalEmployee", name=str, age=int)
+        nick = LocalEmployee('Nick', 25)
+        self.assertIsInstance(nick, tuple)
+        self.assertEqual(nick.name, 'Nick')
+        self.assertEqual(LocalEmployee.__name__, 'LocalEmployee')
+        self.assertEqual(LocalEmployee._fields, ('name', 'age'))
+        self.assertEqual(LocalEmployee._field_types, dict(name=str, age=int))
+        with self.assertRaises(TypeError):
+            NamedTuple('Name', [('x', int)], y=str)
+        with self.assertRaises(TypeError):
+            NamedTuple('Name', x=1, y='a')
+
     def test_pickle(self):
         global Emp  # pickle wants to reference the class by name
         Emp = NamedTuple('Emp', [('name', str), ('id', int)])
diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -1875,6 +1875,8 @@
 
 
 def _make_nmtuple(name, types):
+    msg = "NamedTuple('Name', [(f0, t0), (f1, t1), ...]); each t must be a type"
+    types = [(n, _type_check(t, msg)) for n, t in types]
     nm_tpl = collections.namedtuple(name, [n for n, t in types])
     nm_tpl._field_types = dict(types)
     try:
@@ -1884,55 +1886,55 @@
     return nm_tpl
 
 
-if sys.version_info[:2] >= (3, 6):
-    class NamedTupleMeta(type):
-
-        def __new__(cls, typename, bases, ns, *, _root=False):
-            if _root:
-                return super().__new__(cls, typename, bases, ns)
-            types = ns.get('__annotations__', {})
-            return _make_nmtuple(typename, types.items())
-
-    class NamedTuple(metaclass=NamedTupleMeta, _root=True):
-        """Typed version of namedtuple.
-
-        Usage::
-
-            class Employee(NamedTuple):
-                name: str
-                id: int
-
-        This is equivalent to::
-
-            Employee = collections.namedtuple('Employee', ['name', 'id'])
-
-        The resulting class has one extra attribute: _field_types,
-        giving a dict mapping field names to types.  (The field names
-        are in the _fields attribute, which is part of the namedtuple
-        API.) Backward-compatible usage::
-
-            Employee = NamedTuple('Employee', [('name', str), ('id', int)])
-        """
-
-        def __new__(self, typename, fields):
-            return _make_nmtuple(typename, fields)
-else:
-    def NamedTuple(typename, fields):
-        """Typed version of namedtuple.
-
-        Usage::
-
-            Employee = typing.NamedTuple('Employee', [('name', str), 'id', int)])
-
-        This is equivalent to::
-
-            Employee = collections.namedtuple('Employee', ['name', 'id'])
-
-        The resulting class has one extra attribute: _field_types,
-        giving a dict mapping field names to types.  (The field names
-        are in the _fields attribute, which is part of the namedtuple
-        API.)
-        """
+_PY36 = sys.version_info[:2] >= (3, 6)
+
+
+class NamedTupleMeta(type):
+
+    def __new__(cls, typename, bases, ns):
+        if ns.get('_root', False):
+            return super().__new__(cls, typename, bases, ns)
+        if not _PY36:
+            raise TypeError("Class syntax for NamedTuple is only supported"
+                            " in Python 3.6+")
+        types = ns.get('__annotations__', {})
+        return _make_nmtuple(typename, types.items())
+
+class NamedTuple(metaclass=NamedTupleMeta):
+    """Typed version of namedtuple.
+
+    Usage in Python versions >= 3.6::
+
+        class Employee(NamedTuple):
+            name: str
+            id: int
+
+    This is equivalent to::
+
+        Employee = collections.namedtuple('Employee', ['name', 'id'])
+
+    The resulting class has one extra attribute: _field_types,
+    giving a dict mapping field names to types.  (The field names
+    are in the _fields attribute, which is part of the namedtuple
+    API.) Alternative equivalent keyword syntax is also accepted::
+
+        Employee = NamedTuple('Employee', name=str, id=int)
+
+    In Python versions <= 3.5 use::
+
+        Employee = NamedTuple('Employee', [('name', str), ('id', int)])
+    """
+    _root = True
+
+    def __new__(self, typename, fields=None, **kwargs):
+        if kwargs and not _PY36:
+            raise TypeError("Keyword syntax for NamedTuple is only supported"
+                            " in Python 3.6+")
+        if fields is None:
+            fields = kwargs.items()
+        elif kwargs:
+            raise TypeError("Either list of fields or keywords"
+                            " can be provided to NamedTuple, not both")
         return _make_nmtuple(typename, fields)
 
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 15:21:40 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 15 Nov 2016 20:21:40 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NzAz?=
 =?utf-8?q?=3A_Fix_asyncio=2Eiscoroutinefunction_to_handle_Mock_objects=2E?=
Message-ID: <20161115202139.7876.41553.BC832AC9@psf.io>

https://hg.python.org/cpython/rev/179e556a50ce
changeset:   105131:179e556a50ce
branch:      3.5
parent:      105128:da2ac103d326
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 15 15:20:34 2016 -0500
summary:
  Issue #28703: Fix asyncio.iscoroutinefunction to handle Mock objects.

files:
  Lib/asyncio/coroutines.py           |  16 ++++++++++++++--
  Lib/test/test_asyncio/test_tasks.py |   2 ++
  Misc/NEWS                           |   2 ++
  3 files changed, 18 insertions(+), 2 deletions(-)


diff --git a/Lib/asyncio/coroutines.py b/Lib/asyncio/coroutines.py
--- a/Lib/asyncio/coroutines.py
+++ b/Lib/asyncio/coroutines.py
@@ -33,12 +33,16 @@
 
 try:
     _types_coroutine = types.coroutine
+    _types_CoroutineType = types.CoroutineType
 except AttributeError:
+    # Python 3.4
     _types_coroutine = None
+    _types_CoroutineType = None
 
 try:
     _inspect_iscoroutinefunction = inspect.iscoroutinefunction
 except AttributeError:
+    # Python 3.4
     _inspect_iscoroutinefunction = lambda func: False
 
 try:
@@ -238,19 +242,27 @@
             w.__qualname__ = getattr(func, '__qualname__', None)
             return w
 
-    wrapper._is_coroutine = True  # For iscoroutinefunction().
+    wrapper._is_coroutine = _is_coroutine  # For iscoroutinefunction().
     return wrapper
 
 
+# A marker for iscoroutinefunction.
+_is_coroutine = object()
+
+
 def iscoroutinefunction(func):
     """Return True if func is a decorated coroutine function."""
-    return (getattr(func, '_is_coroutine', False) or
+    return (getattr(func, '_is_coroutine', None) is _is_coroutine or
             _inspect_iscoroutinefunction(func))
 
 
 _COROUTINE_TYPES = (types.GeneratorType, CoroWrapper)
 if _CoroutineABC is not None:
     _COROUTINE_TYPES += (_CoroutineABC,)
+if _types_CoroutineType is not None:
+    # Prioritize native coroutine check to speed-up
+    # asyncio.iscoroutine.
+    _COROUTINE_TYPES = (_types_CoroutineType,) + _COROUTINE_TYPES
 
 
 def iscoroutine(obj):
diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py
--- a/Lib/test/test_asyncio/test_tasks.py
+++ b/Lib/test/test_asyncio/test_tasks.py
@@ -1376,6 +1376,8 @@
             yield
         self.assertTrue(asyncio.iscoroutinefunction(fn2))
 
+        self.assertFalse(asyncio.iscoroutinefunction(mock.Mock()))
+
     def test_yield_vs_yield_from(self):
         fut = asyncio.Future(loop=self.loop)
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -465,6 +465,8 @@
 
 - Issue #28653: Fix a refleak in functools.lru_cache.
 
+- Issue #28703: Fix asyncio.iscoroutinefunction to handle Mock objects.
+
 IDLE
 ----
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 15:21:40 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 15 Nov 2016 20:21:40 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg3MDMp?=
Message-ID: <20161115202140.61578.14122.69601634@psf.io>

https://hg.python.org/cpython/rev/298c53e2461f
changeset:   105133:298c53e2461f
parent:      105130:a3de2d0f49ea
parent:      105132:4d78290b1d8e
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 15 15:21:34 2016 -0500
summary:
  Merge 3.6 (issue #28703)

files:
  Lib/asyncio/coroutines.py           |  16 ++++++++++++++--
  Lib/test/test_asyncio/test_tasks.py |   2 ++
  2 files changed, 16 insertions(+), 2 deletions(-)


diff --git a/Lib/asyncio/coroutines.py b/Lib/asyncio/coroutines.py
--- a/Lib/asyncio/coroutines.py
+++ b/Lib/asyncio/coroutines.py
@@ -33,12 +33,16 @@
 
 try:
     _types_coroutine = types.coroutine
+    _types_CoroutineType = types.CoroutineType
 except AttributeError:
+    # Python 3.4
     _types_coroutine = None
+    _types_CoroutineType = None
 
 try:
     _inspect_iscoroutinefunction = inspect.iscoroutinefunction
 except AttributeError:
+    # Python 3.4
     _inspect_iscoroutinefunction = lambda func: False
 
 try:
@@ -238,19 +242,27 @@
             w.__qualname__ = getattr(func, '__qualname__', None)
             return w
 
-    wrapper._is_coroutine = True  # For iscoroutinefunction().
+    wrapper._is_coroutine = _is_coroutine  # For iscoroutinefunction().
     return wrapper
 
 
+# A marker for iscoroutinefunction.
+_is_coroutine = object()
+
+
 def iscoroutinefunction(func):
     """Return True if func is a decorated coroutine function."""
-    return (getattr(func, '_is_coroutine', False) or
+    return (getattr(func, '_is_coroutine', None) is _is_coroutine or
             _inspect_iscoroutinefunction(func))
 
 
 _COROUTINE_TYPES = (types.GeneratorType, CoroWrapper)
 if _CoroutineABC is not None:
     _COROUTINE_TYPES += (_CoroutineABC,)
+if _types_CoroutineType is not None:
+    # Prioritize native coroutine check to speed-up
+    # asyncio.iscoroutine.
+    _COROUTINE_TYPES = (_types_CoroutineType,) + _COROUTINE_TYPES
 
 
 def iscoroutine(obj):
diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py
--- a/Lib/test/test_asyncio/test_tasks.py
+++ b/Lib/test/test_asyncio/test_tasks.py
@@ -1390,6 +1390,8 @@
             yield
         self.assertTrue(asyncio.iscoroutinefunction(fn2))
 
+        self.assertFalse(asyncio.iscoroutinefunction(mock.Mock()))
+
     def test_yield_vs_yield_from(self):
         fut = self.new_future(self.loop)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 15:21:40 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 15 Nov 2016 20:21:40 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5_=28issue_=2328703=29?=
Message-ID: <20161115202139.5697.55193.35A5F52A@psf.io>

https://hg.python.org/cpython/rev/4d78290b1d8e
changeset:   105132:4d78290b1d8e
branch:      3.6
parent:      105129:38ec88a4e282
parent:      105131:179e556a50ce
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 15 15:21:07 2016 -0500
summary:
  Merge 3.5 (issue #28703)

files:
  Lib/asyncio/coroutines.py           |  16 ++++++++++++++--
  Lib/test/test_asyncio/test_tasks.py |   2 ++
  Misc/NEWS                           |   2 ++
  3 files changed, 18 insertions(+), 2 deletions(-)


diff --git a/Lib/asyncio/coroutines.py b/Lib/asyncio/coroutines.py
--- a/Lib/asyncio/coroutines.py
+++ b/Lib/asyncio/coroutines.py
@@ -33,12 +33,16 @@
 
 try:
     _types_coroutine = types.coroutine
+    _types_CoroutineType = types.CoroutineType
 except AttributeError:
+    # Python 3.4
     _types_coroutine = None
+    _types_CoroutineType = None
 
 try:
     _inspect_iscoroutinefunction = inspect.iscoroutinefunction
 except AttributeError:
+    # Python 3.4
     _inspect_iscoroutinefunction = lambda func: False
 
 try:
@@ -238,19 +242,27 @@
             w.__qualname__ = getattr(func, '__qualname__', None)
             return w
 
-    wrapper._is_coroutine = True  # For iscoroutinefunction().
+    wrapper._is_coroutine = _is_coroutine  # For iscoroutinefunction().
     return wrapper
 
 
+# A marker for iscoroutinefunction.
+_is_coroutine = object()
+
+
 def iscoroutinefunction(func):
     """Return True if func is a decorated coroutine function."""
-    return (getattr(func, '_is_coroutine', False) or
+    return (getattr(func, '_is_coroutine', None) is _is_coroutine or
             _inspect_iscoroutinefunction(func))
 
 
 _COROUTINE_TYPES = (types.GeneratorType, CoroWrapper)
 if _CoroutineABC is not None:
     _COROUTINE_TYPES += (_CoroutineABC,)
+if _types_CoroutineType is not None:
+    # Prioritize native coroutine check to speed-up
+    # asyncio.iscoroutine.
+    _COROUTINE_TYPES = (_types_CoroutineType,) + _COROUTINE_TYPES
 
 
 def iscoroutine(obj):
diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py
--- a/Lib/test/test_asyncio/test_tasks.py
+++ b/Lib/test/test_asyncio/test_tasks.py
@@ -1390,6 +1390,8 @@
             yield
         self.assertTrue(asyncio.iscoroutinefunction(fn2))
 
+        self.assertFalse(asyncio.iscoroutinefunction(mock.Mock()))
+
     def test_yield_vs_yield_from(self):
         fut = self.new_future(self.loop)
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -58,6 +58,8 @@
 
 - Issue #28653: Fix a refleak in functools.lru_cache.
 
+- Issue #28703: Fix asyncio.iscoroutinefunction to handle Mock objects.
+
 Documentation
 -------------
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 15:27:53 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 15 Nov 2016 20:27:53 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NzA0?=
 =?utf-8?q?=3A_Fix_create=5Funix=5Fserver_to_support_Path-like_objects?=
Message-ID: <20161115202753.12078.54013.731FA33E@psf.io>

https://hg.python.org/cpython/rev/f8207c98eb5e
changeset:   105134:f8207c98eb5e
branch:      3.5
parent:      105131:179e556a50ce
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 15 15:26:34 2016 -0500
summary:
  Issue #28704: Fix create_unix_server to support Path-like objects

files:
  Lib/asyncio/unix_events.py                |   8 ++++++++
  Lib/test/test_asyncio/test_unix_events.py |  10 ++++++++++
  2 files changed, 18 insertions(+), 0 deletions(-)


diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py
--- a/Lib/asyncio/unix_events.py
+++ b/Lib/asyncio/unix_events.py
@@ -39,6 +39,13 @@
     pass
 
 
+try:
+    _fspath = os.fspath
+except AttributeError:
+    # Python 3.5 or earlier
+    _fspath = lambda path: path
+
+
 class _UnixSelectorEventLoop(selector_events.BaseSelectorEventLoop):
     """Unix event loop.
 
@@ -256,6 +263,7 @@
                 raise ValueError(
                     'path and sock can not be specified at the same time')
 
+            path = _fspath(path)
             sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
 
             # Check for abstract socket. `str` and `bytes` paths are supported.
diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py
--- a/Lib/test/test_asyncio/test_unix_events.py
+++ b/Lib/test/test_asyncio/test_unix_events.py
@@ -4,6 +4,7 @@
 import errno
 import io
 import os
+import pathlib
 import signal
 import socket
 import stat
@@ -251,6 +252,15 @@
             srv.close()
             self.loop.run_until_complete(srv.wait_closed())
 
+    @unittest.skipUnless(hasattr(os, 'fspath'), 'no os.fspath')
+    def test_create_unix_server_pathlib(self):
+        with test_utils.unix_socket_path() as path:
+            path = pathlib.Path(path)
+            srv_coro = self.loop.create_unix_server(lambda: None, path)
+            srv = self.loop.run_until_complete(srv_coro)
+            srv.close()
+            self.loop.run_until_complete(srv.wait_closed())
+
     def test_create_unix_server_existing_path_nonsock(self):
         with tempfile.NamedTemporaryFile() as file:
             coro = self.loop.create_unix_server(lambda: None, file.name)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 15:27:53 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 15 Nov 2016 20:27:53 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg3MDQp?=
Message-ID: <20161115202753.61259.49119.39084088@psf.io>

https://hg.python.org/cpython/rev/c7d2ec49a80b
changeset:   105136:c7d2ec49a80b
parent:      105133:298c53e2461f
parent:      105135:0d663f758adb
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 15 15:27:48 2016 -0500
summary:
  Merge 3.6 (issue #28704)

files:
  Lib/asyncio/unix_events.py                |   8 ++++++++
  Lib/test/test_asyncio/test_unix_events.py |  10 ++++++++++
  2 files changed, 18 insertions(+), 0 deletions(-)


diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py
--- a/Lib/asyncio/unix_events.py
+++ b/Lib/asyncio/unix_events.py
@@ -39,6 +39,13 @@
     pass
 
 
+try:
+    _fspath = os.fspath
+except AttributeError:
+    # Python 3.5 or earlier
+    _fspath = lambda path: path
+
+
 class _UnixSelectorEventLoop(selector_events.BaseSelectorEventLoop):
     """Unix event loop.
 
@@ -256,6 +263,7 @@
                 raise ValueError(
                     'path and sock can not be specified at the same time')
 
+            path = _fspath(path)
             sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
 
             # Check for abstract socket. `str` and `bytes` paths are supported.
diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py
--- a/Lib/test/test_asyncio/test_unix_events.py
+++ b/Lib/test/test_asyncio/test_unix_events.py
@@ -4,6 +4,7 @@
 import errno
 import io
 import os
+import pathlib
 import signal
 import socket
 import stat
@@ -251,6 +252,15 @@
             srv.close()
             self.loop.run_until_complete(srv.wait_closed())
 
+    @unittest.skipUnless(hasattr(os, 'fspath'), 'no os.fspath')
+    def test_create_unix_server_pathlib(self):
+        with test_utils.unix_socket_path() as path:
+            path = pathlib.Path(path)
+            srv_coro = self.loop.create_unix_server(lambda: None, path)
+            srv = self.loop.run_until_complete(srv_coro)
+            srv.close()
+            self.loop.run_until_complete(srv.wait_closed())
+
     def test_create_unix_server_existing_path_nonsock(self):
         with tempfile.NamedTemporaryFile() as file:
             coro = self.loop.create_unix_server(lambda: None, file.name)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 15:27:53 2016
From: python-checkins at python.org (yury.selivanov)
Date: Tue, 15 Nov 2016 20:27:53 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5_=28issue_=2328704=29?=
Message-ID: <20161115202753.60162.83772.29539C60@psf.io>

https://hg.python.org/cpython/rev/0d663f758adb
changeset:   105135:0d663f758adb
branch:      3.6
parent:      105132:4d78290b1d8e
parent:      105134:f8207c98eb5e
user:        Yury Selivanov <yury at magic.io>
date:        Tue Nov 15 15:27:23 2016 -0500
summary:
  Merge 3.5 (issue #28704)

files:
  Lib/asyncio/unix_events.py                |   8 ++++++++
  Lib/test/test_asyncio/test_unix_events.py |  10 ++++++++++
  Misc/NEWS                                 |   3 +++
  3 files changed, 21 insertions(+), 0 deletions(-)


diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py
--- a/Lib/asyncio/unix_events.py
+++ b/Lib/asyncio/unix_events.py
@@ -39,6 +39,13 @@
     pass
 
 
+try:
+    _fspath = os.fspath
+except AttributeError:
+    # Python 3.5 or earlier
+    _fspath = lambda path: path
+
+
 class _UnixSelectorEventLoop(selector_events.BaseSelectorEventLoop):
     """Unix event loop.
 
@@ -256,6 +263,7 @@
                 raise ValueError(
                     'path and sock can not be specified at the same time')
 
+            path = _fspath(path)
             sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
 
             # Check for abstract socket. `str` and `bytes` paths are supported.
diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py
--- a/Lib/test/test_asyncio/test_unix_events.py
+++ b/Lib/test/test_asyncio/test_unix_events.py
@@ -4,6 +4,7 @@
 import errno
 import io
 import os
+import pathlib
 import signal
 import socket
 import stat
@@ -251,6 +252,15 @@
             srv.close()
             self.loop.run_until_complete(srv.wait_closed())
 
+    @unittest.skipUnless(hasattr(os, 'fspath'), 'no os.fspath')
+    def test_create_unix_server_pathlib(self):
+        with test_utils.unix_socket_path() as path:
+            path = pathlib.Path(path)
+            srv_coro = self.loop.create_unix_server(lambda: None, path)
+            srv = self.loop.run_until_complete(srv_coro)
+            srv.close()
+            self.loop.run_until_complete(srv.wait_closed())
+
     def test_create_unix_server_existing_path_nonsock(self):
         with tempfile.NamedTemporaryFile() as file:
             coro = self.loop.create_unix_server(lambda: None, file.name)
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -60,6 +60,9 @@
 
 - Issue #28703: Fix asyncio.iscoroutinefunction to handle Mock objects.
 
+- Issue #28704: Fix create_unix_server to support Path-like objects 
+  (PEP 519).
+
 Documentation
 -------------
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 21:19:01 2016
From: python-checkins at python.org (donald.stufft)
Date: Wed, 16 Nov 2016 02:19:01 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Upgrade_pip_to?=
 =?utf-8?q?_9=2E0=2E1_and_setuptools_to_28=2E8=2E0?=
Message-ID: <20161116021901.89163.22802.C80FD044@psf.io>

https://hg.python.org/cpython/rev/80d4da2e78ec
changeset:   105137:80d4da2e78ec
branch:      3.4
parent:      105109:f78a05cda5aa
user:        Donald Stufft <donald at stufft.io>
date:        Tue Nov 15 21:17:43 2016 -0500
summary:
  Upgrade pip to 9.0.1 and setuptools to 28.8.0

files:
  Lib/ensurepip/__init__.py                                     |    4 ++--
  Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/pip-9.0.1-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl |  Bin 
  Lib/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl |  Bin 
  5 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -8,9 +8,9 @@
 __all__ = ["version", "bootstrap"]
 
 
-_SETUPTOOLS_VERSION = "28.7.1"
+_SETUPTOOLS_VERSION = "28.8.0"
 
-_PIP_VERSION = "9.0.0"
+_PIP_VERSION = "9.0.1"
 
 # pip currently requires ssl support, so we try to provide a nicer
 # error message when that is missing (http://bugs.python.org/issue19744)
diff --git a/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl
deleted file mode 100644
index 2c631a846c10cf51d32241263aff9c38a5bd8e27..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/pip-9.0.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-9.0.1-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..4b8ecc69db7e37fc6dd7b6dd8f690508f42866a1
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl
deleted file mode 100644
index cbcc2200f46eb05d707ad7c2c581ac36e561c256..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..502e3cb418c154872ad6e677ef8b63557b38ec35
GIT binary patch
[stripped]

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 21:19:01 2016
From: python-checkins at python.org (donald.stufft)
Date: Wed, 16 Nov 2016 02:19:01 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?=
 =?utf-8?q?_Merge_-_upgrade_pip/setuptools?=
Message-ID: <20161116021901.6853.32532.DF0B2CFE@psf.io>

https://hg.python.org/cpython/rev/e824e6113a04
changeset:   105138:e824e6113a04
branch:      3.5
parent:      105134:f8207c98eb5e
parent:      105137:80d4da2e78ec
user:        Donald Stufft <donald at stufft.io>
date:        Tue Nov 15 21:18:20 2016 -0500
summary:
  Merge - upgrade pip/setuptools

files:
  Lib/ensurepip/__init__.py                                     |    4 ++--
  Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/pip-9.0.1-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl |  Bin 
  Lib/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl |  Bin 
  5 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -8,9 +8,9 @@
 __all__ = ["version", "bootstrap"]
 
 
-_SETUPTOOLS_VERSION = "28.7.1"
+_SETUPTOOLS_VERSION = "28.8.0"
 
-_PIP_VERSION = "9.0.0"
+_PIP_VERSION = "9.0.1"
 
 # pip currently requires ssl support, so we try to provide a nicer
 # error message when that is missing (http://bugs.python.org/issue19744)
diff --git a/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl
deleted file mode 100644
index 2c631a846c10cf51d32241263aff9c38a5bd8e27..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/pip-9.0.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-9.0.1-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..4b8ecc69db7e37fc6dd7b6dd8f690508f42866a1
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl
deleted file mode 100644
index cbcc2200f46eb05d707ad7c2c581ac36e561c256..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..502e3cb418c154872ad6e677ef8b63557b38ec35
GIT binary patch
[stripped]

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 21:19:02 2016
From: python-checkins at python.org (donald.stufft)
Date: Wed, 16 Nov 2016 02:19:02 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_-_upgrade_pip/setuptools?=
Message-ID: <20161116021902.106780.19465.BF00E38D@psf.io>

https://hg.python.org/cpython/rev/9c32309e4dec
changeset:   105139:9c32309e4dec
branch:      3.6
parent:      105135:0d663f758adb
parent:      105138:e824e6113a04
user:        Donald Stufft <donald at stufft.io>
date:        Tue Nov 15 21:18:40 2016 -0500
summary:
  Merge - upgrade pip/setuptools

files:
  Lib/ensurepip/__init__.py                                     |    4 ++--
  Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/pip-9.0.1-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl |  Bin 
  Lib/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl |  Bin 
  5 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -8,9 +8,9 @@
 __all__ = ["version", "bootstrap"]
 
 
-_SETUPTOOLS_VERSION = "28.7.1"
+_SETUPTOOLS_VERSION = "28.8.0"
 
-_PIP_VERSION = "9.0.0"
+_PIP_VERSION = "9.0.1"
 
 _PROJECTS = [
     ("setuptools", _SETUPTOOLS_VERSION),
diff --git a/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl
deleted file mode 100644
index 2c631a846c10cf51d32241263aff9c38a5bd8e27..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/pip-9.0.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-9.0.1-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..4b8ecc69db7e37fc6dd7b6dd8f690508f42866a1
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl
deleted file mode 100644
index cbcc2200f46eb05d707ad7c2c581ac36e561c256..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..502e3cb418c154872ad6e677ef8b63557b38ec35
GIT binary patch
[stripped]

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 21:19:02 2016
From: python-checkins at python.org (donald.stufft)
Date: Wed, 16 Nov 2016 02:19:02 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_-_upgrade_pip/setuptools?=
Message-ID: <20161116021902.25549.81101.0D69709D@psf.io>

https://hg.python.org/cpython/rev/2faac299acaa
changeset:   105140:2faac299acaa
parent:      105136:c7d2ec49a80b
parent:      105139:9c32309e4dec
user:        Donald Stufft <donald at stufft.io>
date:        Tue Nov 15 21:18:54 2016 -0500
summary:
  Merge - upgrade pip/setuptools

files:
  Lib/ensurepip/__init__.py                                     |    4 ++--
  Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/pip-9.0.1-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl |  Bin 
  Lib/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl |  Bin 
  5 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -8,9 +8,9 @@
 __all__ = ["version", "bootstrap"]
 
 
-_SETUPTOOLS_VERSION = "28.7.1"
+_SETUPTOOLS_VERSION = "28.8.0"
 
-_PIP_VERSION = "9.0.0"
+_PIP_VERSION = "9.0.1"
 
 _PROJECTS = [
     ("setuptools", _SETUPTOOLS_VERSION),
diff --git a/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl
deleted file mode 100644
index 2c631a846c10cf51d32241263aff9c38a5bd8e27..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/pip-9.0.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-9.0.1-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..4b8ecc69db7e37fc6dd7b6dd8f690508f42866a1
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl
deleted file mode 100644
index cbcc2200f46eb05d707ad7c2c581ac36e561c256..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..502e3cb418c154872ad6e677ef8b63557b38ec35
GIT binary patch
[stripped]

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 15 21:21:42 2016
From: python-checkins at python.org (donald.stufft)
Date: Wed, 16 Nov 2016 02:21:42 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Upgrade_pip_to?=
 =?utf-8?q?_9=2E0=2E1_and_setuptools_to_28=2E8=2E0?=
Message-ID: <20161116022141.6913.35959.11E4C071@psf.io>

https://hg.python.org/cpython/rev/ea91bb92c28b
changeset:   105141:ea91bb92c28b
branch:      2.7
parent:      105108:6ca91a14a555
user:        Donald Stufft <donald at stufft.io>
date:        Tue Nov 15 21:21:35 2016 -0500
summary:
  Upgrade pip to 9.0.1 and setuptools to 28.8.0

files:
  Lib/ensurepip/__init__.py                                     |    4 ++--
  Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/pip-9.0.1-py2.py3-none-any.whl         |  Bin 
  Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl |  Bin 
  Lib/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl |  Bin 
  5 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -12,9 +12,9 @@
 __all__ = ["version", "bootstrap"]
 
 
-_SETUPTOOLS_VERSION = "28.7.1"
+_SETUPTOOLS_VERSION = "28.8.0"
 
-_PIP_VERSION = "9.0.0"
+_PIP_VERSION = "9.0.1"
 
 _PROJECTS = [
     ("setuptools", _SETUPTOOLS_VERSION),
diff --git a/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-9.0.0-py2.py3-none-any.whl
deleted file mode 100644
index 2c631a846c10cf51d32241263aff9c38a5bd8e27..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/pip-9.0.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-9.0.1-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..4b8ecc69db7e37fc6dd7b6dd8f690508f42866a1
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-28.7.1-py2.py3-none-any.whl
deleted file mode 100644
index cbcc2200f46eb05d707ad7c2c581ac36e561c256..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/Lib/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..502e3cb418c154872ad6e677ef8b63557b38ec35
GIT binary patch
[stripped]

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 01:35:08 2016
From: python-checkins at python.org (xavier.degaye)
Date: Wed, 16 Nov 2016 06:35:08 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2326920=3A_Merge_3=2E6?=
Message-ID: <20161116063508.93116.25153.27AEAD96@psf.io>

https://hg.python.org/cpython/rev/f358d849c14e
changeset:   105143:f358d849c14e
parent:      105140:2faac299acaa
parent:      105142:73bd6bbbb5e2
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Wed Nov 16 07:26:10 2016 +0100
summary:
  Issue #26920: Merge 3.6

files:
  Misc/NEWS            |  3 +++
  Python/pylifecycle.c |  2 +-
  2 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #26920: Fix not getting the locale's charset upon initializing the
+  interpreter, on platforms that do not have langinfo.
+
 - Issue #28648: Fixed crash in Py_DecodeLocale() in debug build on Mac OS X
   when decode astral characters.  Patch by Xiang Zhang.
 
diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c
--- a/Python/pylifecycle.c
+++ b/Python/pylifecycle.c
@@ -315,7 +315,7 @@
     initialized = 1;
     _Py_Finalizing = NULL;
 
-#if defined(HAVE_LANGINFO_H) && defined(HAVE_SETLOCALE)
+#ifdef HAVE_SETLOCALE
     /* Set up the LC_CTYPE locale, so we can obtain
        the locale's charset without having to switch
        locales. */

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 01:35:08 2016
From: python-checkins at python.org (xavier.degaye)
Date: Wed, 16 Nov 2016 06:35:08 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI2OTIw?=
 =?utf-8?q?=3A_Fix_not_getting_the_locale=27s_charset_upon_initializing_th?=
 =?utf-8?q?e?=
Message-ID: <20161116063508.107052.90642.679782E8@psf.io>

https://hg.python.org/cpython/rev/73bd6bbbb5e2
changeset:   105142:73bd6bbbb5e2
branch:      3.6
parent:      105139:9c32309e4dec
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Wed Nov 16 07:24:20 2016 +0100
summary:
  Issue #26920: Fix not getting the locale's charset upon initializing the interpreter,
on platforms that do not have langinfo

files:
  Misc/NEWS            |  3 +++
  Python/pylifecycle.c |  2 +-
  2 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #26920: Fix not getting the locale's charset upon initializing the
+  interpreter, on platforms that do not have langinfo.
+
 - Issue #28648: Fixed crash in Py_DecodeLocale() in debug build on Mac OS X
   when decode astral characters.  Patch by Xiang Zhang.
 
diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c
--- a/Python/pylifecycle.c
+++ b/Python/pylifecycle.c
@@ -315,7 +315,7 @@
     initialized = 1;
     _Py_Finalizing = NULL;
 
-#if defined(HAVE_LANGINFO_H) && defined(HAVE_SETLOCALE)
+#ifdef HAVE_SETLOCALE
     /* Set up the LC_CTYPE locale, so we can obtain
        the locale's charset without having to switch
        locales. */

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 02:07:06 2016
From: python-checkins at python.org (xavier.degaye)
Date: Wed, 16 Nov 2016 07:07:06 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2326935=3A_Merge_3=2E6?=
Message-ID: <20161116070706.106218.57731.90145727@psf.io>

https://hg.python.org/cpython/rev/da59b7084dbe
changeset:   105145:da59b7084dbe
parent:      105143:f358d849c14e
parent:      105144:80e4cb5888f3
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Wed Nov 16 08:06:12 2016 +0100
summary:
  Issue #26935: Merge 3.6

files:
  Lib/test/test_os.py |  9 +++++++--
  1 files changed, 7 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
--- a/Lib/test/test_os.py
+++ b/Lib/test/test_os.py
@@ -56,7 +56,7 @@
 try:
     import pwd
     all_users = [u.pw_uid for u in pwd.getpwall()]
-except ImportError:
+except (ImportError, AttributeError):
     all_users = []
 try:
     from _testcapi import INT_MAX, PY_SSIZE_T_MAX
@@ -1423,7 +1423,12 @@
                         break
                 os.closerange(3, 256)
             with open({TESTFN!r}, 'rb') as f:
-                os.dup2(f.fileno(), fd)
+                new_fd = f.fileno()
+                # Issue #26935: posix allows new_fd and fd to be equal but
+                # some libc implementations have dup2 return an error in this
+                # case.
+                if new_fd != fd:
+                    os.dup2(new_fd, fd)
                 sys.stdout.buffer.write(os.urandom(4))
                 sys.stdout.buffer.write(os.urandom(4))
             """.format(TESTFN=support.TESTFN)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 02:07:14 2016
From: python-checkins at python.org (xavier.degaye)
Date: Wed, 16 Nov 2016 07:07:14 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI2OTM1?=
 =?utf-8?q?=3A_Fix_broken_Android_dup2=28=29_in_test=5Fos?=
Message-ID: <20161116070706.13649.1042.BA457643@psf.io>

https://hg.python.org/cpython/rev/80e4cb5888f3
changeset:   105144:80e4cb5888f3
branch:      3.6
parent:      105142:73bd6bbbb5e2
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Wed Nov 16 08:05:27 2016 +0100
summary:
  Issue #26935: Fix broken Android dup2() in test_os

files:
  Lib/test/test_os.py |  9 +++++++--
  1 files changed, 7 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
--- a/Lib/test/test_os.py
+++ b/Lib/test/test_os.py
@@ -56,7 +56,7 @@
 try:
     import pwd
     all_users = [u.pw_uid for u in pwd.getpwall()]
-except ImportError:
+except (ImportError, AttributeError):
     all_users = []
 try:
     from _testcapi import INT_MAX, PY_SSIZE_T_MAX
@@ -1423,7 +1423,12 @@
                         break
                 os.closerange(3, 256)
             with open({TESTFN!r}, 'rb') as f:
-                os.dup2(f.fileno(), fd)
+                new_fd = f.fileno()
+                # Issue #26935: posix allows new_fd and fd to be equal but
+                # some libc implementations have dup2 return an error in this
+                # case.
+                if new_fd != fd:
+                    os.dup2(new_fd, fd)
                 sys.stdout.buffer.write(os.urandom(4))
                 sys.stdout.buffer.write(os.urandom(4))
             """.format(TESTFN=support.TESTFN)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 03:20:16 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 16 Nov 2016 08:20:16 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NzAx?=
 =?utf-8?q?=3A_Replace_PyUnicode=5FCompareWithASCIIString_with?=
Message-ID: <20161116082016.93220.41845.D35950FC@psf.io>

https://hg.python.org/cpython/rev/386c682dcd75
changeset:   105146:386c682dcd75
branch:      3.5
parent:      105138:e824e6113a04
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 16 10:17:58 2016 +0200
summary:
  Issue #28701: Replace PyUnicode_CompareWithASCIIString with _PyUnicode_EqualToASCIIString.

The latter function is more readable, faster and doesn't raise exceptions.

files:
  Doc/c-api/unicode.rst        |   6 +++
  Include/unicodeobject.h      |  11 ++++++
  Modules/_decimal/_decimal.c  |  14 +++---
  Modules/_elementtree.c       |   6 +-
  Modules/_io/textio.c         |   2 +-
  Modules/_lsprof.c            |   2 +-
  Modules/_pickle.c            |   2 +-
  Modules/_sqlite/connection.c |   2 +-
  Modules/_testcapimodule.c    |   2 +-
  Modules/pyexpat.c            |  43 +++++++++++------------
  Objects/longobject.c         |   8 ++--
  Objects/moduleobject.c       |   2 +-
  Objects/typeobject.c         |   4 +-
  Objects/unicodeobject.c      |  35 +++++++++++++++++++
  Python/_warnings.c           |  14 +++---
  Python/ast.c                 |   4 +-
  Python/compile.c             |  10 ++--
  Python/future.c              |   2 +-
  Python/getargs.c             |   2 +-
  Python/import.c              |  20 +++-------
  Python/symtable.c            |   4 +-
  21 files changed, 120 insertions(+), 75 deletions(-)


diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -1638,6 +1638,9 @@
    Compare two strings and return ``-1``, ``0``, ``1`` for less than, equal, and greater than,
    respectively.
 
+   This function returns ``-1`` upon failure, so one should call
+   :c:func:`PyErr_Occurred` to check for errors.
+
 
 .. c:function:: int PyUnicode_CompareWithASCIIString(PyObject *uni, const char *string)
 
@@ -1646,6 +1649,9 @@
    ASCII-encoded strings, but the function interprets the input string as
    ISO-8859-1 if it contains non-ASCII characters.
 
+   This function returns ``-1`` upon failure, so one should call
+   :c:func:`PyErr_Occurred` to check for errors.
+
 
 .. c:function:: PyObject* PyUnicode_RichCompare(PyObject *left,  PyObject *right,  int op)
 
diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -2011,6 +2011,17 @@
     const char *right           /* ASCII-encoded string */
     );
 
+#ifndef Py_LIMITED_API
+/* Test whether a unicode is equal to ASCII string.  Return 1 if true,
+   0 otherwise.  Return 0 if any argument contains non-ASCII characters.
+   Any error occurs inside will be cleared before return. */
+
+PyAPI_FUNC(int) _PyUnicode_EqualToASCIIString(
+    PyObject *left,
+    const char *right           /* ASCII-encoded string */
+    );
+#endif
+
 /* Rich compare two strings and return one of the following:
 
    - NULL in case an exception was raised
diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c
--- a/Modules/_decimal/_decimal.c
+++ b/Modules/_decimal/_decimal.c
@@ -1119,12 +1119,12 @@
     PyObject *retval;
 
     if (PyUnicode_Check(name)) {
-        if (PyUnicode_CompareWithASCIIString(name, "traps") == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, "traps")) {
             retval = ((PyDecContextObject *)self)->traps;
             Py_INCREF(retval);
             return retval;
         }
-        if (PyUnicode_CompareWithASCIIString(name, "flags") == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, "flags")) {
             retval = ((PyDecContextObject *)self)->flags;
             Py_INCREF(retval);
             return retval;
@@ -1144,10 +1144,10 @@
     }
 
     if (PyUnicode_Check(name)) {
-        if (PyUnicode_CompareWithASCIIString(name, "traps") == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, "traps")) {
             return context_settraps_dict(self, value);
         }
-        if (PyUnicode_CompareWithASCIIString(name, "flags") == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, "flags")) {
             return context_setstatus_dict(self, value);
         }
     }
@@ -2446,14 +2446,14 @@
     tmp = PyTuple_GET_ITEM(dectuple, 2);
     if (PyUnicode_Check(tmp)) {
         /* special */
-        if (PyUnicode_CompareWithASCIIString(tmp, "F") == 0) {
+        if (_PyUnicode_EqualToASCIIString(tmp, "F")) {
             strcat(sign_special, "Inf");
             is_infinite = 1;
         }
-        else if (PyUnicode_CompareWithASCIIString(tmp, "n") == 0) {
+        else if (_PyUnicode_EqualToASCIIString(tmp, "n")) {
             strcat(sign_special, "NaN");
         }
-        else if (PyUnicode_CompareWithASCIIString(tmp, "N") == 0) {
+        else if (_PyUnicode_EqualToASCIIString(tmp, "N")) {
             strcat(sign_special, "sNaN");
         }
         else {
diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c
--- a/Modules/_elementtree.c
+++ b/Modules/_elementtree.c
@@ -3686,11 +3686,11 @@
 {
     if (PyUnicode_Check(nameobj)) {
         PyObject* res;
-        if (PyUnicode_CompareWithASCIIString(nameobj, "entity") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "entity"))
             res = self->entity;
-        else if (PyUnicode_CompareWithASCIIString(nameobj, "target") == 0)
+        else if (_PyUnicode_EqualToASCIIString(nameobj, "target"))
             res = self->target;
-        else if (PyUnicode_CompareWithASCIIString(nameobj, "version") == 0) {
+        else if (_PyUnicode_EqualToASCIIString(nameobj, "version")) {
             return PyUnicode_FromFormat(
                 "Expat %d.%d.%d", XML_MAJOR_VERSION,
                 XML_MINOR_VERSION, XML_MICRO_VERSION);
diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c
--- a/Modules/_io/textio.c
+++ b/Modules/_io/textio.c
@@ -1023,7 +1023,7 @@
         else if (PyUnicode_Check(res)) {
             encodefuncentry *e = encodefuncs;
             while (e->name != NULL) {
-                if (!PyUnicode_CompareWithASCIIString(res, e->name)) {
+                if (_PyUnicode_EqualToASCIIString(res, e->name)) {
                     self->encodefunc = e->encodefunc;
                     break;
                 }
diff --git a/Modules/_lsprof.c b/Modules/_lsprof.c
--- a/Modules/_lsprof.c
+++ b/Modules/_lsprof.c
@@ -185,7 +185,7 @@
             }
         }
         if (modname != NULL) {
-            if (PyUnicode_CompareWithASCIIString(modname, "builtins") != 0) {
+            if (!_PyUnicode_EqualToASCIIString(modname, "builtins")) {
                 PyObject *result;
                 result = PyUnicode_FromFormat("<%U.%s>", modname,
                                               fn->m_ml->ml_name);
diff --git a/Modules/_pickle.c b/Modules/_pickle.c
--- a/Modules/_pickle.c
+++ b/Modules/_pickle.c
@@ -1675,7 +1675,7 @@
     while (PyDict_Next(modules_dict, &i, &module_name, &module)) {
         PyObject *candidate;
         if (PyUnicode_Check(module_name) &&
-            !PyUnicode_CompareWithASCIIString(module_name, "__main__"))
+            _PyUnicode_EqualToASCIIString(module_name, "__main__"))
             continue;
         if (module == Py_None)
             continue;
diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c
--- a/Modules/_sqlite/connection.c
+++ b/Modules/_sqlite/connection.c
@@ -1216,7 +1216,7 @@
             return -1;
         }
         for (candidate = begin_statements; *candidate; candidate++) {
-            if (!PyUnicode_CompareWithASCIIString(uppercase_level, *candidate + 6))
+            if (_PyUnicode_EqualToASCIIString(uppercase_level, *candidate + 6))
                 break;
         }
         Py_DECREF(uppercase_level);
diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c
--- a/Modules/_testcapimodule.c
+++ b/Modules/_testcapimodule.c
@@ -2310,7 +2310,7 @@
     result = PyUnicode_FromFormat(FORMAT, (TYPE)1);                 \
     if (result == NULL)                                             \
         return NULL;                                                \
-    if (PyUnicode_CompareWithASCIIString(result, "1")) {     \
+    if (!_PyUnicode_EqualToASCIIString(result, "1")) {              \
         msg = FORMAT " failed at 1";                                \
         goto Fail;                                                  \
     }                                                               \
diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c
--- a/Modules/pyexpat.c
+++ b/Modules/pyexpat.c
@@ -1247,8 +1247,7 @@
 {
     int i;
     for (i = 0; handler_info[i].name != NULL; i++) {
-        if (PyUnicode_CompareWithASCIIString(
-                name, handler_info[i].name) == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, handler_info[i].name)) {
             return i;
         }
     }
@@ -1286,45 +1285,45 @@
 
     first_char = PyUnicode_READ_CHAR(nameobj, 0);
     if (first_char == 'E') {
-        if (PyUnicode_CompareWithASCIIString(nameobj, "ErrorCode") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "ErrorCode"))
             return PyLong_FromLong((long)
                                   XML_GetErrorCode(self->itself));
-        if (PyUnicode_CompareWithASCIIString(nameobj, "ErrorLineNumber") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "ErrorLineNumber"))
             return PyLong_FromLong((long)
                                   XML_GetErrorLineNumber(self->itself));
-        if (PyUnicode_CompareWithASCIIString(nameobj, "ErrorColumnNumber") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "ErrorColumnNumber"))
             return PyLong_FromLong((long)
                                   XML_GetErrorColumnNumber(self->itself));
-        if (PyUnicode_CompareWithASCIIString(nameobj, "ErrorByteIndex") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "ErrorByteIndex"))
             return PyLong_FromLong((long)
                                   XML_GetErrorByteIndex(self->itself));
     }
     if (first_char == 'C') {
-        if (PyUnicode_CompareWithASCIIString(nameobj, "CurrentLineNumber") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "CurrentLineNumber"))
             return PyLong_FromLong((long)
                                   XML_GetCurrentLineNumber(self->itself));
-        if (PyUnicode_CompareWithASCIIString(nameobj, "CurrentColumnNumber") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "CurrentColumnNumber"))
             return PyLong_FromLong((long)
                                   XML_GetCurrentColumnNumber(self->itself));
-        if (PyUnicode_CompareWithASCIIString(nameobj, "CurrentByteIndex") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "CurrentByteIndex"))
             return PyLong_FromLong((long)
                                   XML_GetCurrentByteIndex(self->itself));
     }
     if (first_char == 'b') {
-        if (PyUnicode_CompareWithASCIIString(nameobj, "buffer_size") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "buffer_size"))
             return PyLong_FromLong((long) self->buffer_size);
-        if (PyUnicode_CompareWithASCIIString(nameobj, "buffer_text") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "buffer_text"))
             return get_pybool(self->buffer != NULL);
-        if (PyUnicode_CompareWithASCIIString(nameobj, "buffer_used") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "buffer_used"))
             return PyLong_FromLong((long) self->buffer_used);
     }
-    if (PyUnicode_CompareWithASCIIString(nameobj, "namespace_prefixes") == 0)
+    if (_PyUnicode_EqualToASCIIString(nameobj, "namespace_prefixes"))
         return get_pybool(self->ns_prefixes);
-    if (PyUnicode_CompareWithASCIIString(nameobj, "ordered_attributes") == 0)
+    if (_PyUnicode_EqualToASCIIString(nameobj, "ordered_attributes"))
         return get_pybool(self->ordered_attributes);
-    if (PyUnicode_CompareWithASCIIString(nameobj, "specified_attributes") == 0)
+    if (_PyUnicode_EqualToASCIIString(nameobj, "specified_attributes"))
         return get_pybool((long) self->specified_attributes);
-    if (PyUnicode_CompareWithASCIIString(nameobj, "intern") == 0) {
+    if (_PyUnicode_EqualToASCIIString(nameobj, "intern")) {
         if (self->intern == NULL) {
             Py_INCREF(Py_None);
             return Py_None;
@@ -1388,7 +1387,7 @@
         PyErr_SetString(PyExc_RuntimeError, "Cannot delete attribute");
         return -1;
     }
-    if (PyUnicode_CompareWithASCIIString(name, "buffer_text") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "buffer_text")) {
         int b = PyObject_IsTrue(v);
         if (b < 0)
             return -1;
@@ -1410,7 +1409,7 @@
         }
         return 0;
     }
-    if (PyUnicode_CompareWithASCIIString(name, "namespace_prefixes") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "namespace_prefixes")) {
         int b = PyObject_IsTrue(v);
         if (b < 0)
             return -1;
@@ -1418,14 +1417,14 @@
         XML_SetReturnNSTriplet(self->itself, self->ns_prefixes);
         return 0;
     }
-    if (PyUnicode_CompareWithASCIIString(name, "ordered_attributes") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "ordered_attributes")) {
         int b = PyObject_IsTrue(v);
         if (b < 0)
             return -1;
         self->ordered_attributes = b;
         return 0;
     }
-    if (PyUnicode_CompareWithASCIIString(name, "specified_attributes") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "specified_attributes")) {
         int b = PyObject_IsTrue(v);
         if (b < 0)
             return -1;
@@ -1433,7 +1432,7 @@
         return 0;
     }
 
-    if (PyUnicode_CompareWithASCIIString(name, "buffer_size") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "buffer_size")) {
       long new_buffer_size;
       if (!PyLong_Check(v)) {
         PyErr_SetString(PyExc_TypeError, "buffer_size must be an integer");
@@ -1479,7 +1478,7 @@
       return 0;
     }
 
-    if (PyUnicode_CompareWithASCIIString(name, "CharacterDataHandler") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "CharacterDataHandler")) {
         /* If we're changing the character data handler, flush all
          * cached data with the old handler.  Not sure there's a
          * "right" thing to do, though, but this probably won't
diff --git a/Objects/longobject.c b/Objects/longobject.c
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -4936,9 +4936,9 @@
         return NULL;
     }
 
-    if (!PyUnicode_CompareWithASCIIString(byteorder_str, "little"))
+    if (_PyUnicode_EqualToASCIIString(byteorder_str, "little"))
         little_endian = 1;
-    else if (!PyUnicode_CompareWithASCIIString(byteorder_str, "big"))
+    else if (_PyUnicode_EqualToASCIIString(byteorder_str, "big"))
         little_endian = 0;
     else {
         PyErr_SetString(PyExc_ValueError,
@@ -5019,9 +5019,9 @@
         return NULL;
     }
 
-    if (!PyUnicode_CompareWithASCIIString(byteorder_str, "little"))
+    if (_PyUnicode_EqualToASCIIString(byteorder_str, "little"))
         little_endian = 1;
-    else if (!PyUnicode_CompareWithASCIIString(byteorder_str, "big"))
+    else if (_PyUnicode_EqualToASCIIString(byteorder_str, "big"))
         little_endian = 0;
     else {
         PyErr_SetString(PyExc_ValueError,
diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c
--- a/Objects/moduleobject.c
+++ b/Objects/moduleobject.c
@@ -587,7 +587,7 @@
     while (PyDict_Next(d, &pos, &key, &value)) {
         if (value != Py_None && PyUnicode_Check(key)) {
             if (PyUnicode_READ_CHAR(key, 0) != '_' ||
-                PyUnicode_CompareWithASCIIString(key, "__builtins__") != 0)
+                !_PyUnicode_EqualToASCIIString(key, "__builtins__"))
             {
                 if (Py_VerboseFlag > 1) {
                     const char *s = _PyUnicode_AsString(key);
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -2395,7 +2395,7 @@
                 }
                 add_dict++;
             }
-            if (PyUnicode_CompareWithASCIIString(tmp, "__weakref__") == 0) {
+            if (_PyUnicode_EqualToASCIIString(tmp, "__weakref__")) {
                 if (!may_add_weak || add_weak) {
                     PyErr_SetString(PyExc_TypeError,
                         "__weakref__ slot disallowed: "
@@ -2419,7 +2419,7 @@
             if ((add_dict &&
                  _PyUnicode_CompareWithId(tmp, &PyId___dict__) == 0) ||
                 (add_weak &&
-                 PyUnicode_CompareWithASCIIString(tmp, "__weakref__") == 0))
+                 _PyUnicode_EqualToASCIIString(tmp, "__weakref__")))
                 continue;
             tmp =_Py_Mangle(name, tmp);
             if (!tmp) {
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -10834,6 +10834,41 @@
     }
 }
 
+static int
+non_ready_unicode_equal_to_ascii_string(PyObject *unicode, const char *str)
+{
+    size_t i, len;
+    const wchar_t *p;
+    len = (size_t)_PyUnicode_WSTR_LENGTH(unicode);
+    if (strlen(str) != len)
+        return 0;
+    p = _PyUnicode_WSTR(unicode);
+    assert(p);
+    for (i = 0; i < len; i++) {
+        unsigned char c = (unsigned char)str[i];
+        if (c > 128 || p[i] != (wchar_t)c)
+            return 0;
+    }
+    return 1;
+}
+
+int
+_PyUnicode_EqualToASCIIString(PyObject *unicode, const char *str)
+{
+    size_t len;
+    assert(_PyUnicode_CHECK(unicode));
+    if (PyUnicode_READY(unicode) == -1) {
+        /* Memory error or bad data */
+        PyErr_Clear();
+        return non_ready_unicode_equal_to_ascii_string(unicode, str);
+    }
+    if (!PyUnicode_IS_ASCII(unicode))
+        return 0;
+    len = (size_t)PyUnicode_GET_LENGTH(unicode);
+    return strlen(str) == len &&
+           memcmp(PyUnicode_1BYTE_DATA(unicode), str, len) == 0;
+}
+
 
 #define TEST_COND(cond)                         \
     ((cond) ? Py_True : Py_False)
diff --git a/Python/_warnings.c b/Python/_warnings.c
--- a/Python/_warnings.c
+++ b/Python/_warnings.c
@@ -431,7 +431,7 @@
     if (action == NULL)
         goto cleanup;
 
-    if (PyUnicode_CompareWithASCIIString(action, "error") == 0) {
+    if (_PyUnicode_EqualToASCIIString(action, "error")) {
         PyErr_SetObject(category, message);
         goto cleanup;
     }
@@ -439,13 +439,13 @@
     /* Store in the registry that we've been here, *except* when the action
        is "always". */
     rc = 0;
-    if (PyUnicode_CompareWithASCIIString(action, "always") != 0) {
+    if (!_PyUnicode_EqualToASCIIString(action, "always")) {
         if (registry != NULL && registry != Py_None &&
                 PyDict_SetItem(registry, key, Py_True) < 0)
             goto cleanup;
-        else if (PyUnicode_CompareWithASCIIString(action, "ignore") == 0)
+        else if (_PyUnicode_EqualToASCIIString(action, "ignore"))
             goto return_none;
-        else if (PyUnicode_CompareWithASCIIString(action, "once") == 0) {
+        else if (_PyUnicode_EqualToASCIIString(action, "once")) {
             if (registry == NULL || registry == Py_None) {
                 registry = get_once_registry();
                 if (registry == NULL)
@@ -454,12 +454,12 @@
             /* _once_registry[(text, category)] = 1 */
             rc = update_registry(registry, text, category, 0);
         }
-        else if (PyUnicode_CompareWithASCIIString(action, "module") == 0) {
+        else if (_PyUnicode_EqualToASCIIString(action, "module")) {
             /* registry[(text, category, 0)] = 1 */
             if (registry != NULL && registry != Py_None)
                 rc = update_registry(registry, text, category, 0);
         }
-        else if (PyUnicode_CompareWithASCIIString(action, "default") != 0) {
+        else if (!_PyUnicode_EqualToASCIIString(action, "default")) {
             PyErr_Format(PyExc_RuntimeError,
                         "Unrecognized action (%R) in warnings.filters:\n %R",
                         action, item);
@@ -665,7 +665,7 @@
     }
     else {
         *filename = NULL;
-        if (*module != Py_None && PyUnicode_CompareWithASCIIString(*module, "__main__") == 0) {
+        if (*module != Py_None && _PyUnicode_EqualToASCIIString(*module, "__main__")) {
             PyObject *argv = _PySys_GetObjectId(&PyId_argv);
             /* PyList_Check() is needed because sys.argv is set to None during
                Python finalization */
diff --git a/Python/ast.c b/Python/ast.c
--- a/Python/ast.c
+++ b/Python/ast.c
@@ -876,14 +876,14 @@
                int full_checks)
 {
     assert(PyUnicode_Check(name));
-    if (PyUnicode_CompareWithASCIIString(name, "__debug__") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "__debug__")) {
         ast_error(c, n, "assignment to keyword");
         return 1;
     }
     if (full_checks) {
         const char **p;
         for (p = FORBIDDEN; *p; p++) {
-            if (PyUnicode_CompareWithASCIIString(name, *p) == 0) {
+            if (_PyUnicode_EqualToASCIIString(name, *p)) {
                 ast_error(c, n, "assignment to keyword");
                 return 1;
             }
diff --git a/Python/compile.c b/Python/compile.c
--- a/Python/compile.c
+++ b/Python/compile.c
@@ -1390,7 +1390,7 @@
 {
     int scope;
     if (c->u->u_scope_type == COMPILER_SCOPE_CLASS &&
-        !PyUnicode_CompareWithASCIIString(name, "__class__"))
+        _PyUnicode_EqualToASCIIString(name, "__class__"))
         return CELL;
     scope = PyST_GetScope(c->u->u_ste, name);
     if (scope == 0) {
@@ -2513,7 +2513,7 @@
     }
 
     if (s->lineno > c->c_future->ff_lineno && s->v.ImportFrom.module &&
-        !PyUnicode_CompareWithASCIIString(s->v.ImportFrom.module, "__future__")) {
+        _PyUnicode_EqualToASCIIString(s->v.ImportFrom.module, "__future__")) {
         Py_DECREF(level);
         Py_DECREF(names);
         return compiler_error(c, "from __future__ imports must occur "
@@ -2837,9 +2837,9 @@
     if (!mangled)
         return 0;
 
-    assert(PyUnicode_CompareWithASCIIString(name, "None") &&
-           PyUnicode_CompareWithASCIIString(name, "True") &&
-           PyUnicode_CompareWithASCIIString(name, "False"));
+    assert(!_PyUnicode_EqualToASCIIString(name, "None") &&
+           !_PyUnicode_EqualToASCIIString(name, "True") &&
+           !_PyUnicode_EqualToASCIIString(name, "False"));
 
     op = 0;
     optype = OP_NAME;
diff --git a/Python/future.c b/Python/future.c
--- a/Python/future.c
+++ b/Python/future.c
@@ -99,7 +99,7 @@
         if (s->kind == ImportFrom_kind) {
             identifier modname = s->v.ImportFrom.module;
             if (modname &&
-                !PyUnicode_CompareWithASCIIString(modname, "__future__")) {
+                _PyUnicode_EqualToASCIIString(modname, "__future__")) {
                 if (done) {
                     PyErr_SetString(PyExc_SyntaxError,
                                     ERR_LATE_FUTURE);
diff --git a/Python/getargs.c b/Python/getargs.c
--- a/Python/getargs.c
+++ b/Python/getargs.c
@@ -1618,7 +1618,7 @@
                 return cleanreturn(0, &freelist);
             }
             for (i = 0; i < len; i++) {
-                if (!PyUnicode_CompareWithASCIIString(key, kwlist[i])) {
+                if (_PyUnicode_EqualToASCIIString(key, kwlist[i])) {
                     match = 1;
                     break;
                 }
diff --git a/Python/import.c b/Python/import.c
--- a/Python/import.c
+++ b/Python/import.c
@@ -946,10 +946,9 @@
 static int
 is_builtin(PyObject *name)
 {
-    int i, cmp;
+    int i;
     for (i = 0; PyImport_Inittab[i].name != NULL; i++) {
-        cmp = PyUnicode_CompareWithASCIIString(name, PyImport_Inittab[i].name);
-        if (cmp == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, PyImport_Inittab[i].name)) {
             if (PyImport_Inittab[i].initfunc == NULL)
                 return -1;
             else
@@ -1069,7 +1068,7 @@
 
     for (p = PyImport_Inittab; p->name != NULL; p++) {
         PyModuleDef *def;
-        if (PyUnicode_CompareWithASCIIString(name, p->name) == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, p->name)) {
             if (p->initfunc == NULL) {
                 /* Cannot re-init internal module ("sys" or "builtins") */
                 mod = PyImport_AddModule(namestr);
@@ -1115,7 +1114,7 @@
     for (p = PyImport_FrozenModules; ; p++) {
         if (p->name == NULL)
             return NULL;
-        if (PyUnicode_CompareWithASCIIString(name, p->name) == 0)
+        if (_PyUnicode_EqualToASCIIString(name, p->name))
             break;
     }
     return p;
@@ -1316,12 +1315,8 @@
         int now_in_importlib;
 
         assert(PyTraceBack_Check(tb));
-        now_in_importlib = (PyUnicode_CompareWithASCIIString(
-                                code->co_filename,
-                                importlib_filename) == 0) ||
-                           (PyUnicode_CompareWithASCIIString(
-                                code->co_filename,
-                                external_filename) == 0);
+        now_in_importlib = _PyUnicode_EqualToASCIIString(code->co_filename, importlib_filename) ||
+                           _PyUnicode_EqualToASCIIString(code->co_filename, external_filename);
         if (now_in_importlib && !in_importlib) {
             /* This is the link to this chunk of importlib tracebacks */
             outer_link = prev_link;
@@ -1330,8 +1325,7 @@
 
         if (in_importlib &&
             (always_trim ||
-             PyUnicode_CompareWithASCIIString(code->co_name,
-                                              remove_frames) == 0)) {
+             _PyUnicode_EqualToASCIIString(code->co_name, remove_frames))) {
             PyObject *tmp = *outer_link;
             *outer_link = next;
             Py_XINCREF(next);
diff --git a/Python/symtable.c b/Python/symtable.c
--- a/Python/symtable.c
+++ b/Python/symtable.c
@@ -1472,7 +1472,7 @@
         /* Special-case super: it counts as a use of __class__ */
         if (e->v.Name.ctx == Load &&
             st->st_cur->ste_type == FunctionBlock &&
-            !PyUnicode_CompareWithASCIIString(e->v.Name.id, "super")) {
+            _PyUnicode_EqualToASCIIString(e->v.Name.id, "super")) {
             if (!GET_IDENTIFIER(__class__) ||
                 !symtable_add_def(st, __class__, USE))
                 VISIT_QUIT(st, 0);
@@ -1621,7 +1621,7 @@
         store_name = name;
         Py_INCREF(store_name);
     }
-    if (PyUnicode_CompareWithASCIIString(name, "*")) {
+    if (!_PyUnicode_EqualToASCIIString(name, "*")) {
         int r = symtable_add_def(st, store_name, DEF_IMPORT);
         Py_DECREF(store_name);
         return r;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 03:20:17 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 16 Nov 2016 08:20:17 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328701=3A_Replace_PyUnicode=5FCompareWithASCIISt?=
 =?utf-8?q?ring_with?=
Message-ID: <20161116082016.94006.7185.DBEE1CC8@psf.io>

https://hg.python.org/cpython/rev/6f0f77333da5
changeset:   105148:6f0f77333da5
parent:      105145:da59b7084dbe
parent:      105147:72d07d13869a
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 16 10:19:57 2016 +0200
summary:
  Issue #28701: Replace PyUnicode_CompareWithASCIIString with _PyUnicode_EqualToASCIIString.

The latter function is more readable, faster and doesn't raise exceptions.

files:
  Doc/c-api/unicode.rst        |   6 +++
  Include/unicodeobject.h      |  11 ++++++
  Modules/_decimal/_decimal.c  |  14 +++---
  Modules/_elementtree.c       |   6 +-
  Modules/_io/textio.c         |   2 +-
  Modules/_io/winconsoleio.c   |   6 +-
  Modules/_lsprof.c            |   2 +-
  Modules/_pickle.c            |   2 +-
  Modules/_sqlite/connection.c |   2 +-
  Modules/_testcapimodule.c    |   2 +-
  Modules/pyexpat.c            |  43 +++++++++++------------
  Objects/longobject.c         |   8 ++--
  Objects/moduleobject.c       |   2 +-
  Objects/typeobject.c         |   4 +-
  Objects/unicodeobject.c      |  35 +++++++++++++++++++
  Python/_warnings.c           |  14 +++---
  Python/ast.c                 |   8 ++--
  Python/compile.c             |  10 ++--
  Python/future.c              |   2 +-
  Python/getargs.c             |   2 +-
  Python/import.c              |  20 +++-------
  Python/symtable.c            |   4 +-
  22 files changed, 125 insertions(+), 80 deletions(-)


diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -1643,6 +1643,9 @@
    Compare two strings and return ``-1``, ``0``, ``1`` for less than, equal, and greater than,
    respectively.
 
+   This function returns ``-1`` upon failure, so one should call
+   :c:func:`PyErr_Occurred` to check for errors.
+
 
 .. c:function:: int PyUnicode_CompareWithASCIIString(PyObject *uni, const char *string)
 
@@ -1651,6 +1654,9 @@
    ASCII-encoded strings, but the function interprets the input string as
    ISO-8859-1 if it contains non-ASCII characters.
 
+   This function returns ``-1`` upon failure, so one should call
+   :c:func:`PyErr_Occurred` to check for errors.
+
 
 .. c:function:: PyObject* PyUnicode_RichCompare(PyObject *left,  PyObject *right,  int op)
 
diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -2048,6 +2048,17 @@
     const char *right           /* ASCII-encoded string */
     );
 
+#ifndef Py_LIMITED_API
+/* Test whether a unicode is equal to ASCII string.  Return 1 if true,
+   0 otherwise.  Return 0 if any argument contains non-ASCII characters.
+   Any error occurs inside will be cleared before return. */
+
+PyAPI_FUNC(int) _PyUnicode_EqualToASCIIString(
+    PyObject *left,
+    const char *right           /* ASCII-encoded string */
+    );
+#endif
+
 /* Rich compare two strings and return one of the following:
 
    - NULL in case an exception was raised
diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c
--- a/Modules/_decimal/_decimal.c
+++ b/Modules/_decimal/_decimal.c
@@ -1118,12 +1118,12 @@
     PyObject *retval;
 
     if (PyUnicode_Check(name)) {
-        if (PyUnicode_CompareWithASCIIString(name, "traps") == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, "traps")) {
             retval = ((PyDecContextObject *)self)->traps;
             Py_INCREF(retval);
             return retval;
         }
-        if (PyUnicode_CompareWithASCIIString(name, "flags") == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, "flags")) {
             retval = ((PyDecContextObject *)self)->flags;
             Py_INCREF(retval);
             return retval;
@@ -1143,10 +1143,10 @@
     }
 
     if (PyUnicode_Check(name)) {
-        if (PyUnicode_CompareWithASCIIString(name, "traps") == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, "traps")) {
             return context_settraps_dict(self, value);
         }
-        if (PyUnicode_CompareWithASCIIString(name, "flags") == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, "flags")) {
             return context_setstatus_dict(self, value);
         }
     }
@@ -2449,14 +2449,14 @@
     tmp = PyTuple_GET_ITEM(dectuple, 2);
     if (PyUnicode_Check(tmp)) {
         /* special */
-        if (PyUnicode_CompareWithASCIIString(tmp, "F") == 0) {
+        if (_PyUnicode_EqualToASCIIString(tmp, "F")) {
             strcat(sign_special, "Inf");
             is_infinite = 1;
         }
-        else if (PyUnicode_CompareWithASCIIString(tmp, "n") == 0) {
+        else if (_PyUnicode_EqualToASCIIString(tmp, "n")) {
             strcat(sign_special, "NaN");
         }
-        else if (PyUnicode_CompareWithASCIIString(tmp, "N") == 0) {
+        else if (_PyUnicode_EqualToASCIIString(tmp, "N")) {
             strcat(sign_special, "sNaN");
         }
         else {
diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c
--- a/Modules/_elementtree.c
+++ b/Modules/_elementtree.c
@@ -3661,11 +3661,11 @@
 {
     if (PyUnicode_Check(nameobj)) {
         PyObject* res;
-        if (PyUnicode_CompareWithASCIIString(nameobj, "entity") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "entity"))
             res = self->entity;
-        else if (PyUnicode_CompareWithASCIIString(nameobj, "target") == 0)
+        else if (_PyUnicode_EqualToASCIIString(nameobj, "target"))
             res = self->target;
-        else if (PyUnicode_CompareWithASCIIString(nameobj, "version") == 0) {
+        else if (_PyUnicode_EqualToASCIIString(nameobj, "version")) {
             return PyUnicode_FromFormat(
                 "Expat %d.%d.%d", XML_MAJOR_VERSION,
                 XML_MINOR_VERSION, XML_MICRO_VERSION);
diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c
--- a/Modules/_io/textio.c
+++ b/Modules/_io/textio.c
@@ -1023,7 +1023,7 @@
         else if (PyUnicode_Check(res)) {
             const encodefuncentry *e = encodefuncs;
             while (e->name != NULL) {
-                if (!PyUnicode_CompareWithASCIIString(res, e->name)) {
+                if (_PyUnicode_EqualToASCIIString(res, e->name)) {
                     self->encodefunc = e->encodefunc;
                     break;
                 }
diff --git a/Modules/_io/winconsoleio.c b/Modules/_io/winconsoleio.c
--- a/Modules/_io/winconsoleio.c
+++ b/Modules/_io/winconsoleio.c
@@ -93,11 +93,11 @@
     }
 
     char m = '\0';
-    if (PyUnicode_CompareWithASCIIString(decoded_upper, "CONIN$") == 0) {
+    if (_PyUnicode_EqualToASCIIString(decoded_upper, "CONIN$")) {
         m = 'r';
-    } else if (PyUnicode_CompareWithASCIIString(decoded_upper, "CONOUT$") == 0) {
+    } else if (_PyUnicode_EqualToASCIIString(decoded_upper, "CONOUT$")) {
         m = 'w';
-    } else if (PyUnicode_CompareWithASCIIString(decoded_upper, "CON") == 0) {
+    } else if (_PyUnicode_EqualToASCIIString(decoded_upper, "CON")) {
         m = 'x';
     }
 
diff --git a/Modules/_lsprof.c b/Modules/_lsprof.c
--- a/Modules/_lsprof.c
+++ b/Modules/_lsprof.c
@@ -181,7 +181,7 @@
             }
         }
         if (modname != NULL) {
-            if (PyUnicode_CompareWithASCIIString(modname, "builtins") != 0) {
+            if (!_PyUnicode_EqualToASCIIString(modname, "builtins")) {
                 PyObject *result;
                 result = PyUnicode_FromFormat("<%U.%s>", modname,
                                               fn->m_ml->ml_name);
diff --git a/Modules/_pickle.c b/Modules/_pickle.c
--- a/Modules/_pickle.c
+++ b/Modules/_pickle.c
@@ -1687,7 +1687,7 @@
     while (PyDict_Next(modules_dict, &i, &module_name, &module)) {
         PyObject *candidate;
         if (PyUnicode_Check(module_name) &&
-            !PyUnicode_CompareWithASCIIString(module_name, "__main__"))
+            _PyUnicode_EqualToASCIIString(module_name, "__main__"))
             continue;
         if (module == Py_None)
             continue;
diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c
--- a/Modules/_sqlite/connection.c
+++ b/Modules/_sqlite/connection.c
@@ -1191,7 +1191,7 @@
             return -1;
         }
         for (candidate = begin_statements; *candidate; candidate++) {
-            if (!PyUnicode_CompareWithASCIIString(uppercase_level, *candidate + 6))
+            if (_PyUnicode_EqualToASCIIString(uppercase_level, *candidate + 6))
                 break;
         }
         Py_DECREF(uppercase_level);
diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c
--- a/Modules/_testcapimodule.c
+++ b/Modules/_testcapimodule.c
@@ -2325,7 +2325,7 @@
     result = PyUnicode_FromFormat(FORMAT, (TYPE)1);                 \
     if (result == NULL)                                             \
         return NULL;                                                \
-    if (PyUnicode_CompareWithASCIIString(result, "1")) {     \
+    if (!_PyUnicode_EqualToASCIIString(result, "1")) {              \
         msg = FORMAT " failed at 1";                                \
         goto Fail;                                                  \
     }                                                               \
diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c
--- a/Modules/pyexpat.c
+++ b/Modules/pyexpat.c
@@ -1244,8 +1244,7 @@
 {
     int i;
     for (i = 0; handler_info[i].name != NULL; i++) {
-        if (PyUnicode_CompareWithASCIIString(
-                name, handler_info[i].name) == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, handler_info[i].name)) {
             return i;
         }
     }
@@ -1283,45 +1282,45 @@
 
     first_char = PyUnicode_READ_CHAR(nameobj, 0);
     if (first_char == 'E') {
-        if (PyUnicode_CompareWithASCIIString(nameobj, "ErrorCode") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "ErrorCode"))
             return PyLong_FromLong((long)
                                   XML_GetErrorCode(self->itself));
-        if (PyUnicode_CompareWithASCIIString(nameobj, "ErrorLineNumber") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "ErrorLineNumber"))
             return PyLong_FromLong((long)
                                   XML_GetErrorLineNumber(self->itself));
-        if (PyUnicode_CompareWithASCIIString(nameobj, "ErrorColumnNumber") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "ErrorColumnNumber"))
             return PyLong_FromLong((long)
                                   XML_GetErrorColumnNumber(self->itself));
-        if (PyUnicode_CompareWithASCIIString(nameobj, "ErrorByteIndex") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "ErrorByteIndex"))
             return PyLong_FromLong((long)
                                   XML_GetErrorByteIndex(self->itself));
     }
     if (first_char == 'C') {
-        if (PyUnicode_CompareWithASCIIString(nameobj, "CurrentLineNumber") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "CurrentLineNumber"))
             return PyLong_FromLong((long)
                                   XML_GetCurrentLineNumber(self->itself));
-        if (PyUnicode_CompareWithASCIIString(nameobj, "CurrentColumnNumber") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "CurrentColumnNumber"))
             return PyLong_FromLong((long)
                                   XML_GetCurrentColumnNumber(self->itself));
-        if (PyUnicode_CompareWithASCIIString(nameobj, "CurrentByteIndex") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "CurrentByteIndex"))
             return PyLong_FromLong((long)
                                   XML_GetCurrentByteIndex(self->itself));
     }
     if (first_char == 'b') {
-        if (PyUnicode_CompareWithASCIIString(nameobj, "buffer_size") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "buffer_size"))
             return PyLong_FromLong((long) self->buffer_size);
-        if (PyUnicode_CompareWithASCIIString(nameobj, "buffer_text") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "buffer_text"))
             return get_pybool(self->buffer != NULL);
-        if (PyUnicode_CompareWithASCIIString(nameobj, "buffer_used") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "buffer_used"))
             return PyLong_FromLong((long) self->buffer_used);
     }
-    if (PyUnicode_CompareWithASCIIString(nameobj, "namespace_prefixes") == 0)
+    if (_PyUnicode_EqualToASCIIString(nameobj, "namespace_prefixes"))
         return get_pybool(self->ns_prefixes);
-    if (PyUnicode_CompareWithASCIIString(nameobj, "ordered_attributes") == 0)
+    if (_PyUnicode_EqualToASCIIString(nameobj, "ordered_attributes"))
         return get_pybool(self->ordered_attributes);
-    if (PyUnicode_CompareWithASCIIString(nameobj, "specified_attributes") == 0)
+    if (_PyUnicode_EqualToASCIIString(nameobj, "specified_attributes"))
         return get_pybool((long) self->specified_attributes);
-    if (PyUnicode_CompareWithASCIIString(nameobj, "intern") == 0) {
+    if (_PyUnicode_EqualToASCIIString(nameobj, "intern")) {
         if (self->intern == NULL) {
             Py_INCREF(Py_None);
             return Py_None;
@@ -1383,7 +1382,7 @@
         PyErr_SetString(PyExc_RuntimeError, "Cannot delete attribute");
         return -1;
     }
-    if (PyUnicode_CompareWithASCIIString(name, "buffer_text") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "buffer_text")) {
         int b = PyObject_IsTrue(v);
         if (b < 0)
             return -1;
@@ -1405,7 +1404,7 @@
         }
         return 0;
     }
-    if (PyUnicode_CompareWithASCIIString(name, "namespace_prefixes") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "namespace_prefixes")) {
         int b = PyObject_IsTrue(v);
         if (b < 0)
             return -1;
@@ -1413,14 +1412,14 @@
         XML_SetReturnNSTriplet(self->itself, self->ns_prefixes);
         return 0;
     }
-    if (PyUnicode_CompareWithASCIIString(name, "ordered_attributes") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "ordered_attributes")) {
         int b = PyObject_IsTrue(v);
         if (b < 0)
             return -1;
         self->ordered_attributes = b;
         return 0;
     }
-    if (PyUnicode_CompareWithASCIIString(name, "specified_attributes") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "specified_attributes")) {
         int b = PyObject_IsTrue(v);
         if (b < 0)
             return -1;
@@ -1428,7 +1427,7 @@
         return 0;
     }
 
-    if (PyUnicode_CompareWithASCIIString(name, "buffer_size") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "buffer_size")) {
       long new_buffer_size;
       if (!PyLong_Check(v)) {
         PyErr_SetString(PyExc_TypeError, "buffer_size must be an integer");
@@ -1474,7 +1473,7 @@
       return 0;
     }
 
-    if (PyUnicode_CompareWithASCIIString(name, "CharacterDataHandler") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "CharacterDataHandler")) {
         /* If we're changing the character data handler, flush all
          * cached data with the old handler.  Not sure there's a
          * "right" thing to do, though, but this probably won't
diff --git a/Objects/longobject.c b/Objects/longobject.c
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -5166,9 +5166,9 @@
         return NULL;
     }
 
-    if (!PyUnicode_CompareWithASCIIString(byteorder_str, "little"))
+    if (_PyUnicode_EqualToASCIIString(byteorder_str, "little"))
         little_endian = 1;
-    else if (!PyUnicode_CompareWithASCIIString(byteorder_str, "big"))
+    else if (_PyUnicode_EqualToASCIIString(byteorder_str, "big"))
         little_endian = 0;
     else {
         PyErr_SetString(PyExc_ValueError,
@@ -5249,9 +5249,9 @@
         return NULL;
     }
 
-    if (!PyUnicode_CompareWithASCIIString(byteorder_str, "little"))
+    if (_PyUnicode_EqualToASCIIString(byteorder_str, "little"))
         little_endian = 1;
-    else if (!PyUnicode_CompareWithASCIIString(byteorder_str, "big"))
+    else if (_PyUnicode_EqualToASCIIString(byteorder_str, "big"))
         little_endian = 0;
     else {
         PyErr_SetString(PyExc_ValueError,
diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c
--- a/Objects/moduleobject.c
+++ b/Objects/moduleobject.c
@@ -586,7 +586,7 @@
     while (PyDict_Next(d, &pos, &key, &value)) {
         if (value != Py_None && PyUnicode_Check(key)) {
             if (PyUnicode_READ_CHAR(key, 0) != '_' ||
-                PyUnicode_CompareWithASCIIString(key, "__builtins__") != 0)
+                !_PyUnicode_EqualToASCIIString(key, "__builtins__"))
             {
                 if (Py_VerboseFlag > 1) {
                     const char *s = _PyUnicode_AsString(key);
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -2412,7 +2412,7 @@
                 }
                 add_dict++;
             }
-            if (PyUnicode_CompareWithASCIIString(tmp, "__weakref__") == 0) {
+            if (_PyUnicode_EqualToASCIIString(tmp, "__weakref__")) {
                 if (!may_add_weak || add_weak) {
                     PyErr_SetString(PyExc_TypeError,
                         "__weakref__ slot disallowed: "
@@ -2436,7 +2436,7 @@
             if ((add_dict &&
                  _PyUnicode_CompareWithId(tmp, &PyId___dict__) == 0) ||
                 (add_weak &&
-                 PyUnicode_CompareWithASCIIString(tmp, "__weakref__") == 0))
+                 _PyUnicode_EqualToASCIIString(tmp, "__weakref__")))
                 continue;
             tmp =_Py_Mangle(name, tmp);
             if (!tmp) {
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -11048,6 +11048,41 @@
     }
 }
 
+static int
+non_ready_unicode_equal_to_ascii_string(PyObject *unicode, const char *str)
+{
+    size_t i, len;
+    const wchar_t *p;
+    len = (size_t)_PyUnicode_WSTR_LENGTH(unicode);
+    if (strlen(str) != len)
+        return 0;
+    p = _PyUnicode_WSTR(unicode);
+    assert(p);
+    for (i = 0; i < len; i++) {
+        unsigned char c = (unsigned char)str[i];
+        if (c > 128 || p[i] != (wchar_t)c)
+            return 0;
+    }
+    return 1;
+}
+
+int
+_PyUnicode_EqualToASCIIString(PyObject *unicode, const char *str)
+{
+    size_t len;
+    assert(_PyUnicode_CHECK(unicode));
+    if (PyUnicode_READY(unicode) == -1) {
+        /* Memory error or bad data */
+        PyErr_Clear();
+        return non_ready_unicode_equal_to_ascii_string(unicode, str);
+    }
+    if (!PyUnicode_IS_ASCII(unicode))
+        return 0;
+    len = (size_t)PyUnicode_GET_LENGTH(unicode);
+    return strlen(str) == len &&
+           memcmp(PyUnicode_1BYTE_DATA(unicode), str, len) == 0;
+}
+
 
 #define TEST_COND(cond)                         \
     ((cond) ? Py_True : Py_False)
diff --git a/Python/_warnings.c b/Python/_warnings.c
--- a/Python/_warnings.c
+++ b/Python/_warnings.c
@@ -503,7 +503,7 @@
     if (action == NULL)
         goto cleanup;
 
-    if (PyUnicode_CompareWithASCIIString(action, "error") == 0) {
+    if (_PyUnicode_EqualToASCIIString(action, "error")) {
         PyErr_SetObject(category, message);
         goto cleanup;
     }
@@ -511,13 +511,13 @@
     /* Store in the registry that we've been here, *except* when the action
        is "always". */
     rc = 0;
-    if (PyUnicode_CompareWithASCIIString(action, "always") != 0) {
+    if (!_PyUnicode_EqualToASCIIString(action, "always")) {
         if (registry != NULL && registry != Py_None &&
                 PyDict_SetItem(registry, key, Py_True) < 0)
             goto cleanup;
-        else if (PyUnicode_CompareWithASCIIString(action, "ignore") == 0)
+        else if (_PyUnicode_EqualToASCIIString(action, "ignore"))
             goto return_none;
-        else if (PyUnicode_CompareWithASCIIString(action, "once") == 0) {
+        else if (_PyUnicode_EqualToASCIIString(action, "once")) {
             if (registry == NULL || registry == Py_None) {
                 registry = get_once_registry();
                 if (registry == NULL)
@@ -526,12 +526,12 @@
             /* _once_registry[(text, category)] = 1 */
             rc = update_registry(registry, text, category, 0);
         }
-        else if (PyUnicode_CompareWithASCIIString(action, "module") == 0) {
+        else if (_PyUnicode_EqualToASCIIString(action, "module")) {
             /* registry[(text, category, 0)] = 1 */
             if (registry != NULL && registry != Py_None)
                 rc = update_registry(registry, text, category, 0);
         }
-        else if (PyUnicode_CompareWithASCIIString(action, "default") != 0) {
+        else if (!_PyUnicode_EqualToASCIIString(action, "default")) {
             PyErr_Format(PyExc_RuntimeError,
                         "Unrecognized action (%R) in warnings.filters:\n %R",
                         action, item);
@@ -715,7 +715,7 @@
     }
     else {
         *filename = NULL;
-        if (*module != Py_None && PyUnicode_CompareWithASCIIString(*module, "__main__") == 0) {
+        if (*module != Py_None && _PyUnicode_EqualToASCIIString(*module, "__main__")) {
             PyObject *argv = _PySys_GetObjectId(&PyId_argv);
             /* PyList_Check() is needed because sys.argv is set to None during
                Python finalization */
diff --git a/Python/ast.c b/Python/ast.c
--- a/Python/ast.c
+++ b/Python/ast.c
@@ -934,12 +934,12 @@
                int full_checks)
 {
     assert(PyUnicode_Check(name));
-    if (PyUnicode_CompareWithASCIIString(name, "__debug__") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "__debug__")) {
         ast_error(c, n, "assignment to keyword");
         return 1;
     }
-    if (PyUnicode_CompareWithASCIIString(name, "async") == 0 ||
-        PyUnicode_CompareWithASCIIString(name, "await") == 0)
+    if (_PyUnicode_EqualToASCIIString(name, "async") ||
+        _PyUnicode_EqualToASCIIString(name, "await"))
     {
         PyObject *message = PyUnicode_FromString(
             "'async' and 'await' will become reserved keywords"
@@ -963,7 +963,7 @@
     if (full_checks) {
         const char * const *p;
         for (p = FORBIDDEN; *p; p++) {
-            if (PyUnicode_CompareWithASCIIString(name, *p) == 0) {
+            if (_PyUnicode_EqualToASCIIString(name, *p)) {
                 ast_error(c, n, "assignment to keyword");
                 return 1;
             }
diff --git a/Python/compile.c b/Python/compile.c
--- a/Python/compile.c
+++ b/Python/compile.c
@@ -1522,7 +1522,7 @@
 {
     int scope;
     if (c->u->u_scope_type == COMPILER_SCOPE_CLASS &&
-        !PyUnicode_CompareWithASCIIString(name, "__class__"))
+        _PyUnicode_EqualToASCIIString(name, "__class__"))
         return CELL;
     scope = PyST_GetScope(c->u->u_ste, name);
     if (scope == 0) {
@@ -2688,7 +2688,7 @@
     }
 
     if (s->lineno > c->c_future->ff_lineno && s->v.ImportFrom.module &&
-        !PyUnicode_CompareWithASCIIString(s->v.ImportFrom.module, "__future__")) {
+        _PyUnicode_EqualToASCIIString(s->v.ImportFrom.module, "__future__")) {
         Py_DECREF(level);
         Py_DECREF(names);
         return compiler_error(c, "from __future__ imports must occur "
@@ -3027,9 +3027,9 @@
     if (!mangled)
         return 0;
 
-    assert(PyUnicode_CompareWithASCIIString(name, "None") &&
-           PyUnicode_CompareWithASCIIString(name, "True") &&
-           PyUnicode_CompareWithASCIIString(name, "False"));
+    assert(!_PyUnicode_EqualToASCIIString(name, "None") &&
+           !_PyUnicode_EqualToASCIIString(name, "True") &&
+           !_PyUnicode_EqualToASCIIString(name, "False"));
 
     op = 0;
     optype = OP_NAME;
diff --git a/Python/future.c b/Python/future.c
--- a/Python/future.c
+++ b/Python/future.c
@@ -102,7 +102,7 @@
         if (s->kind == ImportFrom_kind) {
             identifier modname = s->v.ImportFrom.module;
             if (modname &&
-                !PyUnicode_CompareWithASCIIString(modname, "__future__")) {
+                _PyUnicode_EqualToASCIIString(modname, "__future__")) {
                 if (done) {
                     PyErr_SetString(PyExc_SyntaxError,
                                     ERR_LATE_FUTURE);
diff --git a/Python/getargs.c b/Python/getargs.c
--- a/Python/getargs.c
+++ b/Python/getargs.c
@@ -1808,7 +1808,7 @@
                 return cleanreturn(0, &freelist);
             }
             for (i = 0; i < len; i++) {
-                if (*kwlist[i] && !PyUnicode_CompareWithASCIIString(key, kwlist[i])) {
+                if (*kwlist[i] && _PyUnicode_EqualToASCIIString(key, kwlist[i])) {
                     match = 1;
                     break;
                 }
diff --git a/Python/import.c b/Python/import.c
--- a/Python/import.c
+++ b/Python/import.c
@@ -936,10 +936,9 @@
 static int
 is_builtin(PyObject *name)
 {
-    int i, cmp;
+    int i;
     for (i = 0; PyImport_Inittab[i].name != NULL; i++) {
-        cmp = PyUnicode_CompareWithASCIIString(name, PyImport_Inittab[i].name);
-        if (cmp == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, PyImport_Inittab[i].name)) {
             if (PyImport_Inittab[i].initfunc == NULL)
                 return -1;
             else
@@ -1059,7 +1058,7 @@
 
     for (p = PyImport_Inittab; p->name != NULL; p++) {
         PyModuleDef *def;
-        if (PyUnicode_CompareWithASCIIString(name, p->name) == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, p->name)) {
             if (p->initfunc == NULL) {
                 /* Cannot re-init internal module ("sys" or "builtins") */
                 mod = PyImport_AddModule(namestr);
@@ -1109,7 +1108,7 @@
     for (p = PyImport_FrozenModules; ; p++) {
         if (p->name == NULL)
             return NULL;
-        if (PyUnicode_CompareWithASCIIString(name, p->name) == 0)
+        if (_PyUnicode_EqualToASCIIString(name, p->name))
             break;
     }
     return p;
@@ -1310,12 +1309,8 @@
         int now_in_importlib;
 
         assert(PyTraceBack_Check(tb));
-        now_in_importlib = (PyUnicode_CompareWithASCIIString(
-                                code->co_filename,
-                                importlib_filename) == 0) ||
-                           (PyUnicode_CompareWithASCIIString(
-                                code->co_filename,
-                                external_filename) == 0);
+        now_in_importlib = _PyUnicode_EqualToASCIIString(code->co_filename, importlib_filename) ||
+                           _PyUnicode_EqualToASCIIString(code->co_filename, external_filename);
         if (now_in_importlib && !in_importlib) {
             /* This is the link to this chunk of importlib tracebacks */
             outer_link = prev_link;
@@ -1324,8 +1319,7 @@
 
         if (in_importlib &&
             (always_trim ||
-             PyUnicode_CompareWithASCIIString(code->co_name,
-                                              remove_frames) == 0)) {
+             _PyUnicode_EqualToASCIIString(code->co_name, remove_frames))) {
             Py_XINCREF(next);
             Py_XSETREF(*outer_link, next);
             prev_link = outer_link;
diff --git a/Python/symtable.c b/Python/symtable.c
--- a/Python/symtable.c
+++ b/Python/symtable.c
@@ -1503,7 +1503,7 @@
         /* Special-case super: it counts as a use of __class__ */
         if (e->v.Name.ctx == Load &&
             st->st_cur->ste_type == FunctionBlock &&
-            !PyUnicode_CompareWithASCIIString(e->v.Name.id, "super")) {
+            _PyUnicode_EqualToASCIIString(e->v.Name.id, "super")) {
             if (!GET_IDENTIFIER(__class__) ||
                 !symtable_add_def(st, __class__, USE))
                 VISIT_QUIT(st, 0);
@@ -1652,7 +1652,7 @@
         store_name = name;
         Py_INCREF(store_name);
     }
-    if (PyUnicode_CompareWithASCIIString(name, "*")) {
+    if (!_PyUnicode_EqualToASCIIString(name, "*")) {
         int r = symtable_add_def(st, store_name, DEF_IMPORT);
         Py_DECREF(store_name);
         return r;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 03:20:16 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 16 Nov 2016 08:20:16 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328701=3A_Replace_PyUnicode=5FCompareWithASCIIString_w?=
 =?utf-8?q?ith?=
Message-ID: <20161116082016.89035.23748.6D341BA9@psf.io>

https://hg.python.org/cpython/rev/72d07d13869a
changeset:   105147:72d07d13869a
branch:      3.6
parent:      105144:80e4cb5888f3
parent:      105146:386c682dcd75
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 16 10:19:20 2016 +0200
summary:
  Issue #28701: Replace PyUnicode_CompareWithASCIIString with _PyUnicode_EqualToASCIIString.

The latter function is more readable, faster and doesn't raise exceptions.

files:
  Doc/c-api/unicode.rst        |   6 +++
  Include/unicodeobject.h      |  11 ++++++
  Modules/_decimal/_decimal.c  |  14 +++---
  Modules/_elementtree.c       |   6 +-
  Modules/_io/textio.c         |   2 +-
  Modules/_io/winconsoleio.c   |   6 +-
  Modules/_lsprof.c            |   2 +-
  Modules/_pickle.c            |   2 +-
  Modules/_sqlite/connection.c |   2 +-
  Modules/_testcapimodule.c    |   2 +-
  Modules/pyexpat.c            |  43 +++++++++++------------
  Objects/longobject.c         |   8 ++--
  Objects/moduleobject.c       |   2 +-
  Objects/typeobject.c         |   4 +-
  Objects/unicodeobject.c      |  35 +++++++++++++++++++
  Python/_warnings.c           |  14 +++---
  Python/ast.c                 |   8 ++--
  Python/compile.c             |  10 ++--
  Python/future.c              |   2 +-
  Python/getargs.c             |   2 +-
  Python/import.c              |  20 +++-------
  Python/symtable.c            |   4 +-
  22 files changed, 125 insertions(+), 80 deletions(-)


diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -1643,6 +1643,9 @@
    Compare two strings and return ``-1``, ``0``, ``1`` for less than, equal, and greater than,
    respectively.
 
+   This function returns ``-1`` upon failure, so one should call
+   :c:func:`PyErr_Occurred` to check for errors.
+
 
 .. c:function:: int PyUnicode_CompareWithASCIIString(PyObject *uni, const char *string)
 
@@ -1651,6 +1654,9 @@
    ASCII-encoded strings, but the function interprets the input string as
    ISO-8859-1 if it contains non-ASCII characters.
 
+   This function returns ``-1`` upon failure, so one should call
+   :c:func:`PyErr_Occurred` to check for errors.
+
 
 .. c:function:: PyObject* PyUnicode_RichCompare(PyObject *left,  PyObject *right,  int op)
 
diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -2048,6 +2048,17 @@
     const char *right           /* ASCII-encoded string */
     );
 
+#ifndef Py_LIMITED_API
+/* Test whether a unicode is equal to ASCII string.  Return 1 if true,
+   0 otherwise.  Return 0 if any argument contains non-ASCII characters.
+   Any error occurs inside will be cleared before return. */
+
+PyAPI_FUNC(int) _PyUnicode_EqualToASCIIString(
+    PyObject *left,
+    const char *right           /* ASCII-encoded string */
+    );
+#endif
+
 /* Rich compare two strings and return one of the following:
 
    - NULL in case an exception was raised
diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c
--- a/Modules/_decimal/_decimal.c
+++ b/Modules/_decimal/_decimal.c
@@ -1118,12 +1118,12 @@
     PyObject *retval;
 
     if (PyUnicode_Check(name)) {
-        if (PyUnicode_CompareWithASCIIString(name, "traps") == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, "traps")) {
             retval = ((PyDecContextObject *)self)->traps;
             Py_INCREF(retval);
             return retval;
         }
-        if (PyUnicode_CompareWithASCIIString(name, "flags") == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, "flags")) {
             retval = ((PyDecContextObject *)self)->flags;
             Py_INCREF(retval);
             return retval;
@@ -1143,10 +1143,10 @@
     }
 
     if (PyUnicode_Check(name)) {
-        if (PyUnicode_CompareWithASCIIString(name, "traps") == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, "traps")) {
             return context_settraps_dict(self, value);
         }
-        if (PyUnicode_CompareWithASCIIString(name, "flags") == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, "flags")) {
             return context_setstatus_dict(self, value);
         }
     }
@@ -2449,14 +2449,14 @@
     tmp = PyTuple_GET_ITEM(dectuple, 2);
     if (PyUnicode_Check(tmp)) {
         /* special */
-        if (PyUnicode_CompareWithASCIIString(tmp, "F") == 0) {
+        if (_PyUnicode_EqualToASCIIString(tmp, "F")) {
             strcat(sign_special, "Inf");
             is_infinite = 1;
         }
-        else if (PyUnicode_CompareWithASCIIString(tmp, "n") == 0) {
+        else if (_PyUnicode_EqualToASCIIString(tmp, "n")) {
             strcat(sign_special, "NaN");
         }
-        else if (PyUnicode_CompareWithASCIIString(tmp, "N") == 0) {
+        else if (_PyUnicode_EqualToASCIIString(tmp, "N")) {
             strcat(sign_special, "sNaN");
         }
         else {
diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c
--- a/Modules/_elementtree.c
+++ b/Modules/_elementtree.c
@@ -3661,11 +3661,11 @@
 {
     if (PyUnicode_Check(nameobj)) {
         PyObject* res;
-        if (PyUnicode_CompareWithASCIIString(nameobj, "entity") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "entity"))
             res = self->entity;
-        else if (PyUnicode_CompareWithASCIIString(nameobj, "target") == 0)
+        else if (_PyUnicode_EqualToASCIIString(nameobj, "target"))
             res = self->target;
-        else if (PyUnicode_CompareWithASCIIString(nameobj, "version") == 0) {
+        else if (_PyUnicode_EqualToASCIIString(nameobj, "version")) {
             return PyUnicode_FromFormat(
                 "Expat %d.%d.%d", XML_MAJOR_VERSION,
                 XML_MINOR_VERSION, XML_MICRO_VERSION);
diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c
--- a/Modules/_io/textio.c
+++ b/Modules/_io/textio.c
@@ -1023,7 +1023,7 @@
         else if (PyUnicode_Check(res)) {
             const encodefuncentry *e = encodefuncs;
             while (e->name != NULL) {
-                if (!PyUnicode_CompareWithASCIIString(res, e->name)) {
+                if (_PyUnicode_EqualToASCIIString(res, e->name)) {
                     self->encodefunc = e->encodefunc;
                     break;
                 }
diff --git a/Modules/_io/winconsoleio.c b/Modules/_io/winconsoleio.c
--- a/Modules/_io/winconsoleio.c
+++ b/Modules/_io/winconsoleio.c
@@ -93,11 +93,11 @@
     }
 
     char m = '\0';
-    if (PyUnicode_CompareWithASCIIString(decoded_upper, "CONIN$") == 0) {
+    if (_PyUnicode_EqualToASCIIString(decoded_upper, "CONIN$")) {
         m = 'r';
-    } else if (PyUnicode_CompareWithASCIIString(decoded_upper, "CONOUT$") == 0) {
+    } else if (_PyUnicode_EqualToASCIIString(decoded_upper, "CONOUT$")) {
         m = 'w';
-    } else if (PyUnicode_CompareWithASCIIString(decoded_upper, "CON") == 0) {
+    } else if (_PyUnicode_EqualToASCIIString(decoded_upper, "CON")) {
         m = 'x';
     }
 
diff --git a/Modules/_lsprof.c b/Modules/_lsprof.c
--- a/Modules/_lsprof.c
+++ b/Modules/_lsprof.c
@@ -181,7 +181,7 @@
             }
         }
         if (modname != NULL) {
-            if (PyUnicode_CompareWithASCIIString(modname, "builtins") != 0) {
+            if (!_PyUnicode_EqualToASCIIString(modname, "builtins")) {
                 PyObject *result;
                 result = PyUnicode_FromFormat("<%U.%s>", modname,
                                               fn->m_ml->ml_name);
diff --git a/Modules/_pickle.c b/Modules/_pickle.c
--- a/Modules/_pickle.c
+++ b/Modules/_pickle.c
@@ -1687,7 +1687,7 @@
     while (PyDict_Next(modules_dict, &i, &module_name, &module)) {
         PyObject *candidate;
         if (PyUnicode_Check(module_name) &&
-            !PyUnicode_CompareWithASCIIString(module_name, "__main__"))
+            _PyUnicode_EqualToASCIIString(module_name, "__main__"))
             continue;
         if (module == Py_None)
             continue;
diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c
--- a/Modules/_sqlite/connection.c
+++ b/Modules/_sqlite/connection.c
@@ -1191,7 +1191,7 @@
             return -1;
         }
         for (candidate = begin_statements; *candidate; candidate++) {
-            if (!PyUnicode_CompareWithASCIIString(uppercase_level, *candidate + 6))
+            if (_PyUnicode_EqualToASCIIString(uppercase_level, *candidate + 6))
                 break;
         }
         Py_DECREF(uppercase_level);
diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c
--- a/Modules/_testcapimodule.c
+++ b/Modules/_testcapimodule.c
@@ -2329,7 +2329,7 @@
     result = PyUnicode_FromFormat(FORMAT, (TYPE)1);                 \
     if (result == NULL)                                             \
         return NULL;                                                \
-    if (PyUnicode_CompareWithASCIIString(result, "1")) {     \
+    if (!_PyUnicode_EqualToASCIIString(result, "1")) {              \
         msg = FORMAT " failed at 1";                                \
         goto Fail;                                                  \
     }                                                               \
diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c
--- a/Modules/pyexpat.c
+++ b/Modules/pyexpat.c
@@ -1244,8 +1244,7 @@
 {
     int i;
     for (i = 0; handler_info[i].name != NULL; i++) {
-        if (PyUnicode_CompareWithASCIIString(
-                name, handler_info[i].name) == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, handler_info[i].name)) {
             return i;
         }
     }
@@ -1283,45 +1282,45 @@
 
     first_char = PyUnicode_READ_CHAR(nameobj, 0);
     if (first_char == 'E') {
-        if (PyUnicode_CompareWithASCIIString(nameobj, "ErrorCode") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "ErrorCode"))
             return PyLong_FromLong((long)
                                   XML_GetErrorCode(self->itself));
-        if (PyUnicode_CompareWithASCIIString(nameobj, "ErrorLineNumber") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "ErrorLineNumber"))
             return PyLong_FromLong((long)
                                   XML_GetErrorLineNumber(self->itself));
-        if (PyUnicode_CompareWithASCIIString(nameobj, "ErrorColumnNumber") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "ErrorColumnNumber"))
             return PyLong_FromLong((long)
                                   XML_GetErrorColumnNumber(self->itself));
-        if (PyUnicode_CompareWithASCIIString(nameobj, "ErrorByteIndex") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "ErrorByteIndex"))
             return PyLong_FromLong((long)
                                   XML_GetErrorByteIndex(self->itself));
     }
     if (first_char == 'C') {
-        if (PyUnicode_CompareWithASCIIString(nameobj, "CurrentLineNumber") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "CurrentLineNumber"))
             return PyLong_FromLong((long)
                                   XML_GetCurrentLineNumber(self->itself));
-        if (PyUnicode_CompareWithASCIIString(nameobj, "CurrentColumnNumber") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "CurrentColumnNumber"))
             return PyLong_FromLong((long)
                                   XML_GetCurrentColumnNumber(self->itself));
-        if (PyUnicode_CompareWithASCIIString(nameobj, "CurrentByteIndex") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "CurrentByteIndex"))
             return PyLong_FromLong((long)
                                   XML_GetCurrentByteIndex(self->itself));
     }
     if (first_char == 'b') {
-        if (PyUnicode_CompareWithASCIIString(nameobj, "buffer_size") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "buffer_size"))
             return PyLong_FromLong((long) self->buffer_size);
-        if (PyUnicode_CompareWithASCIIString(nameobj, "buffer_text") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "buffer_text"))
             return get_pybool(self->buffer != NULL);
-        if (PyUnicode_CompareWithASCIIString(nameobj, "buffer_used") == 0)
+        if (_PyUnicode_EqualToASCIIString(nameobj, "buffer_used"))
             return PyLong_FromLong((long) self->buffer_used);
     }
-    if (PyUnicode_CompareWithASCIIString(nameobj, "namespace_prefixes") == 0)
+    if (_PyUnicode_EqualToASCIIString(nameobj, "namespace_prefixes"))
         return get_pybool(self->ns_prefixes);
-    if (PyUnicode_CompareWithASCIIString(nameobj, "ordered_attributes") == 0)
+    if (_PyUnicode_EqualToASCIIString(nameobj, "ordered_attributes"))
         return get_pybool(self->ordered_attributes);
-    if (PyUnicode_CompareWithASCIIString(nameobj, "specified_attributes") == 0)
+    if (_PyUnicode_EqualToASCIIString(nameobj, "specified_attributes"))
         return get_pybool((long) self->specified_attributes);
-    if (PyUnicode_CompareWithASCIIString(nameobj, "intern") == 0) {
+    if (_PyUnicode_EqualToASCIIString(nameobj, "intern")) {
         if (self->intern == NULL) {
             Py_INCREF(Py_None);
             return Py_None;
@@ -1383,7 +1382,7 @@
         PyErr_SetString(PyExc_RuntimeError, "Cannot delete attribute");
         return -1;
     }
-    if (PyUnicode_CompareWithASCIIString(name, "buffer_text") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "buffer_text")) {
         int b = PyObject_IsTrue(v);
         if (b < 0)
             return -1;
@@ -1405,7 +1404,7 @@
         }
         return 0;
     }
-    if (PyUnicode_CompareWithASCIIString(name, "namespace_prefixes") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "namespace_prefixes")) {
         int b = PyObject_IsTrue(v);
         if (b < 0)
             return -1;
@@ -1413,14 +1412,14 @@
         XML_SetReturnNSTriplet(self->itself, self->ns_prefixes);
         return 0;
     }
-    if (PyUnicode_CompareWithASCIIString(name, "ordered_attributes") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "ordered_attributes")) {
         int b = PyObject_IsTrue(v);
         if (b < 0)
             return -1;
         self->ordered_attributes = b;
         return 0;
     }
-    if (PyUnicode_CompareWithASCIIString(name, "specified_attributes") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "specified_attributes")) {
         int b = PyObject_IsTrue(v);
         if (b < 0)
             return -1;
@@ -1428,7 +1427,7 @@
         return 0;
     }
 
-    if (PyUnicode_CompareWithASCIIString(name, "buffer_size") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "buffer_size")) {
       long new_buffer_size;
       if (!PyLong_Check(v)) {
         PyErr_SetString(PyExc_TypeError, "buffer_size must be an integer");
@@ -1474,7 +1473,7 @@
       return 0;
     }
 
-    if (PyUnicode_CompareWithASCIIString(name, "CharacterDataHandler") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "CharacterDataHandler")) {
         /* If we're changing the character data handler, flush all
          * cached data with the old handler.  Not sure there's a
          * "right" thing to do, though, but this probably won't
diff --git a/Objects/longobject.c b/Objects/longobject.c
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -5180,9 +5180,9 @@
         return NULL;
     }
 
-    if (!PyUnicode_CompareWithASCIIString(byteorder_str, "little"))
+    if (_PyUnicode_EqualToASCIIString(byteorder_str, "little"))
         little_endian = 1;
-    else if (!PyUnicode_CompareWithASCIIString(byteorder_str, "big"))
+    else if (_PyUnicode_EqualToASCIIString(byteorder_str, "big"))
         little_endian = 0;
     else {
         PyErr_SetString(PyExc_ValueError,
@@ -5263,9 +5263,9 @@
         return NULL;
     }
 
-    if (!PyUnicode_CompareWithASCIIString(byteorder_str, "little"))
+    if (_PyUnicode_EqualToASCIIString(byteorder_str, "little"))
         little_endian = 1;
-    else if (!PyUnicode_CompareWithASCIIString(byteorder_str, "big"))
+    else if (_PyUnicode_EqualToASCIIString(byteorder_str, "big"))
         little_endian = 0;
     else {
         PyErr_SetString(PyExc_ValueError,
diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c
--- a/Objects/moduleobject.c
+++ b/Objects/moduleobject.c
@@ -586,7 +586,7 @@
     while (PyDict_Next(d, &pos, &key, &value)) {
         if (value != Py_None && PyUnicode_Check(key)) {
             if (PyUnicode_READ_CHAR(key, 0) != '_' ||
-                PyUnicode_CompareWithASCIIString(key, "__builtins__") != 0)
+                !_PyUnicode_EqualToASCIIString(key, "__builtins__"))
             {
                 if (Py_VerboseFlag > 1) {
                     const char *s = _PyUnicode_AsString(key);
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -2412,7 +2412,7 @@
                 }
                 add_dict++;
             }
-            if (PyUnicode_CompareWithASCIIString(tmp, "__weakref__") == 0) {
+            if (_PyUnicode_EqualToASCIIString(tmp, "__weakref__")) {
                 if (!may_add_weak || add_weak) {
                     PyErr_SetString(PyExc_TypeError,
                         "__weakref__ slot disallowed: "
@@ -2436,7 +2436,7 @@
             if ((add_dict &&
                  _PyUnicode_CompareWithId(tmp, &PyId___dict__) == 0) ||
                 (add_weak &&
-                 PyUnicode_CompareWithASCIIString(tmp, "__weakref__") == 0))
+                 _PyUnicode_EqualToASCIIString(tmp, "__weakref__")))
                 continue;
             tmp =_Py_Mangle(name, tmp);
             if (!tmp) {
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -11067,6 +11067,41 @@
     }
 }
 
+static int
+non_ready_unicode_equal_to_ascii_string(PyObject *unicode, const char *str)
+{
+    size_t i, len;
+    const wchar_t *p;
+    len = (size_t)_PyUnicode_WSTR_LENGTH(unicode);
+    if (strlen(str) != len)
+        return 0;
+    p = _PyUnicode_WSTR(unicode);
+    assert(p);
+    for (i = 0; i < len; i++) {
+        unsigned char c = (unsigned char)str[i];
+        if (c > 128 || p[i] != (wchar_t)c)
+            return 0;
+    }
+    return 1;
+}
+
+int
+_PyUnicode_EqualToASCIIString(PyObject *unicode, const char *str)
+{
+    size_t len;
+    assert(_PyUnicode_CHECK(unicode));
+    if (PyUnicode_READY(unicode) == -1) {
+        /* Memory error or bad data */
+        PyErr_Clear();
+        return non_ready_unicode_equal_to_ascii_string(unicode, str);
+    }
+    if (!PyUnicode_IS_ASCII(unicode))
+        return 0;
+    len = (size_t)PyUnicode_GET_LENGTH(unicode);
+    return strlen(str) == len &&
+           memcmp(PyUnicode_1BYTE_DATA(unicode), str, len) == 0;
+}
+
 
 #define TEST_COND(cond)                         \
     ((cond) ? Py_True : Py_False)
diff --git a/Python/_warnings.c b/Python/_warnings.c
--- a/Python/_warnings.c
+++ b/Python/_warnings.c
@@ -503,7 +503,7 @@
     if (action == NULL)
         goto cleanup;
 
-    if (PyUnicode_CompareWithASCIIString(action, "error") == 0) {
+    if (_PyUnicode_EqualToASCIIString(action, "error")) {
         PyErr_SetObject(category, message);
         goto cleanup;
     }
@@ -511,13 +511,13 @@
     /* Store in the registry that we've been here, *except* when the action
        is "always". */
     rc = 0;
-    if (PyUnicode_CompareWithASCIIString(action, "always") != 0) {
+    if (!_PyUnicode_EqualToASCIIString(action, "always")) {
         if (registry != NULL && registry != Py_None &&
                 PyDict_SetItem(registry, key, Py_True) < 0)
             goto cleanup;
-        else if (PyUnicode_CompareWithASCIIString(action, "ignore") == 0)
+        else if (_PyUnicode_EqualToASCIIString(action, "ignore"))
             goto return_none;
-        else if (PyUnicode_CompareWithASCIIString(action, "once") == 0) {
+        else if (_PyUnicode_EqualToASCIIString(action, "once")) {
             if (registry == NULL || registry == Py_None) {
                 registry = get_once_registry();
                 if (registry == NULL)
@@ -526,12 +526,12 @@
             /* _once_registry[(text, category)] = 1 */
             rc = update_registry(registry, text, category, 0);
         }
-        else if (PyUnicode_CompareWithASCIIString(action, "module") == 0) {
+        else if (_PyUnicode_EqualToASCIIString(action, "module")) {
             /* registry[(text, category, 0)] = 1 */
             if (registry != NULL && registry != Py_None)
                 rc = update_registry(registry, text, category, 0);
         }
-        else if (PyUnicode_CompareWithASCIIString(action, "default") != 0) {
+        else if (!_PyUnicode_EqualToASCIIString(action, "default")) {
             PyErr_Format(PyExc_RuntimeError,
                         "Unrecognized action (%R) in warnings.filters:\n %R",
                         action, item);
@@ -715,7 +715,7 @@
     }
     else {
         *filename = NULL;
-        if (*module != Py_None && PyUnicode_CompareWithASCIIString(*module, "__main__") == 0) {
+        if (*module != Py_None && _PyUnicode_EqualToASCIIString(*module, "__main__")) {
             PyObject *argv = _PySys_GetObjectId(&PyId_argv);
             /* PyList_Check() is needed because sys.argv is set to None during
                Python finalization */
diff --git a/Python/ast.c b/Python/ast.c
--- a/Python/ast.c
+++ b/Python/ast.c
@@ -934,12 +934,12 @@
                int full_checks)
 {
     assert(PyUnicode_Check(name));
-    if (PyUnicode_CompareWithASCIIString(name, "__debug__") == 0) {
+    if (_PyUnicode_EqualToASCIIString(name, "__debug__")) {
         ast_error(c, n, "assignment to keyword");
         return 1;
     }
-    if (PyUnicode_CompareWithASCIIString(name, "async") == 0 ||
-        PyUnicode_CompareWithASCIIString(name, "await") == 0)
+    if (_PyUnicode_EqualToASCIIString(name, "async") ||
+        _PyUnicode_EqualToASCIIString(name, "await"))
     {
         PyObject *message = PyUnicode_FromString(
             "'async' and 'await' will become reserved keywords"
@@ -963,7 +963,7 @@
     if (full_checks) {
         const char * const *p;
         for (p = FORBIDDEN; *p; p++) {
-            if (PyUnicode_CompareWithASCIIString(name, *p) == 0) {
+            if (_PyUnicode_EqualToASCIIString(name, *p)) {
                 ast_error(c, n, "assignment to keyword");
                 return 1;
             }
diff --git a/Python/compile.c b/Python/compile.c
--- a/Python/compile.c
+++ b/Python/compile.c
@@ -1522,7 +1522,7 @@
 {
     int scope;
     if (c->u->u_scope_type == COMPILER_SCOPE_CLASS &&
-        !PyUnicode_CompareWithASCIIString(name, "__class__"))
+        _PyUnicode_EqualToASCIIString(name, "__class__"))
         return CELL;
     scope = PyST_GetScope(c->u->u_ste, name);
     if (scope == 0) {
@@ -2688,7 +2688,7 @@
     }
 
     if (s->lineno > c->c_future->ff_lineno && s->v.ImportFrom.module &&
-        !PyUnicode_CompareWithASCIIString(s->v.ImportFrom.module, "__future__")) {
+        _PyUnicode_EqualToASCIIString(s->v.ImportFrom.module, "__future__")) {
         Py_DECREF(level);
         Py_DECREF(names);
         return compiler_error(c, "from __future__ imports must occur "
@@ -3027,9 +3027,9 @@
     if (!mangled)
         return 0;
 
-    assert(PyUnicode_CompareWithASCIIString(name, "None") &&
-           PyUnicode_CompareWithASCIIString(name, "True") &&
-           PyUnicode_CompareWithASCIIString(name, "False"));
+    assert(!_PyUnicode_EqualToASCIIString(name, "None") &&
+           !_PyUnicode_EqualToASCIIString(name, "True") &&
+           !_PyUnicode_EqualToASCIIString(name, "False"));
 
     op = 0;
     optype = OP_NAME;
diff --git a/Python/future.c b/Python/future.c
--- a/Python/future.c
+++ b/Python/future.c
@@ -102,7 +102,7 @@
         if (s->kind == ImportFrom_kind) {
             identifier modname = s->v.ImportFrom.module;
             if (modname &&
-                !PyUnicode_CompareWithASCIIString(modname, "__future__")) {
+                _PyUnicode_EqualToASCIIString(modname, "__future__")) {
                 if (done) {
                     PyErr_SetString(PyExc_SyntaxError,
                                     ERR_LATE_FUTURE);
diff --git a/Python/getargs.c b/Python/getargs.c
--- a/Python/getargs.c
+++ b/Python/getargs.c
@@ -1808,7 +1808,7 @@
                 return cleanreturn(0, &freelist);
             }
             for (i = 0; i < len; i++) {
-                if (*kwlist[i] && !PyUnicode_CompareWithASCIIString(key, kwlist[i])) {
+                if (*kwlist[i] && _PyUnicode_EqualToASCIIString(key, kwlist[i])) {
                     match = 1;
                     break;
                 }
diff --git a/Python/import.c b/Python/import.c
--- a/Python/import.c
+++ b/Python/import.c
@@ -936,10 +936,9 @@
 static int
 is_builtin(PyObject *name)
 {
-    int i, cmp;
+    int i;
     for (i = 0; PyImport_Inittab[i].name != NULL; i++) {
-        cmp = PyUnicode_CompareWithASCIIString(name, PyImport_Inittab[i].name);
-        if (cmp == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, PyImport_Inittab[i].name)) {
             if (PyImport_Inittab[i].initfunc == NULL)
                 return -1;
             else
@@ -1059,7 +1058,7 @@
 
     for (p = PyImport_Inittab; p->name != NULL; p++) {
         PyModuleDef *def;
-        if (PyUnicode_CompareWithASCIIString(name, p->name) == 0) {
+        if (_PyUnicode_EqualToASCIIString(name, p->name)) {
             if (p->initfunc == NULL) {
                 /* Cannot re-init internal module ("sys" or "builtins") */
                 mod = PyImport_AddModule(namestr);
@@ -1109,7 +1108,7 @@
     for (p = PyImport_FrozenModules; ; p++) {
         if (p->name == NULL)
             return NULL;
-        if (PyUnicode_CompareWithASCIIString(name, p->name) == 0)
+        if (_PyUnicode_EqualToASCIIString(name, p->name))
             break;
     }
     return p;
@@ -1310,12 +1309,8 @@
         int now_in_importlib;
 
         assert(PyTraceBack_Check(tb));
-        now_in_importlib = (PyUnicode_CompareWithASCIIString(
-                                code->co_filename,
-                                importlib_filename) == 0) ||
-                           (PyUnicode_CompareWithASCIIString(
-                                code->co_filename,
-                                external_filename) == 0);
+        now_in_importlib = _PyUnicode_EqualToASCIIString(code->co_filename, importlib_filename) ||
+                           _PyUnicode_EqualToASCIIString(code->co_filename, external_filename);
         if (now_in_importlib && !in_importlib) {
             /* This is the link to this chunk of importlib tracebacks */
             outer_link = prev_link;
@@ -1324,8 +1319,7 @@
 
         if (in_importlib &&
             (always_trim ||
-             PyUnicode_CompareWithASCIIString(code->co_name,
-                                              remove_frames) == 0)) {
+             _PyUnicode_EqualToASCIIString(code->co_name, remove_frames))) {
             Py_XINCREF(next);
             Py_XSETREF(*outer_link, next);
             prev_link = outer_link;
diff --git a/Python/symtable.c b/Python/symtable.c
--- a/Python/symtable.c
+++ b/Python/symtable.c
@@ -1503,7 +1503,7 @@
         /* Special-case super: it counts as a use of __class__ */
         if (e->v.Name.ctx == Load &&
             st->st_cur->ste_type == FunctionBlock &&
-            !PyUnicode_CompareWithASCIIString(e->v.Name.id, "super")) {
+            _PyUnicode_EqualToASCIIString(e->v.Name.id, "super")) {
             if (!GET_IDENTIFIER(__class__) ||
                 !symtable_add_def(st, __class__, USE))
                 VISIT_QUIT(st, 0);
@@ -1652,7 +1652,7 @@
         store_name = name;
         Py_INCREF(store_name);
     }
-    if (PyUnicode_CompareWithASCIIString(name, "*")) {
+    if (!_PyUnicode_EqualToASCIIString(name, "*")) {
         int r = symtable_add_def(st, store_name, DEF_IMPORT);
         Py_DECREF(store_name);
         return r;

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Wed Nov 16 04:08:28 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Wed, 16 Nov 2016 09:08:28 +0000
Subject: [Python-checkins] Daily reference leaks (2faac299acaa): sum=-2
Message-ID: <20161116090828.25322.90396.4EFA249B@psf.io>

results for 2faac299acaa on branch "default"
--------------------------------------------

test_collections leaked [-7, 1, 0] memory blocks, sum=-6
test_functools leaked [0, 3, 1] memory blocks, sum=4


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflognGWhuH', '--timeout', '7200']

From python-checkins at python.org  Wed Nov 16 08:41:46 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 16 Nov 2016 13:41:46 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328701=3A_Replace_=5FPyUnicode=5FCompareWithId_w?=
 =?utf-8?q?ith_=5FPyUnicode=5FEqualToASCIIId=2E?=
Message-ID: <20161116134146.89269.40256.32E60EFC@psf.io>

https://hg.python.org/cpython/rev/765013f71bc4
changeset:   105151:765013f71bc4
parent:      105148:6f0f77333da5
parent:      105150:ff3dacc98b3a
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 16 15:41:31 2016 +0200
summary:
  Issue #28701: Replace _PyUnicode_CompareWithId with _PyUnicode_EqualToASCIIId.

The latter function is more readable, faster and doesn't raise exceptions.

Based on patch by Xiang Zhang.

files:
  Include/unicodeobject.h |  19 ++++++++++++++
  Objects/typeobject.c    |  12 ++++----
  Objects/unicodeobject.c |  38 +++++++++++++++++++++++++++++
  Python/errors.c         |   2 +-
  Python/pythonrun.c      |   2 +-
  5 files changed, 65 insertions(+), 8 deletions(-)


diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -2037,12 +2037,31 @@
     );
 
 #ifndef Py_LIMITED_API
+/* Compare a string with an identifier and return -1, 0, 1 for less than,
+   equal, and greater than, respectively.
+   Raise an exception and return -1 on error. */
+
 PyAPI_FUNC(int) _PyUnicode_CompareWithId(
     PyObject *left,             /* Left string */
     _Py_Identifier *right       /* Right identifier */
     );
+
+/* Test whether a unicode is equal to ASCII identifier.  Return 1 if true,
+   0 otherwise.  Return 0 if any argument contains non-ASCII characters.
+   Any error occurs inside will be cleared before return. */
+
+PyAPI_FUNC(int) _PyUnicode_EqualToASCIIId(
+    PyObject *left,             /* Left string */
+    _Py_Identifier *right       /* Right identifier */
+    );
 #endif
 
+/* Compare a Unicode object with C string and return -1, 0, 1 for less than,
+   equal, and greater than, respectively.  It is best to pass only
+   ASCII-encoded strings, but the function interprets the input string as
+   ISO-8859-1 if it contains non-ASCII characters.
+   Raise an exception and return -1 on error. */
+
 PyAPI_FUNC(int) PyUnicode_CompareWithASCIIString(
     PyObject *left,
     const char *right           /* ASCII-encoded string */
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -863,7 +863,7 @@
         return NULL;
     }
 
-    if (mod != NULL && _PyUnicode_CompareWithId(mod, &PyId_builtins))
+    if (mod != NULL && !_PyUnicode_EqualToASCIIId(mod, &PyId_builtins))
         rtn = PyUnicode_FromFormat("<class '%U.%U'>", mod, name);
     else
         rtn = PyUnicode_FromFormat("<class '%s'>", type->tp_name);
@@ -2403,7 +2403,7 @@
             if (!valid_identifier(tmp))
                 goto error;
             assert(PyUnicode_Check(tmp));
-            if (_PyUnicode_CompareWithId(tmp, &PyId___dict__) == 0) {
+            if (_PyUnicode_EqualToASCIIId(tmp, &PyId___dict__)) {
                 if (!may_add_dict || add_dict) {
                     PyErr_SetString(PyExc_TypeError,
                         "__dict__ slot disallowed: "
@@ -2434,7 +2434,7 @@
         for (i = j = 0; i < nslots; i++) {
             tmp = PyTuple_GET_ITEM(slots, i);
             if ((add_dict &&
-                 _PyUnicode_CompareWithId(tmp, &PyId___dict__) == 0) ||
+                 _PyUnicode_EqualToASCIIId(tmp, &PyId___dict__)) ||
                 (add_weak &&
                  _PyUnicode_EqualToASCIIString(tmp, "__weakref__")))
                 continue;
@@ -3538,7 +3538,7 @@
         Py_XDECREF(mod);
         return NULL;
     }
-    if (mod != NULL && _PyUnicode_CompareWithId(mod, &PyId_builtins))
+    if (mod != NULL && !_PyUnicode_EqualToASCIIId(mod, &PyId_builtins))
         rtn = PyUnicode_FromFormat("<%U.%U object at %p>", mod, name, self);
     else
         rtn = PyUnicode_FromFormat("<%s object at %p>",
@@ -7238,7 +7238,7 @@
        (i.e. super, or a subclass), not the class of su->obj. */
     if (PyUnicode_Check(name) &&
         PyUnicode_GET_LENGTH(name) == 9 &&
-        _PyUnicode_CompareWithId(name, &PyId___class__) == 0)
+        _PyUnicode_EqualToASCIIId(name, &PyId___class__))
         goto skip;
 
     mro = starttype->tp_mro;
@@ -7450,7 +7450,7 @@
         for (i = 0; i < n; i++) {
             PyObject *name = PyTuple_GET_ITEM(co->co_freevars, i);
             assert(PyUnicode_Check(name));
-            if (!_PyUnicode_CompareWithId(name, &PyId___class__)) {
+            if (_PyUnicode_EqualToASCIIId(name, &PyId___class__)) {
                 Py_ssize_t index = co->co_nlocals +
                     PyTuple_GET_SIZE(co->co_cellvars) + i;
                 PyObject *cell = f->f_localsplus[index];
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -11083,6 +11083,44 @@
            memcmp(PyUnicode_1BYTE_DATA(unicode), str, len) == 0;
 }
 
+int
+_PyUnicode_EqualToASCIIId(PyObject *left, _Py_Identifier *right)
+{
+    PyObject *right_uni;
+    Py_hash_t hash;
+
+    assert(_PyUnicode_CHECK(left));
+    assert(right->string);
+
+    if (PyUnicode_READY(left) == -1) {
+        /* memory error or bad data */
+        PyErr_Clear();
+        return non_ready_unicode_equal_to_ascii_string(left, right->string);
+    }
+
+    if (!PyUnicode_IS_ASCII(left))
+        return 0;
+
+    right_uni = _PyUnicode_FromId(right);       /* borrowed */
+    if (right_uni == NULL) {
+        /* memory error or bad data */
+        PyErr_Clear();
+        return _PyUnicode_EqualToASCIIString(left, right->string);
+    }
+
+    if (left == right_uni)
+        return 1;
+
+    if (PyUnicode_CHECK_INTERNED(left))
+        return 0;
+
+    assert(_PyUnicode_HASH(right_uni) != 1);
+    hash = _PyUnicode_HASH(left);
+    if (hash != -1 && hash != _PyUnicode_HASH(right_uni))
+        return 0;
+
+    return unicode_compare_eq(left, right_uni);
+}
 
 #define TEST_COND(cond)                         \
     ((cond) ? Py_True : Py_False)
diff --git a/Python/errors.c b/Python/errors.c
--- a/Python/errors.c
+++ b/Python/errors.c
@@ -984,7 +984,7 @@
             goto done;
     }
     else {
-        if (_PyUnicode_CompareWithId(moduleName, &PyId_builtins) != 0) {
+        if (!_PyUnicode_EqualToASCIIId(moduleName, &PyId_builtins)) {
             if (PyFile_WriteObject(moduleName, f, Py_PRINT_RAW) < 0)
                 goto done;
             if (PyFile_WriteString(".", f) < 0)
diff --git a/Python/pythonrun.c b/Python/pythonrun.c
--- a/Python/pythonrun.c
+++ b/Python/pythonrun.c
@@ -751,7 +751,7 @@
             err = PyFile_WriteString("<unknown>", f);
         }
         else {
-            if (_PyUnicode_CompareWithId(moduleName, &PyId_builtins) != 0)
+            if (!_PyUnicode_EqualToASCIIId(moduleName, &PyId_builtins))
             {
                 err = PyFile_WriteObject(moduleName, f, Py_PRINT_RAW);
                 err += PyFile_WriteString(".", f);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 08:41:46 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 16 Nov 2016 13:41:46 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328701=3A_Replace_=5FPyUnicode=5FCompareWithId_with_?=
 =?utf-8?q?=5FPyUnicode=5FEqualToASCIIId=2E?=
Message-ID: <20161116134146.7703.23039.1794BC51@psf.io>

https://hg.python.org/cpython/rev/ff3dacc98b3a
changeset:   105150:ff3dacc98b3a
branch:      3.6
parent:      105147:72d07d13869a
parent:      105149:faf04a995031
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 16 15:41:11 2016 +0200
summary:
  Issue #28701: Replace _PyUnicode_CompareWithId with _PyUnicode_EqualToASCIIId.

The latter function is more readable, faster and doesn't raise exceptions.

Based on patch by Xiang Zhang.

files:
  Include/unicodeobject.h |  19 ++++++++++++++
  Objects/typeobject.c    |  12 ++++----
  Objects/unicodeobject.c |  38 +++++++++++++++++++++++++++++
  Python/errors.c         |   2 +-
  Python/pythonrun.c      |   2 +-
  5 files changed, 65 insertions(+), 8 deletions(-)


diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -2037,12 +2037,31 @@
     );
 
 #ifndef Py_LIMITED_API
+/* Compare a string with an identifier and return -1, 0, 1 for less than,
+   equal, and greater than, respectively.
+   Raise an exception and return -1 on error. */
+
 PyAPI_FUNC(int) _PyUnicode_CompareWithId(
     PyObject *left,             /* Left string */
     _Py_Identifier *right       /* Right identifier */
     );
+
+/* Test whether a unicode is equal to ASCII identifier.  Return 1 if true,
+   0 otherwise.  Return 0 if any argument contains non-ASCII characters.
+   Any error occurs inside will be cleared before return. */
+
+PyAPI_FUNC(int) _PyUnicode_EqualToASCIIId(
+    PyObject *left,             /* Left string */
+    _Py_Identifier *right       /* Right identifier */
+    );
 #endif
 
+/* Compare a Unicode object with C string and return -1, 0, 1 for less than,
+   equal, and greater than, respectively.  It is best to pass only
+   ASCII-encoded strings, but the function interprets the input string as
+   ISO-8859-1 if it contains non-ASCII characters.
+   Raise an exception and return -1 on error. */
+
 PyAPI_FUNC(int) PyUnicode_CompareWithASCIIString(
     PyObject *left,
     const char *right           /* ASCII-encoded string */
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -863,7 +863,7 @@
         return NULL;
     }
 
-    if (mod != NULL && _PyUnicode_CompareWithId(mod, &PyId_builtins))
+    if (mod != NULL && !_PyUnicode_EqualToASCIIId(mod, &PyId_builtins))
         rtn = PyUnicode_FromFormat("<class '%U.%U'>", mod, name);
     else
         rtn = PyUnicode_FromFormat("<class '%s'>", type->tp_name);
@@ -2403,7 +2403,7 @@
             if (!valid_identifier(tmp))
                 goto error;
             assert(PyUnicode_Check(tmp));
-            if (_PyUnicode_CompareWithId(tmp, &PyId___dict__) == 0) {
+            if (_PyUnicode_EqualToASCIIId(tmp, &PyId___dict__)) {
                 if (!may_add_dict || add_dict) {
                     PyErr_SetString(PyExc_TypeError,
                         "__dict__ slot disallowed: "
@@ -2434,7 +2434,7 @@
         for (i = j = 0; i < nslots; i++) {
             tmp = PyTuple_GET_ITEM(slots, i);
             if ((add_dict &&
-                 _PyUnicode_CompareWithId(tmp, &PyId___dict__) == 0) ||
+                 _PyUnicode_EqualToASCIIId(tmp, &PyId___dict__)) ||
                 (add_weak &&
                  _PyUnicode_EqualToASCIIString(tmp, "__weakref__")))
                 continue;
@@ -3538,7 +3538,7 @@
         Py_XDECREF(mod);
         return NULL;
     }
-    if (mod != NULL && _PyUnicode_CompareWithId(mod, &PyId_builtins))
+    if (mod != NULL && !_PyUnicode_EqualToASCIIId(mod, &PyId_builtins))
         rtn = PyUnicode_FromFormat("<%U.%U object at %p>", mod, name, self);
     else
         rtn = PyUnicode_FromFormat("<%s object at %p>",
@@ -7238,7 +7238,7 @@
        (i.e. super, or a subclass), not the class of su->obj. */
     if (PyUnicode_Check(name) &&
         PyUnicode_GET_LENGTH(name) == 9 &&
-        _PyUnicode_CompareWithId(name, &PyId___class__) == 0)
+        _PyUnicode_EqualToASCIIId(name, &PyId___class__))
         goto skip;
 
     mro = starttype->tp_mro;
@@ -7450,7 +7450,7 @@
         for (i = 0; i < n; i++) {
             PyObject *name = PyTuple_GET_ITEM(co->co_freevars, i);
             assert(PyUnicode_Check(name));
-            if (!_PyUnicode_CompareWithId(name, &PyId___class__)) {
+            if (_PyUnicode_EqualToASCIIId(name, &PyId___class__)) {
                 Py_ssize_t index = co->co_nlocals +
                     PyTuple_GET_SIZE(co->co_cellvars) + i;
                 PyObject *cell = f->f_localsplus[index];
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -11102,6 +11102,44 @@
            memcmp(PyUnicode_1BYTE_DATA(unicode), str, len) == 0;
 }
 
+int
+_PyUnicode_EqualToASCIIId(PyObject *left, _Py_Identifier *right)
+{
+    PyObject *right_uni;
+    Py_hash_t hash;
+
+    assert(_PyUnicode_CHECK(left));
+    assert(right->string);
+
+    if (PyUnicode_READY(left) == -1) {
+        /* memory error or bad data */
+        PyErr_Clear();
+        return non_ready_unicode_equal_to_ascii_string(left, right->string);
+    }
+
+    if (!PyUnicode_IS_ASCII(left))
+        return 0;
+
+    right_uni = _PyUnicode_FromId(right);       /* borrowed */
+    if (right_uni == NULL) {
+        /* memory error or bad data */
+        PyErr_Clear();
+        return _PyUnicode_EqualToASCIIString(left, right->string);
+    }
+
+    if (left == right_uni)
+        return 1;
+
+    if (PyUnicode_CHECK_INTERNED(left))
+        return 0;
+
+    assert(_PyUnicode_HASH(right_uni) != 1);
+    hash = _PyUnicode_HASH(left);
+    if (hash != -1 && hash != _PyUnicode_HASH(right_uni))
+        return 0;
+
+    return unicode_compare_eq(left, right_uni);
+}
 
 #define TEST_COND(cond)                         \
     ((cond) ? Py_True : Py_False)
diff --git a/Python/errors.c b/Python/errors.c
--- a/Python/errors.c
+++ b/Python/errors.c
@@ -984,7 +984,7 @@
             goto done;
     }
     else {
-        if (_PyUnicode_CompareWithId(moduleName, &PyId_builtins) != 0) {
+        if (!_PyUnicode_EqualToASCIIId(moduleName, &PyId_builtins)) {
             if (PyFile_WriteObject(moduleName, f, Py_PRINT_RAW) < 0)
                 goto done;
             if (PyFile_WriteString(".", f) < 0)
diff --git a/Python/pythonrun.c b/Python/pythonrun.c
--- a/Python/pythonrun.c
+++ b/Python/pythonrun.c
@@ -751,7 +751,7 @@
             err = PyFile_WriteString("<unknown>", f);
         }
         else {
-            if (_PyUnicode_CompareWithId(moduleName, &PyId_builtins) != 0)
+            if (!_PyUnicode_EqualToASCIIId(moduleName, &PyId_builtins))
             {
                 err = PyFile_WriteObject(moduleName, f, Py_PRINT_RAW);
                 err += PyFile_WriteString(".", f);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 08:41:46 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 16 Nov 2016 13:41:46 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NzAx?=
 =?utf-8?q?=3A_Replace_=5FPyUnicode=5FCompareWithId_with_=5FPyUnicode=5FEq?=
 =?utf-8?q?ualToASCIIId=2E?=
Message-ID: <20161116134146.7756.86836.ACFF1C21@psf.io>

https://hg.python.org/cpython/rev/faf04a995031
changeset:   105149:faf04a995031
branch:      3.5
parent:      105146:386c682dcd75
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 16 15:40:39 2016 +0200
summary:
  Issue #28701: Replace _PyUnicode_CompareWithId with _PyUnicode_EqualToASCIIId.

The latter function is more readable, faster and doesn't raise exceptions.

Based on patch by Xiang Zhang.

files:
  Include/unicodeobject.h |  19 ++++++++++++++
  Objects/typeobject.c    |  12 ++++----
  Objects/unicodeobject.c |  38 +++++++++++++++++++++++++++++
  Python/errors.c         |   2 +-
  Python/pythonrun.c      |   2 +-
  5 files changed, 65 insertions(+), 8 deletions(-)


diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -2000,12 +2000,31 @@
     );
 
 #ifndef Py_LIMITED_API
+/* Compare a string with an identifier and return -1, 0, 1 for less than,
+   equal, and greater than, respectively.
+   Raise an exception and return -1 on error. */
+
 PyAPI_FUNC(int) _PyUnicode_CompareWithId(
     PyObject *left,             /* Left string */
     _Py_Identifier *right       /* Right identifier */
     );
+
+/* Test whether a unicode is equal to ASCII identifier.  Return 1 if true,
+   0 otherwise.  Return 0 if any argument contains non-ASCII characters.
+   Any error occurs inside will be cleared before return. */
+
+PyAPI_FUNC(int) _PyUnicode_EqualToASCIIId(
+    PyObject *left,             /* Left string */
+    _Py_Identifier *right       /* Right identifier */
+    );
 #endif
 
+/* Compare a Unicode object with C string and return -1, 0, 1 for less than,
+   equal, and greater than, respectively.  It is best to pass only
+   ASCII-encoded strings, but the function interprets the input string as
+   ISO-8859-1 if it contains non-ASCII characters.
+   Raise an exception and return -1 on error. */
+
 PyAPI_FUNC(int) PyUnicode_CompareWithASCIIString(
     PyObject *left,
     const char *right           /* ASCII-encoded string */
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -858,7 +858,7 @@
         return NULL;
     }
 
-    if (mod != NULL && _PyUnicode_CompareWithId(mod, &PyId_builtins))
+    if (mod != NULL && !_PyUnicode_EqualToASCIIId(mod, &PyId_builtins))
         rtn = PyUnicode_FromFormat("<class '%U.%U'>", mod, name);
     else
         rtn = PyUnicode_FromFormat("<class '%s'>", type->tp_name);
@@ -2386,7 +2386,7 @@
             if (!valid_identifier(tmp))
                 goto error;
             assert(PyUnicode_Check(tmp));
-            if (_PyUnicode_CompareWithId(tmp, &PyId___dict__) == 0) {
+            if (_PyUnicode_EqualToASCIIId(tmp, &PyId___dict__)) {
                 if (!may_add_dict || add_dict) {
                     PyErr_SetString(PyExc_TypeError,
                         "__dict__ slot disallowed: "
@@ -2417,7 +2417,7 @@
         for (i = j = 0; i < nslots; i++) {
             tmp = PyTuple_GET_ITEM(slots, i);
             if ((add_dict &&
-                 _PyUnicode_CompareWithId(tmp, &PyId___dict__) == 0) ||
+                 _PyUnicode_EqualToASCIIId(tmp, &PyId___dict__)) ||
                 (add_weak &&
                  _PyUnicode_EqualToASCIIString(tmp, "__weakref__")))
                 continue;
@@ -3490,7 +3490,7 @@
         Py_XDECREF(mod);
         return NULL;
     }
-    if (mod != NULL && _PyUnicode_CompareWithId(mod, &PyId_builtins))
+    if (mod != NULL && !_PyUnicode_EqualToASCIIId(mod, &PyId_builtins))
         rtn = PyUnicode_FromFormat("<%U.%U object at %p>", mod, name, self);
     else
         rtn = PyUnicode_FromFormat("<%s object at %p>",
@@ -7107,7 +7107,7 @@
        (i.e. super, or a subclass), not the class of su->obj. */
     if (PyUnicode_Check(name) &&
         PyUnicode_GET_LENGTH(name) == 9 &&
-        _PyUnicode_CompareWithId(name, &PyId___class__) == 0)
+        _PyUnicode_EqualToASCIIId(name, &PyId___class__))
         goto skip;
 
     mro = starttype->tp_mro;
@@ -7319,7 +7319,7 @@
         for (i = 0; i < n; i++) {
             PyObject *name = PyTuple_GET_ITEM(co->co_freevars, i);
             assert(PyUnicode_Check(name));
-            if (!_PyUnicode_CompareWithId(name, &PyId___class__)) {
+            if (_PyUnicode_EqualToASCIIId(name, &PyId___class__)) {
                 Py_ssize_t index = co->co_nlocals +
                     PyTuple_GET_SIZE(co->co_cellvars) + i;
                 PyObject *cell = f->f_localsplus[index];
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -10869,6 +10869,44 @@
            memcmp(PyUnicode_1BYTE_DATA(unicode), str, len) == 0;
 }
 
+int
+_PyUnicode_EqualToASCIIId(PyObject *left, _Py_Identifier *right)
+{
+    PyObject *right_uni;
+    Py_hash_t hash;
+
+    assert(_PyUnicode_CHECK(left));
+    assert(right->string);
+
+    if (PyUnicode_READY(left) == -1) {
+        /* memory error or bad data */
+        PyErr_Clear();
+        return non_ready_unicode_equal_to_ascii_string(left, right->string);
+    }
+
+    if (!PyUnicode_IS_ASCII(left))
+        return 0;
+
+    right_uni = _PyUnicode_FromId(right);       /* borrowed */
+    if (right_uni == NULL) {
+        /* memory error or bad data */
+        PyErr_Clear();
+        return _PyUnicode_EqualToASCIIString(left, right->string);
+    }
+
+    if (left == right_uni)
+        return 1;
+
+    if (PyUnicode_CHECK_INTERNED(left))
+        return 0;
+
+    assert(_PyUnicode_HASH(right_uni) != 1);
+    hash = _PyUnicode_HASH(left);
+    if (hash != -1 && hash != _PyUnicode_HASH(right_uni))
+        return 0;
+
+    return unicode_compare_eq(left, right_uni);
+}
 
 #define TEST_COND(cond)                         \
     ((cond) ? Py_True : Py_False)
diff --git a/Python/errors.c b/Python/errors.c
--- a/Python/errors.c
+++ b/Python/errors.c
@@ -934,7 +934,7 @@
             goto done;
     }
     else {
-        if (_PyUnicode_CompareWithId(moduleName, &PyId_builtins) != 0) {
+        if (!_PyUnicode_EqualToASCIIId(moduleName, &PyId_builtins)) {
             if (PyFile_WriteObject(moduleName, f, Py_PRINT_RAW) < 0)
                 goto done;
             if (PyFile_WriteString(".", f) < 0)
diff --git a/Python/pythonrun.c b/Python/pythonrun.c
--- a/Python/pythonrun.c
+++ b/Python/pythonrun.c
@@ -747,7 +747,7 @@
             err = PyFile_WriteString("<unknown>", f);
         }
         else {
-            if (_PyUnicode_CompareWithId(moduleName, &PyId_builtins) != 0)
+            if (!_PyUnicode_EqualToASCIIId(moduleName, &PyId_builtins))
             {
                 err = PyFile_WriteObject(moduleName, f, Py_PRINT_RAW);
                 err += PyFile_WriteString(".", f);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 08:57:10 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 16 Nov 2016 13:57:10 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzIxNDQ5?=
 =?utf-8?q?=3A_Removed_private_function_=5FPyUnicode=5FCompareWithId=2E?=
Message-ID: <20161116135710.12078.25339.CF6A3A83@psf.io>

https://hg.python.org/cpython/rev/b995a6950139
changeset:   105152:b995a6950139
branch:      3.6
parent:      105150:ff3dacc98b3a
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 16 15:56:27 2016 +0200
summary:
  Issue #21449: Removed private function _PyUnicode_CompareWithId.

files:
  Include/unicodeobject.h |  9 ---------
  Objects/unicodeobject.c |  9 ---------
  2 files changed, 0 insertions(+), 18 deletions(-)


diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -2037,15 +2037,6 @@
     );
 
 #ifndef Py_LIMITED_API
-/* Compare a string with an identifier and return -1, 0, 1 for less than,
-   equal, and greater than, respectively.
-   Raise an exception and return -1 on error. */
-
-PyAPI_FUNC(int) _PyUnicode_CompareWithId(
-    PyObject *left,             /* Left string */
-    _Py_Identifier *right       /* Right identifier */
-    );
-
 /* Test whether a unicode is equal to ASCII identifier.  Return 1 if true,
    0 otherwise.  Return 0 if any argument contains non-ASCII characters.
    Any error occurs inside will be cleared before return. */
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -11012,15 +11012,6 @@
 }
 
 int
-_PyUnicode_CompareWithId(PyObject *left, _Py_Identifier *right)
-{
-    PyObject *right_str = _PyUnicode_FromId(right);   /* borrowed */
-    if (right_str == NULL)
-        return -1;
-    return PyUnicode_Compare(left, right_str);
-}
-
-int
 PyUnicode_CompareWithASCIIString(PyObject* uni, const char* str)
 {
     Py_ssize_t i;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 08:57:10 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 16 Nov 2016 13:57:10 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2321449=3A_Removed_private_function_=5FPyUnicode?=
 =?utf-8?q?=5FCompareWithId=2E?=
Message-ID: <20161116135710.7901.24161.E158D88A@psf.io>

https://hg.python.org/cpython/rev/9b053d3c74dc
changeset:   105153:9b053d3c74dc
parent:      105151:765013f71bc4
parent:      105152:b995a6950139
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 16 15:56:50 2016 +0200
summary:
  Issue #21449: Removed private function _PyUnicode_CompareWithId.

files:
  Include/unicodeobject.h |  9 ---------
  Objects/unicodeobject.c |  9 ---------
  2 files changed, 0 insertions(+), 18 deletions(-)


diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -2037,15 +2037,6 @@
     );
 
 #ifndef Py_LIMITED_API
-/* Compare a string with an identifier and return -1, 0, 1 for less than,
-   equal, and greater than, respectively.
-   Raise an exception and return -1 on error. */
-
-PyAPI_FUNC(int) _PyUnicode_CompareWithId(
-    PyObject *left,             /* Left string */
-    _Py_Identifier *right       /* Right identifier */
-    );
-
 /* Test whether a unicode is equal to ASCII identifier.  Return 1 if true,
    0 otherwise.  Return 0 if any argument contains non-ASCII characters.
    Any error occurs inside will be cleared before return. */
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -10993,15 +10993,6 @@
 }
 
 int
-_PyUnicode_CompareWithId(PyObject *left, _Py_Identifier *right)
-{
-    PyObject *right_str = _PyUnicode_FromId(right);   /* borrowed */
-    if (right_str == NULL)
-        return -1;
-    return PyUnicode_Compare(left, right_str);
-}
-
-int
 PyUnicode_CompareWithASCIIString(PyObject* uni, const char* str)
 {
     Py_ssize_t i;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 09:13:28 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 16 Nov 2016 14:13:28 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Fixed_an_off-by-one_error_in_=5FPyUnicode=5FEqualToASCIIString?=
 =?utf-8?q?_=28issue_=2328701=29=2E?=
Message-ID: <20161116141328.93528.61593.B69C913B@psf.io>

https://hg.python.org/cpython/rev/1369e51182b7
changeset:   105155:1369e51182b7
branch:      3.6
parent:      105152:b995a6950139
parent:      105154:b607f835f170
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 16 16:12:56 2016 +0200
summary:
  Fixed an off-by-one error in _PyUnicode_EqualToASCIIString (issue #28701).

files:
  Objects/unicodeobject.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -11070,7 +11070,7 @@
     assert(p);
     for (i = 0; i < len; i++) {
         unsigned char c = (unsigned char)str[i];
-        if (c > 128 || p[i] != (wchar_t)c)
+        if (c >= 128 || p[i] != (wchar_t)c)
             return 0;
     }
     return 1;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 09:13:28 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 16 Nov 2016 14:13:28 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fixed_an_off-b?=
 =?utf-8?q?y-one_error_in_=5FPyUnicode=5FEqualToASCIIString_=28issue_=2328?=
 =?utf-8?b?NzAxKS4=?=
Message-ID: <20161116141328.106536.37741.B0C97503@psf.io>

https://hg.python.org/cpython/rev/b607f835f170
changeset:   105154:b607f835f170
branch:      3.5
parent:      105149:faf04a995031
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 16 16:12:34 2016 +0200
summary:
  Fixed an off-by-one error in _PyUnicode_EqualToASCIIString (issue #28701).

files:
  Objects/unicodeobject.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -10846,7 +10846,7 @@
     assert(p);
     for (i = 0; i < len; i++) {
         unsigned char c = (unsigned char)str[i];
-        if (c > 128 || p[i] != (wchar_t)c)
+        if (c >= 128 || p[i] != (wchar_t)c)
             return 0;
     }
     return 1;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 09:13:28 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 16 Nov 2016 14:13:28 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Fixed_an_off-by-one_error_in_=5FPyUnicode=5FEqualToASCII?=
 =?utf-8?q?String_=28issue_=2328701=29=2E?=
Message-ID: <20161116141328.11937.82160.667973CF@psf.io>

https://hg.python.org/cpython/rev/ba14f8b61bd8
changeset:   105156:ba14f8b61bd8
parent:      105153:9b053d3c74dc
parent:      105155:1369e51182b7
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 16 16:13:13 2016 +0200
summary:
  Fixed an off-by-one error in _PyUnicode_EqualToASCIIString (issue #28701).

files:
  Objects/unicodeobject.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -11051,7 +11051,7 @@
     assert(p);
     for (i = 0; i < len; i++) {
         unsigned char c = (unsigned char)str[i];
-        if (c > 128 || p[i] != (wchar_t)c)
+        if (c >= 128 || p[i] != (wchar_t)c)
             return 0;
     }
     return 1;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 10:49:30 2016
From: python-checkins at python.org (kushal.das)
Date: Wed, 16 Nov 2016 15:49:30 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogQ2xvc2VzICMyODcx?=
 =?utf-8?q?3_uses_OSError_in_the_tutorial?=
Message-ID: <20161116154930.107052.16258.93D84228@psf.io>

https://hg.python.org/cpython/rev/3375c111d1ff
changeset:   105157:3375c111d1ff
branch:      3.6
parent:      105155:1369e51182b7
user:        Kushal Das <kushaldas at gmail.com>
date:        Wed Nov 16 21:13:43 2016 +0530
summary:
  Closes #28713 uses OSError in the tutorial

files:
  Doc/tutorial/errors.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst
--- a/Doc/tutorial/errors.rst
+++ b/Doc/tutorial/errors.rst
@@ -174,7 +174,7 @@
    for arg in sys.argv[1:]:
        try:
            f = open(arg, 'r')
-       except IOError:
+       except OSError:
            print('cannot open', arg)
        else:
            print(arg, 'has', len(f.readlines()), 'lines')

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 10:49:32 2016
From: python-checkins at python.org (kushal.das)
Date: Wed, 16 Nov 2016 15:49:32 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Closes_=2328713_uses_OSError_in_the_tutorial?=
Message-ID: <20161116154930.7841.9805.E98FA461@psf.io>

https://hg.python.org/cpython/rev/15e5e476e4a1
changeset:   105158:15e5e476e4a1
parent:      105156:ba14f8b61bd8
parent:      105157:3375c111d1ff
user:        Kushal Das <kushaldas at gmail.com>
date:        Wed Nov 16 21:17:54 2016 +0530
summary:
  Closes #28713 uses OSError in the tutorial

files:
  Doc/tutorial/errors.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst
--- a/Doc/tutorial/errors.rst
+++ b/Doc/tutorial/errors.rst
@@ -174,7 +174,7 @@
    for arg in sys.argv[1:]:
        try:
            f = open(arg, 'r')
-       except IOError:
+       except OSError:
            print('cannot open', arg)
        else:
            print(arg, 'has', len(f.readlines()), 'lines')

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Wed Nov 16 12:17:12 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Wed, 16 Nov 2016 17:17:12 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python 2.7 2016-11-16
Message-ID: <88d4fa96-842e-47f9-956d-51f3aa7814a4@irsmsx151.ger.corp.intel.com>

Results for project Python 2.7, build date 2016-11-16 03:47:45 +0000
commit:		ea91bb92c28b
previous commit:	6ca91a14a555
revision date:	2016-11-16 02:21:35 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.11%          1.01%          4.56%             7.07%
:-)             pybench      0.17%         -0.02%          5.73%             3.98%
:-(            regex_v8      0.56%          0.04%         -2.16%            10.99%
:-)               nbody      0.09%          0.00%          8.42%             3.92%
:-)        json_dump_v2      0.31%          0.18%          3.64%             9.81%
:-|      normal_startup      0.59%         -0.51%         -0.90%             2.54%
:-)             ssbench      0.17%         -0.16%          2.30%             2.23%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-2-7-2016-11-16/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Wed Nov 16 12:19:30 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Wed, 16 Nov 2016 17:19:30 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python Default
 2016-11-16
Message-ID: <17571c13-ae0a-49ac-980e-7629e3dfebe0@irsmsx151.ger.corp.intel.com>

Results for project Python default, build date 2016-11-16 03:01:42 +0000
commit:		2faac299acaa
previous commit:	35f510158490
revision date:	2016-11-16 02:18:54 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.14%          2.32%          5.67%            17.29%
:-)             pybench      0.77%          0.56%          5.72%             5.38%
:-(            regex_v8      3.72%         -0.11%         -3.82%             4.54%
:-|               nbody      0.21%          0.08%          1.50%             4.69%
:-(        json_dump_v2      0.30%         -0.26%         -4.67%            10.14%
:-|      normal_startup      1.25%          1.20%          1.83%             5.32%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-11-16/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Wed Nov 16 13:03:21 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 16 Nov 2016 18:03:21 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NzAx?=
 =?utf-8?q?=3A_=5FPyUnicode=5FEqualToASCIIId_and_=5FPyUnicode=5FEqualToASC?=
 =?utf-8?q?IIString_now?=
Message-ID: <20161116180321.106810.84253.C340C9BC@psf.io>

https://hg.python.org/cpython/rev/6dd22ed7140e
changeset:   105159:6dd22ed7140e
branch:      3.6
parent:      105157:3375c111d1ff
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 16 20:02:44 2016 +0200
summary:
  Issue #28701: _PyUnicode_EqualToASCIIId and _PyUnicode_EqualToASCIIString now
require ASCII right argument and assert this condition in debug build.

files:
  Include/unicodeobject.h |   4 ++--
  Objects/unicodeobject.c |  11 +++++++++++
  2 files changed, 13 insertions(+), 2 deletions(-)


diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -2038,7 +2038,7 @@
 
 #ifndef Py_LIMITED_API
 /* Test whether a unicode is equal to ASCII identifier.  Return 1 if true,
-   0 otherwise.  Return 0 if any argument contains non-ASCII characters.
+   0 otherwise.  The right argument must be ASCII identifier.
    Any error occurs inside will be cleared before return. */
 
 PyAPI_FUNC(int) _PyUnicode_EqualToASCIIId(
@@ -2060,7 +2060,7 @@
 
 #ifndef Py_LIMITED_API
 /* Test whether a unicode is equal to ASCII string.  Return 1 if true,
-   0 otherwise.  Return 0 if any argument contains non-ASCII characters.
+   0 otherwise.  The right argument must be ASCII-encoded string.
    Any error occurs inside will be cleared before return. */
 
 PyAPI_FUNC(int) _PyUnicode_EqualToASCIIString(
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -11081,6 +11081,12 @@
 {
     size_t len;
     assert(_PyUnicode_CHECK(unicode));
+    assert(str);
+#ifndef NDEBUG
+    for (const char *p = str; *p; p++) {
+        assert((unsigned char)*p < 128);
+    }
+#endif
     if (PyUnicode_READY(unicode) == -1) {
         /* Memory error or bad data */
         PyErr_Clear();
@@ -11101,6 +11107,11 @@
 
     assert(_PyUnicode_CHECK(left));
     assert(right->string);
+#ifndef NDEBUG
+    for (const char *p = right->string; *p; p++) {
+        assert((unsigned char)*p < 128);
+    }
+#endif
 
     if (PyUnicode_READY(left) == -1) {
         /* memory error or bad data */

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 13:03:21 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 16 Nov 2016 18:03:21 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328701=3A_=5FPyUnicode=5FEqualToASCIIId_and_=5FP?=
 =?utf-8?q?yUnicode=5FEqualToASCIIString_now?=
Message-ID: <20161116180321.106780.68318.95795775@psf.io>

https://hg.python.org/cpython/rev/44874b20e612
changeset:   105160:44874b20e612
parent:      105158:15e5e476e4a1
parent:      105159:6dd22ed7140e
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 16 20:03:03 2016 +0200
summary:
  Issue #28701: _PyUnicode_EqualToASCIIId and _PyUnicode_EqualToASCIIString now
require ASCII right argument and assert this condition in debug build.

files:
  Include/unicodeobject.h |   4 ++--
  Objects/unicodeobject.c |  11 +++++++++++
  2 files changed, 13 insertions(+), 2 deletions(-)


diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -2038,7 +2038,7 @@
 
 #ifndef Py_LIMITED_API
 /* Test whether a unicode is equal to ASCII identifier.  Return 1 if true,
-   0 otherwise.  Return 0 if any argument contains non-ASCII characters.
+   0 otherwise.  The right argument must be ASCII identifier.
    Any error occurs inside will be cleared before return. */
 
 PyAPI_FUNC(int) _PyUnicode_EqualToASCIIId(
@@ -2060,7 +2060,7 @@
 
 #ifndef Py_LIMITED_API
 /* Test whether a unicode is equal to ASCII string.  Return 1 if true,
-   0 otherwise.  Return 0 if any argument contains non-ASCII characters.
+   0 otherwise.  The right argument must be ASCII-encoded string.
    Any error occurs inside will be cleared before return. */
 
 PyAPI_FUNC(int) _PyUnicode_EqualToASCIIString(
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -11062,6 +11062,12 @@
 {
     size_t len;
     assert(_PyUnicode_CHECK(unicode));
+    assert(str);
+#ifndef NDEBUG
+    for (const char *p = str; *p; p++) {
+        assert((unsigned char)*p < 128);
+    }
+#endif
     if (PyUnicode_READY(unicode) == -1) {
         /* Memory error or bad data */
         PyErr_Clear();
@@ -11082,6 +11088,11 @@
 
     assert(_PyUnicode_CHECK(left));
     assert(right->string);
+#ifndef NDEBUG
+    for (const char *p = right->string; *p; p++) {
+        assert((unsigned char)*p < 128);
+    }
+#endif
 
     if (PyUnicode_READY(left) == -1) {
         /* memory error or bad data */

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 18:16:47 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 16 Nov 2016 23:16:47 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NzIx?=
 =?utf-8?q?=3A_Fix_asynchronous_generators_aclose=28=29_and_athrow=28=29?=
Message-ID: <20161116231647.93606.49957.AFD6E365@psf.io>

https://hg.python.org/cpython/rev/0f12a1d3a737
changeset:   105161:0f12a1d3a737
branch:      3.6
parent:      105159:6dd22ed7140e
user:        Yury Selivanov <yury at magic.io>
date:        Wed Nov 16 18:16:17 2016 -0500
summary:
  Issue #28721: Fix asynchronous generators aclose() and athrow()

files:
  Lib/test/test_asyncgen.py |  140 ++++++++++++++++++++++++++
  Misc/NEWS                 |    3 +
  Objects/genobject.c       |   14 ++-
  3 files changed, 154 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_asyncgen.py b/Lib/test/test_asyncgen.py
--- a/Lib/test/test_asyncgen.py
+++ b/Lib/test/test_asyncgen.py
@@ -450,6 +450,41 @@
 
         self.loop.run_until_complete(run())
 
+    def test_async_gen_asyncio_anext_06(self):
+        DONE = 0
+
+        # test synchronous generators
+        def foo():
+            try:
+                yield
+            except:
+                pass
+        g = foo()
+        g.send(None)
+        with self.assertRaises(StopIteration):
+            g.send(None)
+
+        # now with asynchronous generators
+
+        async def gen():
+            nonlocal DONE
+            try:
+                yield
+            except:
+                pass
+            DONE = 1
+
+        async def run():
+            nonlocal DONE
+            g = gen()
+            await g.asend(None)
+            with self.assertRaises(StopAsyncIteration):
+                await g.asend(None)
+            DONE += 10
+
+        self.loop.run_until_complete(run())
+        self.assertEqual(DONE, 11)
+
     def test_async_gen_asyncio_anext_tuple(self):
         async def foo():
             try:
@@ -594,6 +629,76 @@
         self.loop.run_until_complete(run())
         self.assertEqual(DONE, 1)
 
+    def test_async_gen_asyncio_aclose_10(self):
+        DONE = 0
+
+        # test synchronous generators
+        def foo():
+            try:
+                yield
+            except:
+                pass
+        g = foo()
+        g.send(None)
+        g.close()
+
+        # now with asynchronous generators
+
+        async def gen():
+            nonlocal DONE
+            try:
+                yield
+            except:
+                pass
+            DONE = 1
+
+        async def run():
+            nonlocal DONE
+            g = gen()
+            await g.asend(None)
+            await g.aclose()
+            DONE += 10
+
+        self.loop.run_until_complete(run())
+        self.assertEqual(DONE, 11)
+
+    def test_async_gen_asyncio_aclose_11(self):
+        DONE = 0
+
+        # test synchronous generators
+        def foo():
+            try:
+                yield
+            except:
+                pass
+            yield
+        g = foo()
+        g.send(None)
+        with self.assertRaisesRegex(RuntimeError, 'ignored GeneratorExit'):
+            g.close()
+
+        # now with asynchronous generators
+
+        async def gen():
+            nonlocal DONE
+            try:
+                yield
+            except:
+                pass
+            yield
+            DONE += 1
+
+        async def run():
+            nonlocal DONE
+            g = gen()
+            await g.asend(None)
+            with self.assertRaisesRegex(RuntimeError, 'ignored GeneratorExit'):
+                await g.aclose()
+            DONE += 10
+
+        self.loop.run_until_complete(run())
+        self.assertEqual(DONE, 10)
+
     def test_async_gen_asyncio_asend_01(self):
         DONE = 0
 
@@ -801,6 +906,41 @@
             self.loop.run_until_complete(run())
         self.assertEqual(DONE, 1)
 
+    def test_async_gen_asyncio_athrow_03(self):
+        DONE = 0
+
+        # test synchronous generators
+        def foo():
+            try:
+                yield
+            except:
+                pass
+        g = foo()
+        g.send(None)
+        with self.assertRaises(StopIteration):
+            g.throw(ValueError)
+
+        # now with asynchronous generators
+
+        async def gen():
+            nonlocal DONE
+            try:
+                yield
+            except:
+                pass
+            DONE = 1
+
+        async def run():
+            nonlocal DONE
+            g = gen()
+            await g.asend(None)
+            with self.assertRaises(StopAsyncIteration):
+                await g.athrow(ValueError)
+            DONE += 10
+
+        self.loop.run_until_complete(run())
+        self.assertEqual(DONE, 11)
+
     def test_async_gen_asyncio_athrow_tuple(self):
         async def gen():
             try:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -31,6 +31,9 @@
 
 - Issue #26182: Fix a refleak in code that raises DeprecationWarning.
 
+- Issue #28721: Fix asynchronous generators aclose() and athrow() to
+  handle StopAsyncIteration propagation properly.
+
 Library
 -------
 
diff --git a/Objects/genobject.c b/Objects/genobject.c
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -1931,9 +1931,17 @@
     return NULL;
 
 check_error:
-    if (PyErr_ExceptionMatches(PyExc_StopAsyncIteration)
-        || PyErr_ExceptionMatches(PyExc_GeneratorExit)
-    ) {
+    if (PyErr_ExceptionMatches(PyExc_StopAsyncIteration)) {
+        o->agt_state = AWAITABLE_STATE_CLOSED;
+        if (o->agt_args == NULL) {
+            /* when aclose() is called we don't want to propagate
+               StopAsyncIteration; just raise StopIteration, signalling
+               that 'aclose()' is done. */
+            PyErr_Clear();
+            PyErr_SetNone(PyExc_StopIteration);
+        }
+    }
+    else if (PyErr_ExceptionMatches(PyExc_GeneratorExit)) {
         o->agt_state = AWAITABLE_STATE_CLOSED;
         PyErr_Clear();          /* ignore these errors */
         PyErr_SetNone(PyExc_StopIteration);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 18:16:47 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 16 Nov 2016 23:16:47 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg3MjEp?=
Message-ID: <20161116231647.93412.61933.354F5388@psf.io>

https://hg.python.org/cpython/rev/22cd78026ad1
changeset:   105162:22cd78026ad1
parent:      105160:44874b20e612
parent:      105161:0f12a1d3a737
user:        Yury Selivanov <yury at magic.io>
date:        Wed Nov 16 18:16:32 2016 -0500
summary:
  Merge 3.6 (issue #28721)

files:
  Lib/test/test_asyncgen.py |  140 ++++++++++++++++++++++++++
  Misc/NEWS                 |    3 +
  Objects/genobject.c       |   14 ++-
  3 files changed, 154 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_asyncgen.py b/Lib/test/test_asyncgen.py
--- a/Lib/test/test_asyncgen.py
+++ b/Lib/test/test_asyncgen.py
@@ -450,6 +450,41 @@
 
         self.loop.run_until_complete(run())
 
+    def test_async_gen_asyncio_anext_06(self):
+        DONE = 0
+
+        # test synchronous generators
+        def foo():
+            try:
+                yield
+            except:
+                pass
+        g = foo()
+        g.send(None)
+        with self.assertRaises(StopIteration):
+            g.send(None)
+
+        # now with asynchronous generators
+
+        async def gen():
+            nonlocal DONE
+            try:
+                yield
+            except:
+                pass
+            DONE = 1
+
+        async def run():
+            nonlocal DONE
+            g = gen()
+            await g.asend(None)
+            with self.assertRaises(StopAsyncIteration):
+                await g.asend(None)
+            DONE += 10
+
+        self.loop.run_until_complete(run())
+        self.assertEqual(DONE, 11)
+
     def test_async_gen_asyncio_anext_tuple(self):
         async def foo():
             try:
@@ -594,6 +629,76 @@
         self.loop.run_until_complete(run())
         self.assertEqual(DONE, 1)
 
+    def test_async_gen_asyncio_aclose_10(self):
+        DONE = 0
+
+        # test synchronous generators
+        def foo():
+            try:
+                yield
+            except:
+                pass
+        g = foo()
+        g.send(None)
+        g.close()
+
+        # now with asynchronous generators
+
+        async def gen():
+            nonlocal DONE
+            try:
+                yield
+            except:
+                pass
+            DONE = 1
+
+        async def run():
+            nonlocal DONE
+            g = gen()
+            await g.asend(None)
+            await g.aclose()
+            DONE += 10
+
+        self.loop.run_until_complete(run())
+        self.assertEqual(DONE, 11)
+
+    def test_async_gen_asyncio_aclose_11(self):
+        DONE = 0
+
+        # test synchronous generators
+        def foo():
+            try:
+                yield
+            except:
+                pass
+            yield
+        g = foo()
+        g.send(None)
+        with self.assertRaisesRegex(RuntimeError, 'ignored GeneratorExit'):
+            g.close()
+
+        # now with asynchronous generators
+
+        async def gen():
+            nonlocal DONE
+            try:
+                yield
+            except:
+                pass
+            yield
+            DONE += 1
+
+        async def run():
+            nonlocal DONE
+            g = gen()
+            await g.asend(None)
+            with self.assertRaisesRegex(RuntimeError, 'ignored GeneratorExit'):
+                await g.aclose()
+            DONE += 10
+
+        self.loop.run_until_complete(run())
+        self.assertEqual(DONE, 10)
+
     def test_async_gen_asyncio_asend_01(self):
         DONE = 0
 
@@ -801,6 +906,41 @@
             self.loop.run_until_complete(run())
         self.assertEqual(DONE, 1)
 
+    def test_async_gen_asyncio_athrow_03(self):
+        DONE = 0
+
+        # test synchronous generators
+        def foo():
+            try:
+                yield
+            except:
+                pass
+        g = foo()
+        g.send(None)
+        with self.assertRaises(StopIteration):
+            g.throw(ValueError)
+
+        # now with asynchronous generators
+
+        async def gen():
+            nonlocal DONE
+            try:
+                yield
+            except:
+                pass
+            DONE = 1
+
+        async def run():
+            nonlocal DONE
+            g = gen()
+            await g.asend(None)
+            with self.assertRaises(StopAsyncIteration):
+                await g.athrow(ValueError)
+            DONE += 10
+
+        self.loop.run_until_complete(run())
+        self.assertEqual(DONE, 11)
+
     def test_async_gen_asyncio_athrow_tuple(self):
         async def gen():
             try:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -122,6 +122,9 @@
 
 - Issue #26182: Fix a refleak in code that raises DeprecationWarning.
 
+- Issue #28721: Fix asynchronous generators aclose() and athrow() to
+  handle StopAsyncIteration propagation properly.
+
 Library
 -------
 
diff --git a/Objects/genobject.c b/Objects/genobject.c
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -1931,9 +1931,17 @@
     return NULL;
 
 check_error:
-    if (PyErr_ExceptionMatches(PyExc_StopAsyncIteration)
-        || PyErr_ExceptionMatches(PyExc_GeneratorExit)
-    ) {
+    if (PyErr_ExceptionMatches(PyExc_StopAsyncIteration)) {
+        o->agt_state = AWAITABLE_STATE_CLOSED;
+        if (o->agt_args == NULL) {
+            /* when aclose() is called we don't want to propagate
+               StopAsyncIteration; just raise StopIteration, signalling
+               that 'aclose()' is done. */
+            PyErr_Clear();
+            PyErr_SetNone(PyExc_StopIteration);
+        }
+    }
+    else if (PyErr_ExceptionMatches(PyExc_GeneratorExit)) {
         o->agt_state = AWAITABLE_STATE_CLOSED;
         PyErr_Clear();          /* ignore these errors */
         PyErr_SetNone(PyExc_StopIteration);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 18:25:53 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 16 Nov 2016 23:25:53 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NzIw?=
 =?utf-8?q?=3A_Add_collections=2Eabc=2EAsyncGenerator=2E?=
Message-ID: <20161116232553.8510.57023.A2B52830@psf.io>

https://hg.python.org/cpython/rev/ae1dba7e7d04
changeset:   105163:ae1dba7e7d04
branch:      3.6
parent:      105161:0f12a1d3a737
user:        Yury Selivanov <yury at magic.io>
date:        Wed Nov 16 18:25:04 2016 -0500
summary:
  Issue #28720: Add collections.abc.AsyncGenerator.

files:
  Doc/library/collections.abc.rst |   8 ++
  Doc/whatsnew/3.6.rst            |   4 +
  Lib/_collections_abc.py         |  59 ++++++++++++++-
  Lib/test/test_collections.py    |  84 ++++++++++++++++++++-
  Misc/NEWS                       |   2 +
  5 files changed, 155 insertions(+), 2 deletions(-)


diff --git a/Doc/library/collections.abc.rst b/Doc/library/collections.abc.rst
--- a/Doc/library/collections.abc.rst
+++ b/Doc/library/collections.abc.rst
@@ -92,6 +92,7 @@
 :class:`Coroutine`         :class:`Awaitable`     ``send``, ``throw``     ``close``
 :class:`AsyncIterable`                            ``__aiter__``
 :class:`AsyncIterator`     :class:`AsyncIterable` ``__anext__``           ``__aiter__``
+:class:`AsyncGenerator`    :class:`AsyncIterator` ``asend``, ``athrow``   ``aclose``, ``__aiter__``, ``__anext__``
 ========================== ====================== ======================= ====================================================
 
 
@@ -222,6 +223,13 @@
 
    .. versionadded:: 3.5
 
+.. class:: Generator
+
+   ABC for asynchronous generator classes that implement the protocol
+   defined in :pep:`525` and :pep:`492`.
+
+   .. versionadded:: 3.6
+
 
 These ABCs allow us to ask classes or instances if they provide
 particular functionality, for example::
diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -912,6 +912,10 @@
 iterable classes that also provide the :meth:`__reversed__`.
 (Contributed by Ivan Levkivskyi in :issue:`25987`.)
 
+The new :class:`~collections.abc.AsyncGenerator` abstract base class represents
+asynchronous generators.
+(Contributed by Yury Selivanov in :issue:`28720`.)
+
 The :func:`~collections.namedtuple` function now accepts an optional
 keyword argument *module*, which, when specified, is used for
 the ``__module__`` attribute of the returned named tuple class.
diff --git a/Lib/_collections_abc.py b/Lib/_collections_abc.py
--- a/Lib/_collections_abc.py
+++ b/Lib/_collections_abc.py
@@ -9,7 +9,8 @@
 from abc import ABCMeta, abstractmethod
 import sys
 
-__all__ = ["Awaitable", "Coroutine", "AsyncIterable", "AsyncIterator",
+__all__ = ["Awaitable", "Coroutine",
+           "AsyncIterable", "AsyncIterator", "AsyncGenerator",
            "Hashable", "Iterable", "Iterator", "Generator", "Reversible",
            "Sized", "Container", "Callable", "Collection",
            "Set", "MutableSet",
@@ -59,6 +60,11 @@
 coroutine = type(_coro)
 _coro.close()  # Prevent ResourceWarning
 del _coro
+## asynchronous generator ##
+async def _ag(): yield
+_ag = _ag()
+async_generator = type(_ag)
+del _ag
 
 
 ### ONE-TRICK PONIES ###
@@ -183,6 +189,57 @@
         return NotImplemented
 
 
+class AsyncGenerator(AsyncIterator):
+
+    __slots__ = ()
+
+    async def __anext__(self):
+        """Return the next item from the asynchronous generator.
+        When exhausted, raise StopAsyncIteration.
+        """
+        return await self.asend(None)
+
+    @abstractmethod
+    async def asend(self, value):
+        """Send a value into the asynchronous generator.
+        Return next yielded value or raise StopAsyncIteration.
+        """
+        raise StopAsyncIteration
+
+    @abstractmethod
+    async def athrow(self, typ, val=None, tb=None):
+        """Raise an exception in the asynchronous generator.
+        Return next yielded value or raise StopAsyncIteration.
+        """
+        if val is None:
+            if tb is None:
+                raise typ
+            val = typ()
+        if tb is not None:
+            val = val.with_traceback(tb)
+        raise val
+
+    async def aclose(self):
+        """Raise GeneratorExit inside coroutine.
+        """
+        try:
+            await self.athrow(GeneratorExit)
+        except (GeneratorExit, StopAsyncIteration):
+            pass
+        else:
+            raise RuntimeError("asynchronous generator ignored GeneratorExit")
+
+    @classmethod
+    def __subclasshook__(cls, C):
+        if cls is AsyncGenerator:
+            return _check_methods(C, '__aiter__', '__anext__',
+                                  'asend', 'athrow', 'aclose')
+        return NotImplemented
+
+
+AsyncGenerator.register(async_generator)
+
+
 class Iterable(metaclass=ABCMeta):
 
     __slots__ = ()
diff --git a/Lib/test/test_collections.py b/Lib/test/test_collections.py
--- a/Lib/test/test_collections.py
+++ b/Lib/test/test_collections.py
@@ -19,7 +19,8 @@
 from collections import UserDict, UserString, UserList
 from collections import ChainMap
 from collections import deque
-from collections.abc import Awaitable, Coroutine, AsyncIterator, AsyncIterable
+from collections.abc import Awaitable, Coroutine
+from collections.abc import AsyncIterator, AsyncIterable, AsyncGenerator
 from collections.abc import Hashable, Iterable, Iterator, Generator, Reversible
 from collections.abc import Sized, Container, Callable, Collection
 from collections.abc import Set, MutableSet
@@ -959,6 +960,87 @@
 
         self.assertRaises(RuntimeError, IgnoreGeneratorExit().close)
 
+    def test_AsyncGenerator(self):
+        class NonAGen1:
+            def __aiter__(self): return self
+            def __anext__(self): return None
+            def aclose(self): pass
+            def athrow(self, typ, val=None, tb=None): pass
+
+        class NonAGen2:
+            def __aiter__(self): return self
+            def __anext__(self): return None
+            def aclose(self): pass
+            def asend(self, value): return value
+
+        class NonAGen3:
+            def aclose(self): pass
+            def asend(self, value): return value
+            def athrow(self, typ, val=None, tb=None): pass
+
+        non_samples = [
+            None, 42, 3.14, 1j, b"", "", (), [], {}, set(),
+            iter(()), iter([]), NonAGen1(), NonAGen2(), NonAGen3()]
+        for x in non_samples:
+            self.assertNotIsInstance(x, AsyncGenerator)
+            self.assertFalse(issubclass(type(x), AsyncGenerator), repr(type(x)))
+
+        class Gen:
+            def __aiter__(self): return self
+            async def __anext__(self): return None
+            async def aclose(self): pass
+            async def asend(self, value): return value
+            async def athrow(self, typ, val=None, tb=None): pass
+
+        class MinimalAGen(AsyncGenerator):
+            async def asend(self, value):
+                return value
+            async def athrow(self, typ, val=None, tb=None):
+                await super().athrow(typ, val, tb)
+
+        async def gen():
+            yield 1
+
+        samples = [gen(), Gen(), MinimalAGen()]
+        for x in samples:
+            self.assertIsInstance(x, AsyncIterator)
+            self.assertIsInstance(x, AsyncGenerator)
+            self.assertTrue(issubclass(type(x), AsyncGenerator), repr(type(x)))
+        self.validate_abstract_methods(AsyncGenerator, 'asend', 'athrow')
+
+        def run_async(coro):
+            result = None
+            while True:
+                try:
+                    coro.send(None)
+                except StopIteration as ex:
+                    result = ex.args[0] if ex.args else None
+                    break
+            return result
+
+        # mixin tests
+        mgen = MinimalAGen()
+        self.assertIs(mgen, mgen.__aiter__())
+        self.assertIs(run_async(mgen.asend(None)), run_async(mgen.__anext__()))
+        self.assertEqual(2, run_async(mgen.asend(2)))
+        self.assertIsNone(run_async(mgen.aclose()))
+        with self.assertRaises(ValueError):
+            run_async(mgen.athrow(ValueError))
+
+        class FailOnClose(AsyncGenerator):
+            async def asend(self, value): return value
+            async def athrow(self, *args): raise ValueError
+
+        with self.assertRaises(ValueError):
+            run_async(FailOnClose().aclose())
+
+        class IgnoreGeneratorExit(AsyncGenerator):
+            async def asend(self, value): return value
+            async def athrow(self, *args): pass
+
+        with self.assertRaises(RuntimeError):
+            run_async(IgnoreGeneratorExit().aclose())
+
     def test_Sized(self):
         non_samples = [None, 42, 3.14, 1j,
                        _test_gen(),
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -69,6 +69,8 @@
 - Issue #28704: Fix create_unix_server to support Path-like objects 
   (PEP 519).
 
+- Issue #28720: Add collections.abc.AsyncGenerator.
+
 Documentation
 -------------
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 18:25:54 2016
From: python-checkins at python.org (yury.selivanov)
Date: Wed, 16 Nov 2016 23:25:54 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg3MjAp?=
Message-ID: <20161116232553.93319.22067.1932568E@psf.io>

https://hg.python.org/cpython/rev/fb9c8fdef3ec
changeset:   105164:fb9c8fdef3ec
parent:      105162:22cd78026ad1
parent:      105163:ae1dba7e7d04
user:        Yury Selivanov <yury at magic.io>
date:        Wed Nov 16 18:25:48 2016 -0500
summary:
  Merge 3.6 (issue #28720)

files:
  Doc/library/collections.abc.rst |   8 ++
  Doc/whatsnew/3.6.rst            |   4 +
  Lib/_collections_abc.py         |  59 ++++++++++++++-
  Lib/test/test_collections.py    |  84 ++++++++++++++++++++-
  4 files changed, 153 insertions(+), 2 deletions(-)


diff --git a/Doc/library/collections.abc.rst b/Doc/library/collections.abc.rst
--- a/Doc/library/collections.abc.rst
+++ b/Doc/library/collections.abc.rst
@@ -92,6 +92,7 @@
 :class:`Coroutine`         :class:`Awaitable`     ``send``, ``throw``     ``close``
 :class:`AsyncIterable`                            ``__aiter__``
 :class:`AsyncIterator`     :class:`AsyncIterable` ``__anext__``           ``__aiter__``
+:class:`AsyncGenerator`    :class:`AsyncIterator` ``asend``, ``athrow``   ``aclose``, ``__aiter__``, ``__anext__``
 ========================== ====================== ======================= ====================================================
 
 
@@ -222,6 +223,13 @@
 
    .. versionadded:: 3.5
 
+.. class:: Generator
+
+   ABC for asynchronous generator classes that implement the protocol
+   defined in :pep:`525` and :pep:`492`.
+
+   .. versionadded:: 3.6
+
 
 These ABCs allow us to ask classes or instances if they provide
 particular functionality, for example::
diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -912,6 +912,10 @@
 iterable classes that also provide the :meth:`__reversed__`.
 (Contributed by Ivan Levkivskyi in :issue:`25987`.)
 
+The new :class:`~collections.abc.AsyncGenerator` abstract base class represents
+asynchronous generators.
+(Contributed by Yury Selivanov in :issue:`28720`.)
+
 The :func:`~collections.namedtuple` function now accepts an optional
 keyword argument *module*, which, when specified, is used for
 the ``__module__`` attribute of the returned named tuple class.
diff --git a/Lib/_collections_abc.py b/Lib/_collections_abc.py
--- a/Lib/_collections_abc.py
+++ b/Lib/_collections_abc.py
@@ -9,7 +9,8 @@
 from abc import ABCMeta, abstractmethod
 import sys
 
-__all__ = ["Awaitable", "Coroutine", "AsyncIterable", "AsyncIterator",
+__all__ = ["Awaitable", "Coroutine",
+           "AsyncIterable", "AsyncIterator", "AsyncGenerator",
            "Hashable", "Iterable", "Iterator", "Generator", "Reversible",
            "Sized", "Container", "Callable", "Collection",
            "Set", "MutableSet",
@@ -59,6 +60,11 @@
 coroutine = type(_coro)
 _coro.close()  # Prevent ResourceWarning
 del _coro
+## asynchronous generator ##
+async def _ag(): yield
+_ag = _ag()
+async_generator = type(_ag)
+del _ag
 
 
 ### ONE-TRICK PONIES ###
@@ -183,6 +189,57 @@
         return NotImplemented
 
 
+class AsyncGenerator(AsyncIterator):
+
+    __slots__ = ()
+
+    async def __anext__(self):
+        """Return the next item from the asynchronous generator.
+        When exhausted, raise StopAsyncIteration.
+        """
+        return await self.asend(None)
+
+    @abstractmethod
+    async def asend(self, value):
+        """Send a value into the asynchronous generator.
+        Return next yielded value or raise StopAsyncIteration.
+        """
+        raise StopAsyncIteration
+
+    @abstractmethod
+    async def athrow(self, typ, val=None, tb=None):
+        """Raise an exception in the asynchronous generator.
+        Return next yielded value or raise StopAsyncIteration.
+        """
+        if val is None:
+            if tb is None:
+                raise typ
+            val = typ()
+        if tb is not None:
+            val = val.with_traceback(tb)
+        raise val
+
+    async def aclose(self):
+        """Raise GeneratorExit inside coroutine.
+        """
+        try:
+            await self.athrow(GeneratorExit)
+        except (GeneratorExit, StopAsyncIteration):
+            pass
+        else:
+            raise RuntimeError("asynchronous generator ignored GeneratorExit")
+
+    @classmethod
+    def __subclasshook__(cls, C):
+        if cls is AsyncGenerator:
+            return _check_methods(C, '__aiter__', '__anext__',
+                                  'asend', 'athrow', 'aclose')
+        return NotImplemented
+
+
+AsyncGenerator.register(async_generator)
+
+
 class Iterable(metaclass=ABCMeta):
 
     __slots__ = ()
diff --git a/Lib/test/test_collections.py b/Lib/test/test_collections.py
--- a/Lib/test/test_collections.py
+++ b/Lib/test/test_collections.py
@@ -19,7 +19,8 @@
 from collections import UserDict, UserString, UserList
 from collections import ChainMap
 from collections import deque
-from collections.abc import Awaitable, Coroutine, AsyncIterator, AsyncIterable
+from collections.abc import Awaitable, Coroutine
+from collections.abc import AsyncIterator, AsyncIterable, AsyncGenerator
 from collections.abc import Hashable, Iterable, Iterator, Generator, Reversible
 from collections.abc import Sized, Container, Callable, Collection
 from collections.abc import Set, MutableSet
@@ -959,6 +960,87 @@
 
         self.assertRaises(RuntimeError, IgnoreGeneratorExit().close)
 
+    def test_AsyncGenerator(self):
+        class NonAGen1:
+            def __aiter__(self): return self
+            def __anext__(self): return None
+            def aclose(self): pass
+            def athrow(self, typ, val=None, tb=None): pass
+
+        class NonAGen2:
+            def __aiter__(self): return self
+            def __anext__(self): return None
+            def aclose(self): pass
+            def asend(self, value): return value
+
+        class NonAGen3:
+            def aclose(self): pass
+            def asend(self, value): return value
+            def athrow(self, typ, val=None, tb=None): pass
+
+        non_samples = [
+            None, 42, 3.14, 1j, b"", "", (), [], {}, set(),
+            iter(()), iter([]), NonAGen1(), NonAGen2(), NonAGen3()]
+        for x in non_samples:
+            self.assertNotIsInstance(x, AsyncGenerator)
+            self.assertFalse(issubclass(type(x), AsyncGenerator), repr(type(x)))
+
+        class Gen:
+            def __aiter__(self): return self
+            async def __anext__(self): return None
+            async def aclose(self): pass
+            async def asend(self, value): return value
+            async def athrow(self, typ, val=None, tb=None): pass
+
+        class MinimalAGen(AsyncGenerator):
+            async def asend(self, value):
+                return value
+            async def athrow(self, typ, val=None, tb=None):
+                await super().athrow(typ, val, tb)
+
+        async def gen():
+            yield 1
+
+        samples = [gen(), Gen(), MinimalAGen()]
+        for x in samples:
+            self.assertIsInstance(x, AsyncIterator)
+            self.assertIsInstance(x, AsyncGenerator)
+            self.assertTrue(issubclass(type(x), AsyncGenerator), repr(type(x)))
+        self.validate_abstract_methods(AsyncGenerator, 'asend', 'athrow')
+
+        def run_async(coro):
+            result = None
+            while True:
+                try:
+                    coro.send(None)
+                except StopIteration as ex:
+                    result = ex.args[0] if ex.args else None
+                    break
+            return result
+
+        # mixin tests
+        mgen = MinimalAGen()
+        self.assertIs(mgen, mgen.__aiter__())
+        self.assertIs(run_async(mgen.asend(None)), run_async(mgen.__anext__()))
+        self.assertEqual(2, run_async(mgen.asend(2)))
+        self.assertIsNone(run_async(mgen.aclose()))
+        with self.assertRaises(ValueError):
+            run_async(mgen.athrow(ValueError))
+
+        class FailOnClose(AsyncGenerator):
+            async def asend(self, value): return value
+            async def athrow(self, *args): raise ValueError
+
+        with self.assertRaises(ValueError):
+            run_async(FailOnClose().aclose())
+
+        class IgnoreGeneratorExit(AsyncGenerator):
+            async def asend(self, value): return value
+            async def athrow(self, *args): pass
+
+        with self.assertRaises(RuntimeError):
+            run_async(IgnoreGeneratorExit().aclose())
+
     def test_Sized(self):
         non_samples = [None, 42, 3.14, 1j,
                        _test_gen(),

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 19:36:21 2016
From: python-checkins at python.org (steve.dower)
Date: Thu, 17 Nov 2016 00:36:21 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_from_3=2E5?=
Message-ID: <20161117003621.25141.84086.84DEF831@psf.io>

https://hg.python.org/cpython/rev/8b964ea8c26f
changeset:   105166:8b964ea8c26f
branch:      3.6
parent:      105163:ae1dba7e7d04
parent:      105165:20f62e4a9c2f
user:        Steve Dower <steve.dower at microsoft.com>
date:        Wed Nov 16 16:35:40 2016 -0800
summary:
  Merge from 3.5

files:



-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 19:36:21 2016
From: python-checkins at python.org (steve.dower)
Date: Thu, 17 Nov 2016 00:36:21 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fixes_incorrec?=
 =?utf-8?q?t_path_in_nuget_package_build=2E?=
Message-ID: <20161117003621.14262.25303.8C256018@psf.io>

https://hg.python.org/cpython/rev/20f62e4a9c2f
changeset:   105165:20f62e4a9c2f
branch:      3.5
parent:      105154:b607f835f170
user:        Steve Dower <steve.dower at microsoft.com>
date:        Wed Nov 16 16:32:22 2016 -0800
summary:
  Fixes incorrect path in nuget package build.

files:
  Tools/nuget/make_pkg.proj |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Tools/nuget/make_pkg.proj b/Tools/nuget/make_pkg.proj
--- a/Tools/nuget/make_pkg.proj
+++ b/Tools/nuget/make_pkg.proj
@@ -14,6 +14,7 @@
     <PropertyGroup>
         <Nuget Condition="$(Nuget) == ''">$(ExternalsDir)\windows-installer\nuget\nuget.exe</Nuget>
         <NuspecVersion>$(MajorVersionNumber).$(MinorVersionNumber).$(MicroVersionNumber)</NuspecVersion>
+        <NuspecVersion Condition="$(ReleaseLevelName) != ''">$(NuspecVersion)-$(ReleaseLevelName)</NuspecVersion>
         <SignOutput>false</SignOutput>
         <TargetName>$(OutputName).$(NuspecVersion)</TargetName>
         <TargetExt>.nupkg</TargetExt>
@@ -23,7 +24,7 @@
         <CleanCommand>rmdir /q/s "$(IntermediateOutputPath)"</CleanCommand>
         
         <PythonArguments>"$(PythonExe)" "$(MSBuildThisFileDirectory)\..\msi\make_zip.py"</PythonArguments>
-        <PythonArguments>$(PythonArguments) -t "$(IntermediateOutputPath)" -b "$(OutDir.TrimEnd('\'))\.."</PythonArguments>
+        <PythonArguments>$(PythonArguments) -t "$(IntermediateOutputPath)" -b "$(OutDir.TrimEnd('\'))"</PythonArguments>
         
         <PipArguments>"$(IntermediateOutputPath)\python.exe" -B -c "import sys; sys.path.append(r'$(PySourcePath)\Lib'); import ensurepip; ensurepip._main()"</PipArguments>
         <PackageArguments Condition="$(Packages) != ''">"$(IntermediateOutputPath)\python.exe" -B -m pip install -U $(Packages)</PackageArguments>

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 16 19:36:25 2016
From: python-checkins at python.org (steve.dower)
Date: Thu, 17 Nov 2016 00:36:25 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Null_merge_from_3=2E6?=
Message-ID: <20161117003621.93319.2640.732690BB@psf.io>

https://hg.python.org/cpython/rev/05a728e1da15
changeset:   105167:05a728e1da15
parent:      105164:fb9c8fdef3ec
parent:      105166:8b964ea8c26f
user:        Steve Dower <steve.dower at microsoft.com>
date:        Wed Nov 16 16:35:53 2016 -0800
summary:
  Null merge from 3.6

files:



-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 17 00:34:44 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Thu, 17 Nov 2016 05:34:44 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Minor_touch-up?=
 =?utf-8?q?s_to_the_random_module_examples?=
Message-ID: <20161117053444.93754.82001.052C80D7@psf.io>

https://hg.python.org/cpython/rev/331ac810495f
changeset:   105168:331ac810495f
branch:      3.6
parent:      105166:8b964ea8c26f
user:        Raymond Hettinger <python at rcn.com>
date:        Wed Nov 16 21:34:17 2016 -0800
summary:
  Minor touch-ups to the random module examples

files:
  Doc/library/random.rst |  11 ++++++-----
  1 files changed, 6 insertions(+), 5 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -344,15 +344,15 @@
    >>> deck
    ['king', 'queen', 'ace', 'jack']
 
-   >>> random.sample([1, 2, 3, 4, 5],  3)   # Three samples without replacement
+   >>> random.sample([1, 2, 3, 4, 5], k=3)  # Three samples without replacement
    [4, 1, 5]
 
    >>>                                      # Six weighted samples with replacement
    >>> choices(['red', 'black', 'green'], [18, 18, 2], k=6)
    ['red', 'green', 'black', 'black', 'red', 'black']
 
-   # Probability of getting 5 or more heads from 7 spins of a biased coin
-   # that settles on heads 60% of the time.
+   # Probability of getting 5 or more heads from 7 spins
+   # of a biased coin that settles on heads 60% of the time.
    >>> n = 10000
    >>> cw = [0.60, 1.00]
    >>> sum(choices('HT', cum_weights=cw, k=7).count('H') >= 5 for i in range(n)) / n
@@ -369,5 +369,6 @@
 
    data = 1, 2, 4, 4, 10
    means = sorted(mean(choices(data, k=5)) for i in range(20))
-   print('The sample mean of {:.1f} has a 90% confidence interval '
-         'from {:.1f} to {:.1f}'.format(mean(data), means[1], means[-2]))
+   print(f'The sample mean of {mean(data):.1f} has a 90% confidence '
+         f'interval from {means[1]:.1f} to {means[-2]:.1f}')
+

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 17 00:34:44 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Thu, 17 Nov 2016 05:34:44 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161117053444.106780.60804.7E5C5483@psf.io>

https://hg.python.org/cpython/rev/89073136925d
changeset:   105169:89073136925d
parent:      105167:05a728e1da15
parent:      105168:331ac810495f
user:        Raymond Hettinger <python at rcn.com>
date:        Wed Nov 16 21:34:39 2016 -0800
summary:
  merge

files:
  Doc/library/random.rst |  11 ++++++-----
  1 files changed, 6 insertions(+), 5 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -344,15 +344,15 @@
    >>> deck
    ['king', 'queen', 'ace', 'jack']
 
-   >>> random.sample([1, 2, 3, 4, 5],  3)   # Three samples without replacement
+   >>> random.sample([1, 2, 3, 4, 5], k=3)  # Three samples without replacement
    [4, 1, 5]
 
    >>>                                      # Six weighted samples with replacement
    >>> choices(['red', 'black', 'green'], [18, 18, 2], k=6)
    ['red', 'green', 'black', 'black', 'red', 'black']
 
-   # Probability of getting 5 or more heads from 7 spins of a biased coin
-   # that settles on heads 60% of the time.
+   # Probability of getting 5 or more heads from 7 spins
+   # of a biased coin that settles on heads 60% of the time.
    >>> n = 10000
    >>> cw = [0.60, 1.00]
    >>> sum(choices('HT', cum_weights=cw, k=7).count('H') >= 5 for i in range(n)) / n
@@ -369,5 +369,6 @@
 
    data = 1, 2, 4, 4, 10
    means = sorted(mean(choices(data, k=5)) for i in range(20))
-   print('The sample mean of {:.1f} has a 90% confidence interval '
-         'from {:.1f} to {:.1f}'.format(mean(data), means[1], means[-2]))
+   print(f'The sample mean of {mean(data):.1f} has a 90% confidence '
+         f'interval from {means[1]:.1f} to {means[-2]:.1f}')
+

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 17 01:56:43 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Thu, 17 Nov 2016 06:56:43 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Add_another_ex?=
 =?utf-8?q?ample_to_the_recipes_section_of_the_random_docs?=
Message-ID: <20161117065643.106870.73880.1388696E@psf.io>

https://hg.python.org/cpython/rev/5d216c21f01f
changeset:   105170:5d216c21f01f
branch:      3.6
parent:      105168:331ac810495f
user:        Raymond Hettinger <python at rcn.com>
date:        Wed Nov 16 22:56:11 2016 -0800
summary:
  Add another example to the recipes section of the random docs

files:
  Doc/library/random.rst |  26 ++++++++++++++++++++++++++
  1 files changed, 26 insertions(+), 0 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -372,3 +372,29 @@
    print(f'The sample mean of {mean(data):.1f} has a 90% confidence '
          f'interval from {means[1]:.1f} to {means[-2]:.1f}')
 
+Example of a `resampling permutation test
+<https://en.wikipedia.org/wiki/Resampling_(statistics)#Permutation_tests>`_
+to determine the statistical significance or `p-value
+<https://en.wikipedia.org/wiki/P-value>`_ of an observed difference
+between the effects of a drug versus a placebo::
+
+    # Example from "Statistics is Easy" by Dennis Shasha and Manda Wilson
+    from statistics import mean
+    from random import shuffle
+
+    drug = [54, 73, 53, 70, 73, 68, 52, 65, 65]
+    placebo = [54, 51, 58, 44, 55, 52, 42, 47, 58, 46]
+    observed_diff = mean(drug) - mean(placebo)
+
+    n = 10000
+    count = 0
+    combined = drug + placebo
+    for i in range(n):
+        shuffle(combined)
+        new_diff = mean(combined[:len(drug)]) - mean(combined[len(drug):])
+        count += (new_diff >= observed_diff)
+
+    print(f'{n} label reshufflings produced only {count} instances with a difference')
+    print(f'at least as extreme as the observed difference of {observed_diff:.1f}.')
+    print(f'The one-sided p-value of {count / n:.4f} leads us to reject the null')
+    print(f'hypothesis that the observed difference occurred due to chance.')

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 17 01:56:45 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Thu, 17 Nov 2016 06:56:45 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161117065643.24958.31249.18F1583A@psf.io>

https://hg.python.org/cpython/rev/020822a2a930
changeset:   105171:020822a2a930
parent:      105169:89073136925d
parent:      105170:5d216c21f01f
user:        Raymond Hettinger <python at rcn.com>
date:        Wed Nov 16 22:56:37 2016 -0800
summary:
  merge

files:
  Doc/library/random.rst |  26 ++++++++++++++++++++++++++
  1 files changed, 26 insertions(+), 0 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -372,3 +372,29 @@
    print(f'The sample mean of {mean(data):.1f} has a 90% confidence '
          f'interval from {means[1]:.1f} to {means[-2]:.1f}')
 
+Example of a `resampling permutation test
+<https://en.wikipedia.org/wiki/Resampling_(statistics)#Permutation_tests>`_
+to determine the statistical significance or `p-value
+<https://en.wikipedia.org/wiki/P-value>`_ of an observed difference
+between the effects of a drug versus a placebo::
+
+    # Example from "Statistics is Easy" by Dennis Shasha and Manda Wilson
+    from statistics import mean
+    from random import shuffle
+
+    drug = [54, 73, 53, 70, 73, 68, 52, 65, 65]
+    placebo = [54, 51, 58, 44, 55, 52, 42, 47, 58, 46]
+    observed_diff = mean(drug) - mean(placebo)
+
+    n = 10000
+    count = 0
+    combined = drug + placebo
+    for i in range(n):
+        shuffle(combined)
+        new_diff = mean(combined[:len(drug)]) - mean(combined[len(drug):])
+        count += (new_diff >= observed_diff)
+
+    print(f'{n} label reshufflings produced only {count} instances with a difference')
+    print(f'at least as extreme as the observed difference of {observed_diff:.1f}.')
+    print(f'The one-sided p-value of {count / n:.4f} leads us to reject the null')
+    print(f'hypothesis that the observed difference occurred due to chance.')

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 17 03:02:55 2016
From: python-checkins at python.org (xavier.degaye)
Date: Thu, 17 Nov 2016 08:02:55 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_26931=3A_Merge_3=2E6?=
Message-ID: <20161117080254.12078.14817.58554512@psf.io>

https://hg.python.org/cpython/rev/99d69fd1b24e
changeset:   105173:99d69fd1b24e
parent:      105171:020822a2a930
parent:      105172:cea3b621973f
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Thu Nov 17 09:01:58 2016 +0100
summary:
  Issue 26931: Merge 3.6

files:
  Lib/distutils/tests/test_build_clib.py |  20 +++-------
  Lib/distutils/tests/test_build_ext.py  |   6 +++
  Lib/distutils/tests/test_config_cmd.py |   5 ++-
  Lib/distutils/tests/test_install.py    |   4 ++
  Lib/distutils/tests/test_sysconfig.py  |   9 ----
  Lib/test/support/__init__.py           |  27 +++++++++++++-
  6 files changed, 46 insertions(+), 25 deletions(-)


diff --git a/Lib/distutils/tests/test_build_clib.py b/Lib/distutils/tests/test_build_clib.py
--- a/Lib/distutils/tests/test_build_clib.py
+++ b/Lib/distutils/tests/test_build_clib.py
@@ -3,7 +3,7 @@
 import os
 import sys
 
-from test.support import run_unittest
+from test.support import run_unittest, missing_compiler_executable
 
 from distutils.command.build_clib import build_clib
 from distutils.errors import DistutilsSetupError
@@ -116,19 +116,11 @@
         cmd.build_temp = build_temp
         cmd.build_clib = build_temp
 
-        # before we run the command, we want to make sure
-        # all commands are present on the system
-        # by creating a compiler and checking its executables
-        from distutils.ccompiler import new_compiler
-        from distutils.sysconfig import customize_compiler
-
-        compiler = new_compiler()
-        customize_compiler(compiler)
-        for ccmd in compiler.executables.values():
-            if ccmd is None:
-                continue
-            if find_executable(ccmd[0]) is None:
-                self.skipTest('The %r command is not found' % ccmd[0])
+        # Before we run the command, we want to make sure
+        # all commands are present on the system.
+        ccmd = missing_compiler_executable()
+        if ccmd is not None:
+            self.skipTest('The %r command is not found' % ccmd)
 
         # this should work
         cmd.run()
diff --git a/Lib/distutils/tests/test_build_ext.py b/Lib/distutils/tests/test_build_ext.py
--- a/Lib/distutils/tests/test_build_ext.py
+++ b/Lib/distutils/tests/test_build_ext.py
@@ -41,6 +41,9 @@
         return build_ext(*args, **kwargs)
 
     def test_build_ext(self):
+        cmd = support.missing_compiler_executable()
+        if cmd is not None:
+            self.skipTest('The %r command is not found' % cmd)
         global ALREADY_TESTED
         copy_xxmodule_c(self.tmp_dir)
         xx_c = os.path.join(self.tmp_dir, 'xxmodule.c')
@@ -295,6 +298,9 @@
         self.assertEqual(cmd.compiler, 'unix')
 
     def test_get_outputs(self):
+        cmd = support.missing_compiler_executable()
+        if cmd is not None:
+            self.skipTest('The %r command is not found' % cmd)
         tmp_dir = self.mkdtemp()
         c_file = os.path.join(tmp_dir, 'foo.c')
         self.write_file(c_file, 'void PyInit_foo(void) {}\n')
diff --git a/Lib/distutils/tests/test_config_cmd.py b/Lib/distutils/tests/test_config_cmd.py
--- a/Lib/distutils/tests/test_config_cmd.py
+++ b/Lib/distutils/tests/test_config_cmd.py
@@ -2,7 +2,7 @@
 import unittest
 import os
 import sys
-from test.support import run_unittest
+from test.support import run_unittest, missing_compiler_executable
 
 from distutils.command.config import dump_file, config
 from distutils.tests import support
@@ -39,6 +39,9 @@
 
     @unittest.skipIf(sys.platform == 'win32', "can't test on Windows")
     def test_search_cpp(self):
+        cmd = missing_compiler_executable(['preprocessor'])
+        if cmd is not None:
+            self.skipTest('The %r command is not found' % cmd)
         pkg_dir, dist = self.create_dist()
         cmd = config(dist)
 
diff --git a/Lib/distutils/tests/test_install.py b/Lib/distutils/tests/test_install.py
--- a/Lib/distutils/tests/test_install.py
+++ b/Lib/distutils/tests/test_install.py
@@ -17,6 +17,7 @@
 from distutils.extension import Extension
 
 from distutils.tests import support
+from test import support as test_support
 
 
 def _make_ext_name(modname):
@@ -196,6 +197,9 @@
         self.assertEqual(found, expected)
 
     def test_record_extensions(self):
+        cmd = test_support.missing_compiler_executable()
+        if cmd is not None:
+            self.skipTest('The %r command is not found' % cmd)
         install_dir = self.mkdtemp()
         project_dir, dist = self.create_dist(ext_modules=[
             Extension('xx', ['xxmodule.c'])])
diff --git a/Lib/distutils/tests/test_sysconfig.py b/Lib/distutils/tests/test_sysconfig.py
--- a/Lib/distutils/tests/test_sysconfig.py
+++ b/Lib/distutils/tests/test_sysconfig.py
@@ -39,15 +39,6 @@
         self.assertNotEqual(sysconfig.get_python_lib(),
                             sysconfig.get_python_lib(prefix=TESTFN))
 
-    def test_get_python_inc(self):
-        inc_dir = sysconfig.get_python_inc()
-        # This is not much of a test.  We make sure Python.h exists
-        # in the directory returned by get_python_inc() but we don't know
-        # it is the correct file.
-        self.assertTrue(os.path.isdir(inc_dir), inc_dir)
-        python_h = os.path.join(inc_dir, "Python.h")
-        self.assertTrue(os.path.isfile(python_h), python_h)
-
     def test_get_config_vars(self):
         cvars = sysconfig.get_config_vars()
         self.assertIsInstance(cvars, dict)
diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -105,7 +105,7 @@
     "check_warnings", "check_no_resource_warning", "EnvironmentVarGuard",
     "run_with_locale", "swap_item",
     "swap_attr", "Matcher", "set_memlimit", "SuppressCrashReport", "sortdict",
-    "run_with_tz", "PGO",
+    "run_with_tz", "PGO", "missing_compiler_executable",
     ]
 
 class Error(Exception):
@@ -2491,3 +2491,28 @@
     # The sequence should be deallocated just after the end of iterating
     gc_collect()
     test.assertTrue(done)
+
+
+def missing_compiler_executable(cmd_names=[]):
+    """Check if the compiler components used to build the interpreter exist.
+
+    Check for the existence of the compiler executables whose names are listed
+    in 'cmd_names' or all the compiler executables when 'cmd_names' is empty
+    and return the first missing executable or None when none is found
+    missing.
+
+    """
+    from distutils import ccompiler, sysconfig, spawn
+    compiler = ccompiler.new_compiler()
+    sysconfig.customize_compiler(compiler)
+    for name in compiler.executables:
+        if cmd_names and name not in cmd_names:
+            continue
+        cmd = getattr(compiler, name)
+        if cmd_names:
+            assert cmd is not None, \
+                    "the '%s' executable is not configured" % name
+        elif cmd is None:
+            continue
+        if spawn.find_executable(cmd[0]) is None:
+            return cmd[0]

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 17 03:02:55 2016
From: python-checkins at python.org (xavier.degaye)
Date: Thu, 17 Nov 2016 08:02:55 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgMjY5MzE6?=
 =?utf-8?q?_Skip_the_test=5Fdistutils_tests_using_a_compiler_executable?=
Message-ID: <20161117080254.7341.39074.C95F1FD0@psf.io>

https://hg.python.org/cpython/rev/cea3b621973f
changeset:   105172:cea3b621973f
branch:      3.6
parent:      105170:5d216c21f01f
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Thu Nov 17 09:00:19 2016 +0100
summary:
  Issue 26931: Skip the test_distutils tests using a compiler executable
that is not found

files:
  Lib/distutils/tests/test_build_clib.py |  20 +++-------
  Lib/distutils/tests/test_build_ext.py  |   6 +++
  Lib/distutils/tests/test_config_cmd.py |   5 ++-
  Lib/distutils/tests/test_install.py    |   4 ++
  Lib/distutils/tests/test_sysconfig.py  |   9 ----
  Lib/test/support/__init__.py           |  27 +++++++++++++-
  6 files changed, 46 insertions(+), 25 deletions(-)


diff --git a/Lib/distutils/tests/test_build_clib.py b/Lib/distutils/tests/test_build_clib.py
--- a/Lib/distutils/tests/test_build_clib.py
+++ b/Lib/distutils/tests/test_build_clib.py
@@ -3,7 +3,7 @@
 import os
 import sys
 
-from test.support import run_unittest
+from test.support import run_unittest, missing_compiler_executable
 
 from distutils.command.build_clib import build_clib
 from distutils.errors import DistutilsSetupError
@@ -116,19 +116,11 @@
         cmd.build_temp = build_temp
         cmd.build_clib = build_temp
 
-        # before we run the command, we want to make sure
-        # all commands are present on the system
-        # by creating a compiler and checking its executables
-        from distutils.ccompiler import new_compiler
-        from distutils.sysconfig import customize_compiler
-
-        compiler = new_compiler()
-        customize_compiler(compiler)
-        for ccmd in compiler.executables.values():
-            if ccmd is None:
-                continue
-            if find_executable(ccmd[0]) is None:
-                self.skipTest('The %r command is not found' % ccmd[0])
+        # Before we run the command, we want to make sure
+        # all commands are present on the system.
+        ccmd = missing_compiler_executable()
+        if ccmd is not None:
+            self.skipTest('The %r command is not found' % ccmd)
 
         # this should work
         cmd.run()
diff --git a/Lib/distutils/tests/test_build_ext.py b/Lib/distutils/tests/test_build_ext.py
--- a/Lib/distutils/tests/test_build_ext.py
+++ b/Lib/distutils/tests/test_build_ext.py
@@ -41,6 +41,9 @@
         return build_ext(*args, **kwargs)
 
     def test_build_ext(self):
+        cmd = support.missing_compiler_executable()
+        if cmd is not None:
+            self.skipTest('The %r command is not found' % cmd)
         global ALREADY_TESTED
         copy_xxmodule_c(self.tmp_dir)
         xx_c = os.path.join(self.tmp_dir, 'xxmodule.c')
@@ -295,6 +298,9 @@
         self.assertEqual(cmd.compiler, 'unix')
 
     def test_get_outputs(self):
+        cmd = support.missing_compiler_executable()
+        if cmd is not None:
+            self.skipTest('The %r command is not found' % cmd)
         tmp_dir = self.mkdtemp()
         c_file = os.path.join(tmp_dir, 'foo.c')
         self.write_file(c_file, 'void PyInit_foo(void) {}\n')
diff --git a/Lib/distutils/tests/test_config_cmd.py b/Lib/distutils/tests/test_config_cmd.py
--- a/Lib/distutils/tests/test_config_cmd.py
+++ b/Lib/distutils/tests/test_config_cmd.py
@@ -2,7 +2,7 @@
 import unittest
 import os
 import sys
-from test.support import run_unittest
+from test.support import run_unittest, missing_compiler_executable
 
 from distutils.command.config import dump_file, config
 from distutils.tests import support
@@ -39,6 +39,9 @@
 
     @unittest.skipIf(sys.platform == 'win32', "can't test on Windows")
     def test_search_cpp(self):
+        cmd = missing_compiler_executable(['preprocessor'])
+        if cmd is not None:
+            self.skipTest('The %r command is not found' % cmd)
         pkg_dir, dist = self.create_dist()
         cmd = config(dist)
 
diff --git a/Lib/distutils/tests/test_install.py b/Lib/distutils/tests/test_install.py
--- a/Lib/distutils/tests/test_install.py
+++ b/Lib/distutils/tests/test_install.py
@@ -17,6 +17,7 @@
 from distutils.extension import Extension
 
 from distutils.tests import support
+from test import support as test_support
 
 
 def _make_ext_name(modname):
@@ -196,6 +197,9 @@
         self.assertEqual(found, expected)
 
     def test_record_extensions(self):
+        cmd = test_support.missing_compiler_executable()
+        if cmd is not None:
+            self.skipTest('The %r command is not found' % cmd)
         install_dir = self.mkdtemp()
         project_dir, dist = self.create_dist(ext_modules=[
             Extension('xx', ['xxmodule.c'])])
diff --git a/Lib/distutils/tests/test_sysconfig.py b/Lib/distutils/tests/test_sysconfig.py
--- a/Lib/distutils/tests/test_sysconfig.py
+++ b/Lib/distutils/tests/test_sysconfig.py
@@ -39,15 +39,6 @@
         self.assertNotEqual(sysconfig.get_python_lib(),
                             sysconfig.get_python_lib(prefix=TESTFN))
 
-    def test_get_python_inc(self):
-        inc_dir = sysconfig.get_python_inc()
-        # This is not much of a test.  We make sure Python.h exists
-        # in the directory returned by get_python_inc() but we don't know
-        # it is the correct file.
-        self.assertTrue(os.path.isdir(inc_dir), inc_dir)
-        python_h = os.path.join(inc_dir, "Python.h")
-        self.assertTrue(os.path.isfile(python_h), python_h)
-
     def test_get_config_vars(self):
         cvars = sysconfig.get_config_vars()
         self.assertIsInstance(cvars, dict)
diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -105,7 +105,7 @@
     "check_warnings", "check_no_resource_warning", "EnvironmentVarGuard",
     "run_with_locale", "swap_item",
     "swap_attr", "Matcher", "set_memlimit", "SuppressCrashReport", "sortdict",
-    "run_with_tz", "PGO",
+    "run_with_tz", "PGO", "missing_compiler_executable",
     ]
 
 class Error(Exception):
@@ -2491,3 +2491,28 @@
     # The sequence should be deallocated just after the end of iterating
     gc_collect()
     test.assertTrue(done)
+
+
+def missing_compiler_executable(cmd_names=[]):
+    """Check if the compiler components used to build the interpreter exist.
+
+    Check for the existence of the compiler executables whose names are listed
+    in 'cmd_names' or all the compiler executables when 'cmd_names' is empty
+    and return the first missing executable or None when none is found
+    missing.
+
+    """
+    from distutils import ccompiler, sysconfig, spawn
+    compiler = ccompiler.new_compiler()
+    sysconfig.customize_compiler(compiler)
+    for name in compiler.executables:
+        if cmd_names and name not in cmd_names:
+            continue
+        cmd = getattr(compiler, name)
+        if cmd_names:
+            assert cmd is not None, \
+                    "the '%s' executable is not configured" % name
+        elif cmd is None:
+            continue
+        if spawn.find_executable(cmd[0]) is None:
+            return cmd[0]

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 17 03:23:16 2016
From: python-checkins at python.org (xavier.degaye)
Date: Thu, 17 Nov 2016 08:23:16 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2326926=3A_Merge_3=2E6?=
Message-ID: <20161117082315.93069.96178.A1D80B1B@psf.io>

https://hg.python.org/cpython/rev/878f91b4ad19
changeset:   105175:878f91b4ad19
parent:      105173:99d69fd1b24e
parent:      105174:d616304b82aa
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Thu Nov 17 09:22:43 2016 +0100
summary:
  Issue #26926: Merge 3.6

files:
  Lib/test/test_io.py |  5 ++++-
  1 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py
--- a/Lib/test/test_io.py
+++ b/Lib/test/test_io.py
@@ -350,7 +350,10 @@
     def large_file_ops(self, f):
         assert f.readable()
         assert f.writable()
-        self.assertEqual(f.seek(self.LARGE), self.LARGE)
+        try:
+            self.assertEqual(f.seek(self.LARGE), self.LARGE)
+        except (OverflowError, ValueError):
+            self.skipTest("no largefile support")
         self.assertEqual(f.tell(), self.LARGE)
         self.assertEqual(f.write(b"xxx"), 3)
         self.assertEqual(f.tell(), self.LARGE + 3)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 17 03:23:16 2016
From: python-checkins at python.org (xavier.degaye)
Date: Thu, 17 Nov 2016 08:23:16 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI2OTI2?=
 =?utf-8?q?=3A_Skip_some_test=5Fio_tests_on_platforms_without_large_file_s?=
 =?utf-8?q?upport?=
Message-ID: <20161117082315.93710.38510.3C45589B@psf.io>

https://hg.python.org/cpython/rev/d616304b82aa
changeset:   105174:d616304b82aa
branch:      3.6
parent:      105172:cea3b621973f
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Thu Nov 17 09:20:28 2016 +0100
summary:
  Issue #26926: Skip some test_io tests on platforms without large file support

files:
  Lib/test/test_io.py |  5 ++++-
  1 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py
--- a/Lib/test/test_io.py
+++ b/Lib/test/test_io.py
@@ -350,7 +350,10 @@
     def large_file_ops(self, f):
         assert f.readable()
         assert f.writable()
-        self.assertEqual(f.seek(self.LARGE), self.LARGE)
+        try:
+            self.assertEqual(f.seek(self.LARGE), self.LARGE)
+        except (OverflowError, ValueError):
+            self.skipTest("no largefile support")
         self.assertEqual(f.tell(), self.LARGE)
         self.assertEqual(f.write(b"xxx"), 3)
         self.assertEqual(f.tell(), self.LARGE + 3)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 17 03:45:57 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Thu, 17 Nov 2016 08:45:57 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Further_refine?=
 =?utf-8?q?ments_to_the_examples_and_recipes_for_the_random_module?=
Message-ID: <20161117084557.6983.38539.3D50F028@psf.io>

https://hg.python.org/cpython/rev/e699f3a004ea
changeset:   105176:e699f3a004ea
branch:      3.6
parent:      105174:d616304b82aa
user:        Raymond Hettinger <python at rcn.com>
date:        Thu Nov 17 00:45:35 2016 -0800
summary:
  Further refinements to the examples and recipes for the random module

files:
  Doc/library/random.rst |  45 +++++++++++++++++++----------
  1 files changed, 29 insertions(+), 16 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -322,36 +322,49 @@
 Examples and Recipes
 --------------------
 
-Basic usage::
+Basic examples::
 
-   >>> random.random()                      # Random float x, 0.0 <= x < 1.0
+   >>> random()                             # Random float:  0.0 <= x < 1.0
    0.37444887175646646
 
-   >>> random.uniform(1, 10)                # Random float x, 1.0 <= x < 10.0
-   1.1800146073117523
+   >>> uniform(2, 10)                       # Random float:  2.0 <= x < 10.0
+   3.1800146073117523
 
-   >>> random.randrange(10)                 # Integer from 0 to 9
+   >>> expovariate(1/5)                     # Interval between arrivals averaging 5 seconds
+   5.148957571865031
+
+   >>> randrange(10)                        # Integer from 0 to 9
    7
 
-   >>> random.randrange(0, 101, 2)          # Even integer from 0 to 100
+   >>> randrange(0, 101, 2)                 # Even integer from 0 to 100 inclusive
    26
 
-   >>> random.choice('abcdefghij')          # Single random element
+   >>> choice('abcdefghij')                 # Single random element from a sequence
    'c'
 
-   >>> deck = ['jack', 'queen', 'king', 'ace']
-   >>> shuffle(deck)
+   >>> deck = 'ace two three four'.split()
+   >>> shuffle(deck)                        # Shuffle a list
    >>> deck
-   ['king', 'queen', 'ace', 'jack']
+   ['four', 'two', 'ace', 'three']
 
-   >>> random.sample([1, 2, 3, 4, 5], k=3)  # Three samples without replacement
-   [4, 1, 5]
+   >>> sample([10, 20, 30, 40, 50], k=4)    # Four samples without replacement
+   [40, 10, 50, 30]
 
-   >>>                                      # Six weighted samples with replacement
+Simulations::
+
+   # Six roulette wheel spins (weighted sampling with replacement)
    >>> choices(['red', 'black', 'green'], [18, 18, 2], k=6)
    ['red', 'green', 'black', 'black', 'red', 'black']
 
-   # Probability of getting 5 or more heads from 7 spins
+   # Deal 20 cards without replacement from a deck of 52
+   # playing cards and determine the proportion of cards
+   # with a ten-value (i.e. a ten, jack, queen, or king).
+   >>> deck = collections.Counter(tens=16, low_cards=36)
+   >>> seen = sample(list(deck.elements()), k=20)
+   >>> print(seen.count('tens') / 20)
+   0.15
+
+   # Estimate the probability of getting 5 or more heads from 7 spins
    # of a biased coin that settles on heads 60% of the time.
    >>> n = 10000
    >>> cw = [0.60, 1.00]
@@ -360,8 +373,8 @@
 
 Example of `statistical bootstrapping
 <https://en.wikipedia.org/wiki/Bootstrapping_(statistics)>`_ using resampling
-with replacement to estimate a confidence interval for the mean of a small
-sample of size five::
+with replacement to estimate a confidence interval for the mean of a sample of
+size five::
 
    # http://statistics.about.com/od/Applications/a/Example-Of-Bootstrapping.htm
    from statistics import mean

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 17 03:45:57 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Thu, 17 Nov 2016 08:45:57 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161117084557.25398.41994.7A131B2A@psf.io>

https://hg.python.org/cpython/rev/8fab4ba536fc
changeset:   105177:8fab4ba536fc
parent:      105175:878f91b4ad19
parent:      105176:e699f3a004ea
user:        Raymond Hettinger <python at rcn.com>
date:        Thu Nov 17 00:45:51 2016 -0800
summary:
  merge

files:
  Doc/library/random.rst |  45 +++++++++++++++++++----------
  1 files changed, 29 insertions(+), 16 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -322,36 +322,49 @@
 Examples and Recipes
 --------------------
 
-Basic usage::
+Basic examples::
 
-   >>> random.random()                      # Random float x, 0.0 <= x < 1.0
+   >>> random()                             # Random float:  0.0 <= x < 1.0
    0.37444887175646646
 
-   >>> random.uniform(1, 10)                # Random float x, 1.0 <= x < 10.0
-   1.1800146073117523
+   >>> uniform(2, 10)                       # Random float:  2.0 <= x < 10.0
+   3.1800146073117523
 
-   >>> random.randrange(10)                 # Integer from 0 to 9
+   >>> expovariate(1/5)                     # Interval between arrivals averaging 5 seconds
+   5.148957571865031
+
+   >>> randrange(10)                        # Integer from 0 to 9
    7
 
-   >>> random.randrange(0, 101, 2)          # Even integer from 0 to 100
+   >>> randrange(0, 101, 2)                 # Even integer from 0 to 100 inclusive
    26
 
-   >>> random.choice('abcdefghij')          # Single random element
+   >>> choice('abcdefghij')                 # Single random element from a sequence
    'c'
 
-   >>> deck = ['jack', 'queen', 'king', 'ace']
-   >>> shuffle(deck)
+   >>> deck = 'ace two three four'.split()
+   >>> shuffle(deck)                        # Shuffle a list
    >>> deck
-   ['king', 'queen', 'ace', 'jack']
+   ['four', 'two', 'ace', 'three']
 
-   >>> random.sample([1, 2, 3, 4, 5], k=3)  # Three samples without replacement
-   [4, 1, 5]
+   >>> sample([10, 20, 30, 40, 50], k=4)    # Four samples without replacement
+   [40, 10, 50, 30]
 
-   >>>                                      # Six weighted samples with replacement
+Simulations::
+
+   # Six roulette wheel spins (weighted sampling with replacement)
    >>> choices(['red', 'black', 'green'], [18, 18, 2], k=6)
    ['red', 'green', 'black', 'black', 'red', 'black']
 
-   # Probability of getting 5 or more heads from 7 spins
+   # Deal 20 cards without replacement from a deck of 52
+   # playing cards and determine the proportion of cards
+   # with a ten-value (i.e. a ten, jack, queen, or king).
+   >>> deck = collections.Counter(tens=16, low_cards=36)
+   >>> seen = sample(list(deck.elements()), k=20)
+   >>> print(seen.count('tens') / 20)
+   0.15
+
+   # Estimate the probability of getting 5 or more heads from 7 spins
    # of a biased coin that settles on heads 60% of the time.
    >>> n = 10000
    >>> cw = [0.60, 1.00]
@@ -360,8 +373,8 @@
 
 Example of `statistical bootstrapping
 <https://en.wikipedia.org/wiki/Bootstrapping_(statistics)>`_ using resampling
-with replacement to estimate a confidence interval for the mean of a small
-sample of size five::
+with replacement to estimate a confidence interval for the mean of a sample of
+size five::
 
    # http://statistics.about.com/od/Applications/a/Example-Of-Bootstrapping.htm
    from statistics import mean

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Thu Nov 17 04:05:07 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Thu, 17 Nov 2016 09:05:07 +0000
Subject: [Python-checkins] Daily reference leaks (89073136925d): sum=5
Message-ID: <20161117090507.7027.6838.208E57DC@psf.io>

results for 89073136925d on branch "default"
--------------------------------------------

test_collections leaked [-7, 1, 7] memory blocks, sum=1
test_functools leaked [0, 3, 1] memory blocks, sum=4
test_multiprocessing_forkserver leaked [-2, 0, 2] memory blocks, sum=0


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogz5c9Wv', '--timeout', '7200']

From python-checkins at python.org  Thu Nov 17 04:50:22 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Thu, 17 Nov 2016 09:50:22 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161117095022.88549.32154.84A43040@psf.io>

https://hg.python.org/cpython/rev/53f5e16695d0
changeset:   105179:53f5e16695d0
parent:      105177:8fab4ba536fc
parent:      105178:3e6f23fd0c34
user:        Raymond Hettinger <python at rcn.com>
date:        Thu Nov 17 01:50:16 2016 -0800
summary:
  merge

files:
  Doc/library/random.rst |  22 ++++++++++++++--------
  1 files changed, 14 insertions(+), 8 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -152,13 +152,19 @@
 
 .. function:: shuffle(x[, random])
 
-   Shuffle the sequence *x* in place. The optional argument *random* is a
-   0-argument function returning a random float in [0.0, 1.0); by default, this is
-   the function :func:`.random`.
+   Shuffle the sequence *x* in place.
 
-   Note that for even rather small ``len(x)``, the total number of permutations of
-   *x* is larger than the period of most random number generators; this implies
-   that most permutations of a long sequence can never be generated.
+   The optional argument *random* is a 0-argument function returning a random
+   float in [0.0, 1.0); by default, this is the function :func:`.random`.
+
+   To shuffle an immutable sequence and return a new shuffled list, use
+   ``sample(x, k=len(x))`` instead.
+
+   Note that even for small ``len(x)``, the total number of permutations of *x*
+   can quickly grow larger than the period of most random number generators.
+   This implies that most permutations of a long sequence can never be
+   generated.  For example, a sequence of length 2080 is the largest that
+   can fit within the period of the Mersenne Twister random number generator.
 
 
 .. function:: sample(population, k)
@@ -175,9 +181,9 @@
    Members of the population need not be :term:`hashable` or unique.  If the population
    contains repeats, then each occurrence is a possible selection in the sample.
 
-   To choose a sample from a range of integers, use an :func:`range` object as an
+   To choose a sample from a range of integers, use a :func:`range` object as an
    argument.  This is especially fast and space efficient for sampling from a large
-   population:  ``sample(range(10000000), 60)``.
+   population:  ``sample(range(10000000), k=60)``.
 
    If the sample size is larger than the population size, a :exc:`ValueError`
    is raised.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 17 04:50:22 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Thu, 17 Nov 2016 09:50:22 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Small_edits_to?=
 =?utf-8?q?_the_docs_for_sample=28=29_and_shuffle=28=29=2E?=
Message-ID: <20161117095022.93412.13039.C7F96693@psf.io>

https://hg.python.org/cpython/rev/3e6f23fd0c34
changeset:   105178:3e6f23fd0c34
branch:      3.6
parent:      105176:e699f3a004ea
user:        Raymond Hettinger <python at rcn.com>
date:        Thu Nov 17 01:49:54 2016 -0800
summary:
  Small edits to the docs for sample() and shuffle().

files:
  Doc/library/random.rst |  22 ++++++++++++++--------
  1 files changed, 14 insertions(+), 8 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -152,13 +152,19 @@
 
 .. function:: shuffle(x[, random])
 
-   Shuffle the sequence *x* in place. The optional argument *random* is a
-   0-argument function returning a random float in [0.0, 1.0); by default, this is
-   the function :func:`.random`.
+   Shuffle the sequence *x* in place.
 
-   Note that for even rather small ``len(x)``, the total number of permutations of
-   *x* is larger than the period of most random number generators; this implies
-   that most permutations of a long sequence can never be generated.
+   The optional argument *random* is a 0-argument function returning a random
+   float in [0.0, 1.0); by default, this is the function :func:`.random`.
+
+   To shuffle an immutable sequence and return a new shuffled list, use
+   ``sample(x, k=len(x))`` instead.
+
+   Note that even for small ``len(x)``, the total number of permutations of *x*
+   can quickly grow larger than the period of most random number generators.
+   This implies that most permutations of a long sequence can never be
+   generated.  For example, a sequence of length 2080 is the largest that
+   can fit within the period of the Mersenne Twister random number generator.
 
 
 .. function:: sample(population, k)
@@ -175,9 +181,9 @@
    Members of the population need not be :term:`hashable` or unique.  If the population
    contains repeats, then each occurrence is a possible selection in the sample.
 
-   To choose a sample from a range of integers, use an :func:`range` object as an
+   To choose a sample from a range of integers, use a :func:`range` object as an
    argument.  This is especially fast and space efficient for sampling from a large
-   population:  ``sample(range(10000000), 60)``.
+   population:  ``sample(range(10000000), k=60)``.
 
    If the sample size is larger than the population size, a :exc:`ValueError`
    is raised.

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Thu Nov 17 11:26:52 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Thu, 17 Nov 2016 16:26:52 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python 2.7 2016-11-17
Message-ID: <511f2e21-4c8d-418c-861f-f2773ee21935@irsmsx104.ger.corp.intel.com>

No new revisions. Here are the previous results:

Results for project Python 2.7, build date 2016-11-17 03:02:17 +0000
commit:		ea91bb92c28b
previous commit:	6ca91a14a555
revision date:	2016-11-16 02:21:35 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.11%          1.01%          4.56%             7.07%
:-)             pybench      0.17%         -0.02%          5.73%             3.98%
:-(            regex_v8      0.56%          0.04%         -2.16%            10.99%
:-)               nbody      0.09%          0.00%          8.42%             3.92%
:-)        json_dump_v2      0.31%          0.18%          3.64%             9.81%
:-|      normal_startup      0.59%         -0.51%         -0.90%             2.54%
:-)             ssbench      0.17%         -0.16%          2.30%             2.23%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-2-7-2016-11-17/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Thu Nov 17 11:30:04 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Thu, 17 Nov 2016 16:30:04 +0000
Subject: [Python-checkins] UGLY Benchmark Results for Python Default
 2016-11-17
Message-ID: <e1f6f0bd-ba2d-4950-a820-2733e00c1fef@irsmsx104.ger.corp.intel.com>

Results for project Python default, build date 2016-11-17 13:11:29 +0000
commit:		53f5e16695d0
previous commit:	2faac299acaa
revision date:	2016-11-17 09:50:16 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.21%         -1.05%          4.68%            16.70%
:-)             pybench      0.19%          0.02%          5.74%             3.95%
:-(            regex_v8      3.90%          1.52%         -2.24%             3.25%
:-)               nbody      0.12%          4.07%          5.51%             5.12%
:-(        json_dump_v2      0.41%         -1.60%         -6.35%            10.64%
:-(      normal_startup      0.23%         -2.00%         -1.41%             7.44%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-default-2016-11-17/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Fri Nov 18 00:32:50 2016
From: python-checkins at python.org (zach.ware)
Date: Fri, 18 Nov 2016 05:32:50 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Ignore_newly_a?=
 =?utf-8?q?dded_suspicious_line?=
Message-ID: <20161118053250.47948.25393.9363CFC8@psf.io>

https://hg.python.org/cpython/rev/728c473798b3
changeset:   105180:728c473798b3
branch:      3.6
parent:      105178:3e6f23fd0c34
user:        Zachary Ware <zachary.ware at gmail.com>
date:        Thu Nov 17 23:30:27 2016 -0600
summary:
  Ignore newly added suspicious line

files:
  Doc/tools/susp-ignored.csv |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv
--- a/Doc/tools/susp-ignored.csv
+++ b/Doc/tools/susp-ignored.csv
@@ -189,6 +189,7 @@
 library/profile,,:lineno,filename:lineno(function)
 library/pyexpat,,:elem1,<py:elem1 />
 library/pyexpat,,:py,"xmlns:py = ""http://www.python.org/ns/"">"
+library/random,,:len,new_diff = mean(combined[:len(drug)]) - mean(combined[len(drug):])
 library/smtplib,,:port,method must support that as well as a regular host:port
 library/socket,,::,'5aef:2b::8'
 library/socket,,:can,"return (can_id, can_dlc, data[:can_dlc])"

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 18 00:32:50 2016
From: python-checkins at python.org (zach.ware)
Date: Fri, 18 Nov 2016 05:32:50 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_with_3=2E6?=
Message-ID: <20161118053250.2149.37440.C7BDEA44@psf.io>

https://hg.python.org/cpython/rev/d0ed9643cd57
changeset:   105181:d0ed9643cd57
parent:      105179:53f5e16695d0
parent:      105180:728c473798b3
user:        Zachary Ware <zachary.ware at gmail.com>
date:        Thu Nov 17 23:31:29 2016 -0600
summary:
  Merge with 3.6

files:
  Doc/tools/susp-ignored.csv |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv
--- a/Doc/tools/susp-ignored.csv
+++ b/Doc/tools/susp-ignored.csv
@@ -189,6 +189,7 @@
 library/profile,,:lineno,filename:lineno(function)
 library/pyexpat,,:elem1,<py:elem1 />
 library/pyexpat,,:py,"xmlns:py = ""http://www.python.org/ns/"">"
+library/random,,:len,new_diff = mean(combined[:len(drug)]) - mean(combined[len(drug):])
 library/smtplib,,:port,method must support that as well as a regular host:port
 library/socket,,::,'5aef:2b::8'
 library/socket,,:can,"return (can_id, can_dlc, data[:can_dlc])"

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Fri Nov 18 04:05:30 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Fri, 18 Nov 2016 09:05:30 +0000
Subject: [Python-checkins] Daily reference leaks (d0ed9643cd57): sum=11
Message-ID: <20161118090530.48863.54816.E886DAF7@psf.io>

results for d0ed9643cd57 on branch "default"
--------------------------------------------

test_collections leaked [0, 7, 0] memory blocks, sum=7
test_functools leaked [0, 3, 1] memory blocks, sum=4


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog0sK62Y', '--timeout', '7200']

From lp_benchmark_robot at intel.com  Fri Nov 18 08:29:12 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Fri, 18 Nov 2016 13:29:12 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python 2.7 2016-11-18
Message-ID: <fb03f952-756e-457d-bbc2-6e2ba63979cd@irsmsx151.ger.corp.intel.com>

No new revisions. Here are the previous results:

Results for project Python 2.7, build date 2016-11-18 03:04:06 +0000
commit:		ea91bb92c28b
previous commit:	6ca91a14a555
revision date:	2016-11-16 02:21:35 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.11%          1.01%          4.56%             7.07%
:-)             pybench      0.17%         -0.02%          5.73%             3.98%
:-(            regex_v8      0.56%          0.04%         -2.16%            10.99%
:-)               nbody      0.09%          0.00%          8.42%             3.92%
:-)        json_dump_v2      0.31%          0.18%          3.64%             9.81%
:-|      normal_startup      0.59%         -0.51%         -0.90%             2.54%
:-)             ssbench      0.17%         -0.16%          2.30%             2.23%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-2-7-2016-11-18/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Fri Nov 18 08:29:41 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Fri, 18 Nov 2016 13:29:41 +0000
Subject: [Python-checkins] UGLY Benchmark Results for Python Default
 2016-11-18
Message-ID: <df2a7295-dde4-4f98-97e1-65ba41347bce@irsmsx151.ger.corp.intel.com>

No new revisions. Here are the previous results:

Results for project Python default, build date 2016-11-18 03:01:59 +0000
commit:		53f5e16695d0
previous commit:	2faac299acaa
revision date:	2016-11-17 09:50:16 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.21%         -1.05%          4.68%            16.70%
:-)             pybench      0.19%          0.02%          5.74%             3.95%
:-(            regex_v8      3.90%          1.52%         -2.24%             3.25%
:-)               nbody      0.12%          4.07%          5.51%             5.12%
:-(        json_dump_v2      0.41%         -1.60%         -6.35%            10.64%
:-(      normal_startup      0.23%         -2.00%         -1.41%             7.44%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-default-2016-11-18/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Fri Nov 18 13:28:15 2016
From: python-checkins at python.org (brett.cannon)
Date: Fri, 18 Nov 2016 18:28:15 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2324452=3A_add_attr?=
 =?utf-8?q?ibution?=
Message-ID: <20161118182815.48911.92236.0303C160@psf.io>

https://hg.python.org/cpython/rev/0c8270cbdc62
changeset:   105182:0c8270cbdc62
user:        Brett Cannon <brett at python.org>
date:        Fri Nov 18 10:28:10 2016 -0800
summary:
  Issue #24452: add attribution

files:
  Misc/NEWS |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -219,7 +219,8 @@
   later. See `setuptools 100
   <https://github.com/pypa/setuptools/issues/100>`_ for rationale.
 
-- Issue #24452: Make webbrowser support Chrome on Mac OS X.
+- Issue #24452: Make webbrowser support Chrome on Mac OS X. Patch by
+  Ned Batchelder.
 
 - Issue #20766: Fix references leaked by pdb in the handling of SIGINT
   handlers.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 18 13:42:15 2016
From: python-checkins at python.org (brett.cannon)
Date: Fri, 18 Nov 2016 18:42:15 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_for_issue_=2328705?=
Message-ID: <20161118184215.47248.20364.FE34B4EB@psf.io>

https://hg.python.org/cpython/rev/89e2201142f9
changeset:   105184:89e2201142f9
parent:      105182:0c8270cbdc62
parent:      105183:a0a3dab4ed66
user:        Brett Cannon <brett at python.org>
date:        Fri Nov 18 10:42:10 2016 -0800
summary:
  Merge for issue #28705

files:
  Doc/faq/design.rst |  32 +++++---------------------------
  1 files changed, 5 insertions(+), 27 deletions(-)


diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst
--- a/Doc/faq/design.rst
+++ b/Doc/faq/design.rst
@@ -366,33 +366,11 @@
 Can Python be compiled to machine code, C or some other language?
 -----------------------------------------------------------------
 
-Practical answer:
-
-`Cython <http://cython.org/>`_ and `Pyrex <https://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/>`_
-compile a modified version of Python with optional annotations into C
-extensions.  `Weave <https://scipy.github.io/devdocs/tutorial/weave.html>`_ makes it easy to
-intermingle Python and C code in various ways to increase performance.
-`Nuitka <http://www.nuitka.net/>`_ is an up-and-coming compiler of Python
-into C++ code, aiming to support the full Python language.
-
-Theoretical answer:
-
-   .. XXX not sure what to make of this
-
-Not trivially.  Python's high level data types, dynamic typing of objects and
-run-time invocation of the interpreter (using :func:`eval` or :func:`exec`)
-together mean that a na?vely "compiled" Python program would probably consist
-mostly of calls into the Python run-time system, even for seemingly simple
-operations like ``x+1``.
-
-Several projects described in the Python newsgroup or at past `Python
-conferences <https://www.python.org/community/workshops/>`_ have shown that this
-approach is feasible, although the speedups reached so far are only modest
-(e.g. 2x).  Jython uses the same strategy for compiling to Java bytecode.  (Jim
-Hugunin has demonstrated that in combination with whole-program analysis,
-speedups of 1000x are feasible for small demo programs.  See the proceedings
-from the `1997 Python conference
-<http://legacy.python.org/workshops/1997-10/proceedings/>`_ for more information.)
+`Cython <http://cython.org/>`_ compiles a modified version of Python with
+optional annotations into C extensions.  `Nuitka <http://www.nuitka.net/>`_ is
+an up-and-coming compiler of Python into C++ code, aiming to support the full
+Python language. For compiling to Java you can consider
+`VOC <https://voc.readthedocs.io>`_.
 
 
 How does Python manage memory?

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 18 13:42:21 2016
From: python-checkins at python.org (brett.cannon)
Date: Fri, 18 Nov 2016 18:42:21 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NzA1?=
 =?utf-8?q?=3A_greatly_simplify_the_FAQ_entry_on_transpiling=2E?=
Message-ID: <20161118184215.2291.99379.682DCE4C@psf.io>

https://hg.python.org/cpython/rev/a0a3dab4ed66
changeset:   105183:a0a3dab4ed66
branch:      3.6
parent:      105180:728c473798b3
user:        Brett Cannon <brett at python.org>
date:        Fri Nov 18 10:41:28 2016 -0800
summary:
  Issue #28705: greatly simplify the FAQ entry on transpiling.

This also eliminats a dead link to Weave in the process.

files:
  Doc/faq/design.rst |  32 +++++---------------------------
  1 files changed, 5 insertions(+), 27 deletions(-)


diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst
--- a/Doc/faq/design.rst
+++ b/Doc/faq/design.rst
@@ -366,33 +366,11 @@
 Can Python be compiled to machine code, C or some other language?
 -----------------------------------------------------------------
 
-Practical answer:
-
-`Cython <http://cython.org/>`_ and `Pyrex <https://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/>`_
-compile a modified version of Python with optional annotations into C
-extensions.  `Weave <https://scipy.github.io/devdocs/tutorial/weave.html>`_ makes it easy to
-intermingle Python and C code in various ways to increase performance.
-`Nuitka <http://www.nuitka.net/>`_ is an up-and-coming compiler of Python
-into C++ code, aiming to support the full Python language.
-
-Theoretical answer:
-
-   .. XXX not sure what to make of this
-
-Not trivially.  Python's high level data types, dynamic typing of objects and
-run-time invocation of the interpreter (using :func:`eval` or :func:`exec`)
-together mean that a na?vely "compiled" Python program would probably consist
-mostly of calls into the Python run-time system, even for seemingly simple
-operations like ``x+1``.
-
-Several projects described in the Python newsgroup or at past `Python
-conferences <https://www.python.org/community/workshops/>`_ have shown that this
-approach is feasible, although the speedups reached so far are only modest
-(e.g. 2x).  Jython uses the same strategy for compiling to Java bytecode.  (Jim
-Hugunin has demonstrated that in combination with whole-program analysis,
-speedups of 1000x are feasible for small demo programs.  See the proceedings
-from the `1997 Python conference
-<http://legacy.python.org/workshops/1997-10/proceedings/>`_ for more information.)
+`Cython <http://cython.org/>`_ compiles a modified version of Python with
+optional annotations into C extensions.  `Nuitka <http://www.nuitka.net/>`_ is
+an up-and-coming compiler of Python into C++ code, aiming to support the full
+Python language. For compiling to Java you can consider
+`VOC <https://voc.readthedocs.io>`_.
 
 
 How does Python manage memory?

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 18 20:19:26 2016
From: python-checkins at python.org (martin.panter)
Date: Sat, 19 Nov 2016 01:19:26 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328548=3A_Parse_HT?=
 =?utf-8?q?TP_request_version_even_if_too_many_words_received?=
Message-ID: <20161119011926.41115.49700.5A75B016@psf.io>

https://hg.python.org/cpython/rev/7c98768368cb
changeset:   105185:7c98768368cb
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sat Nov 19 01:06:37 2016 +0000
summary:
  Issue #28548: Parse HTTP request version even if too many words received

files:
  Lib/http/server.py           |  33 +++++++++++++----------
  Lib/test/test_httpservers.py |  10 +++++++
  Misc/NEWS                    |   3 ++
  3 files changed, 31 insertions(+), 15 deletions(-)


diff --git a/Lib/http/server.py b/Lib/http/server.py
--- a/Lib/http/server.py
+++ b/Lib/http/server.py
@@ -267,8 +267,8 @@
         are in self.command, self.path, self.request_version and
         self.headers.
 
-        Return True for success, False for failure; on failure, an
-        error is sent back.
+        Return True for success, False for failure; on failure, any relevant
+        error response has already been sent back.
 
         """
         self.command = None  # set in case of error on the first line
@@ -278,10 +278,13 @@
         requestline = requestline.rstrip('\r\n')
         self.requestline = requestline
         words = requestline.split()
-        if len(words) == 3:
-            command, path, version = words
+        if len(words) == 0:
+            return False
+
+        if len(words) >= 3:  # Enough to determine protocol version
+            version = words[-1]
             try:
-                if version[:5] != 'HTTP/':
+                if not version.startswith('HTTP/'):
                     raise ValueError
                 base_version_number = version.split('/', 1)[1]
                 version_number = base_version_number.split(".")
@@ -306,22 +309,22 @@
                     HTTPStatus.HTTP_VERSION_NOT_SUPPORTED,
                     "Invalid HTTP version (%s)" % base_version_number)
                 return False
-        elif len(words) == 2:
-            command, path = words
+            self.request_version = version
+
+        if not 2 <= len(words) <= 3:
+            self.send_error(
+                HTTPStatus.BAD_REQUEST,
+                "Bad request syntax (%r)" % requestline)
+            return False
+        command, path = words[:2]
+        if len(words) == 2:
             self.close_connection = True
             if command != 'GET':
                 self.send_error(
                     HTTPStatus.BAD_REQUEST,
                     "Bad HTTP/0.9 request type (%r)" % command)
                 return False
-        elif not words:
-            return False
-        else:
-            self.send_error(
-                HTTPStatus.BAD_REQUEST,
-                "Bad request syntax (%r)" % requestline)
-            return False
-        self.command, self.path, self.request_version = command, path, version
+        self.command, self.path = command, path
 
         # Examine the headers and look for a Connection directive.
         try:
diff --git a/Lib/test/test_httpservers.py b/Lib/test/test_httpservers.py
--- a/Lib/test/test_httpservers.py
+++ b/Lib/test/test_httpservers.py
@@ -822,6 +822,16 @@
         self.assertEqual(result[0], b'<html><body>Data</body></html>\r\n')
         self.verify_get_called()
 
+    def test_extra_space(self):
+        result = self.send_typical_request(
+            b'GET /spaced out HTTP/1.1\r\n'
+            b'Host: dummy\r\n'
+            b'\r\n'
+        )
+        self.assertTrue(result[0].startswith(b'HTTP/1.1 400 '))
+        self.verify_expected_headers(result[1:result.index(b'\r\n')])
+        self.assertFalse(self.handler.get_called)
+
     def test_with_continue_1_0(self):
         result = self.send_typical_request(b'GET / HTTP/1.0\r\nExpect: 100-continue\r\n\r\n')
         self.verify_http_server_response(result[0])
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -128,6 +128,9 @@
 Library
 -------
 
+- Issue #28548: In the "http.server" module, parse the protocol version if
+  possible, to avoid using HTTP 0.9 in some error responses.
+
 - Issue #19717: Makes Path.resolve() succeed on paths that do not exist.
   Patch by Vajrasky Kok
 

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Sat Nov 19 04:59:13 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sat, 19 Nov 2016 09:59:13 +0000
Subject: [Python-checkins] Daily reference leaks (7c98768368cb): sum=4
Message-ID: <20161119095913.45076.8579.80BD4AD7@psf.io>

results for 7c98768368cb on branch "default"
--------------------------------------------

test_functools leaked [0, 3, 1] memory blocks, sum=4


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogC9sSOK', '--timeout', '7200']

From python-checkins at python.org  Sat Nov 19 10:21:17 2016
From: python-checkins at python.org (xavier.degaye)
Date: Sat, 19 Nov 2016 15:21:17 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328746=3A_Merge_3=2E6?=
Message-ID: <20161119152116.46084.32929.F290F413@psf.io>

https://hg.python.org/cpython/rev/3248782c3176
changeset:   105187:3248782c3176
parent:      105185:7c98768368cb
parent:      105186:2fb2e3dc450e
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Sat Nov 19 16:20:31 2016 +0100
summary:
  Issue #28746: Merge 3.6

files:
  Misc/NEWS          |  3 +++
  Python/fileutils.c |  2 +-
  2 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #28746: Fix the set_inheritable() file descriptor method on platforms
+  that do not have the ioctl FIOCLEX and FIONCLEX commands.
+
 - Issue #26920: Fix not getting the locale's charset upon initializing the
   interpreter, on platforms that do not have langinfo.
 
diff --git a/Python/fileutils.c b/Python/fileutils.c
--- a/Python/fileutils.c
+++ b/Python/fileutils.c
@@ -886,7 +886,7 @@
         return 0;
     }
 
-    res = fcntl(fd, F_SETFD, flags);
+    res = fcntl(fd, F_SETFD, new_flags);
     if (res < 0) {
         if (raise)
             PyErr_SetFromErrno(PyExc_OSError);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 19 10:21:17 2016
From: python-checkins at python.org (xavier.degaye)
Date: Sat, 19 Nov 2016 15:21:17 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NzQ2?=
 =?utf-8?q?=3A_Fix_the_set=5Finheritable=28=29_file_descriptor_method_on_p?=
 =?utf-8?q?latforms?=
Message-ID: <20161119152116.12421.37303.BFE01026@psf.io>

https://hg.python.org/cpython/rev/2fb2e3dc450e
changeset:   105186:2fb2e3dc450e
branch:      3.6
parent:      105183:a0a3dab4ed66
user:        Xavier de Gaye <xdegaye at users.sourceforge.net>
date:        Sat Nov 19 16:19:29 2016 +0100
summary:
  Issue #28746: Fix the set_inheritable() file descriptor method on platforms
that do not have the ioctl FIOCLEX and FIONCLEX commands

files:
  Misc/NEWS          |  3 +++
  Python/fileutils.c |  2 +-
  2 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #28746: Fix the set_inheritable() file descriptor method on platforms
+  that do not have the ioctl FIOCLEX and FIONCLEX commands.
+
 - Issue #26920: Fix not getting the locale's charset upon initializing the
   interpreter, on platforms that do not have langinfo.
 
diff --git a/Python/fileutils.c b/Python/fileutils.c
--- a/Python/fileutils.c
+++ b/Python/fileutils.c
@@ -886,7 +886,7 @@
         return 0;
     }
 
-    res = fcntl(fd, F_SETFD, flags);
+    res = fcntl(fd, F_SETFD, new_flags);
     if (res < 0) {
         if (raise)
             PyErr_SetFromErrno(PyExc_OSError);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 19 13:33:28 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Sat, 19 Nov 2016 18:33:28 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NTU2?=
 =?utf-8?q?=3A_two_more_small_upstream_changes_by_Ivan_Levkivskyi_=28=2332?=
 =?utf-8?b?OSwgIzMzMCk=?=
Message-ID: <20161119183328.45713.5799.83764F7B@psf.io>

https://hg.python.org/cpython/rev/1e49abb03e0f
changeset:   105188:1e49abb03e0f
branch:      3.5
parent:      105165:20f62e4a9c2f
user:        Guido van Rossum <guido at python.org>
date:        Sat Nov 19 10:32:41 2016 -0800
summary:
  Issue #28556: two more small upstream changes by Ivan Levkivskyi (#329, #330)

files:
  Lib/test/test_typing.py |  21 +++++++++++++++++++--
  Lib/typing.py           |   2 ++
  2 files changed, 21 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -45,6 +45,10 @@
                 message += ' : %s' % msg
             raise self.failureException(message)
 
+    def clear_caches(self):
+        for f in typing._cleanups:
+            f()
+
 
 class Employee:
     pass
@@ -509,6 +513,13 @@
     def test_protocol_instance_type_error(self):
         with self.assertRaises(TypeError):
             isinstance(0, typing.SupportsAbs)
+        class C1(typing.SupportsInt):
+            def __int__(self) -> int:
+                return 42
+        class C2(C1):
+            pass
+        c = C2()
+        self.assertIsInstance(c, C1)
 
 
 class GenericTests(BaseTestCase):
@@ -748,8 +759,12 @@
         class CC: ...
         self.assertEqual(get_type_hints(foobar, globals(), locals()), {'x': List[List[CC]]})
         T = TypeVar('T')
-        def barfoo(x: Tuple[T, ...]): ...
-        self.assertIs(get_type_hints(barfoo, globals(), locals())['x'], Tuple[T, ...])
+        AT = Tuple[T, ...]
+        def barfoo(x: AT): ...
+        self.assertIs(get_type_hints(barfoo, globals(), locals())['x'], AT)
+        CT = Callable[..., List[T]]
+        def barfoo2(x: CT): ...
+        self.assertIs(get_type_hints(barfoo2, globals(), locals())['x'], CT)
 
     def test_extended_generic_rules_subclassing(self):
         class T1(Tuple[T, KT]): ...
@@ -800,6 +815,8 @@
 
     def test_type_erasure_special(self):
         T = TypeVar('T')
+        # this is the only test that checks type caching
+        self.clear_caches()
         class MyTup(Tuple[T, T]): ...
         self.assertIs(MyTup[int]().__class__, MyTup)
         self.assertIs(MyTup[int]().__orig_class__, MyTup[int])
diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -1503,6 +1503,8 @@
     """
 
     def __instancecheck__(self, obj):
+        if _Protocol not in self.__bases__:
+            return super().__instancecheck__(obj)
         raise TypeError("Protocols cannot be used with isinstance().")
 
     def __subclasscheck__(self, cls):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 19 13:33:28 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Sat, 19 Nov 2016 18:33:28 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328556=3A_two_more_small_upstream_changes_by_Ivan_Levk?=
 =?utf-8?b?aXZza3lpICgjMzI5LCAjMzMwKQ==?=
Message-ID: <20161119183328.45947.39852.21400ABA@psf.io>

https://hg.python.org/cpython/rev/cdddf4ee0e00
changeset:   105189:cdddf4ee0e00
branch:      3.6
parent:      105186:2fb2e3dc450e
parent:      105188:1e49abb03e0f
user:        Guido van Rossum <guido at python.org>
date:        Sat Nov 19 10:32:43 2016 -0800
summary:
  Issue #28556: two more small upstream changes by Ivan Levkivskyi (#329, #330) (3.5->3.6)

files:
  Lib/test/test_typing.py |  21 +++++++++++++++++++--
  Lib/typing.py           |   2 ++
  2 files changed, 21 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -45,6 +45,10 @@
                 message += ' : %s' % msg
             raise self.failureException(message)
 
+    def clear_caches(self):
+        for f in typing._cleanups:
+            f()
+
 
 class Employee:
     pass
@@ -509,6 +513,13 @@
     def test_protocol_instance_type_error(self):
         with self.assertRaises(TypeError):
             isinstance(0, typing.SupportsAbs)
+        class C1(typing.SupportsInt):
+            def __int__(self) -> int:
+                return 42
+        class C2(C1):
+            pass
+        c = C2()
+        self.assertIsInstance(c, C1)
 
 
 class GenericTests(BaseTestCase):
@@ -748,8 +759,12 @@
         class CC: ...
         self.assertEqual(get_type_hints(foobar, globals(), locals()), {'x': List[List[CC]]})
         T = TypeVar('T')
-        def barfoo(x: Tuple[T, ...]): ...
-        self.assertIs(get_type_hints(barfoo, globals(), locals())['x'], Tuple[T, ...])
+        AT = Tuple[T, ...]
+        def barfoo(x: AT): ...
+        self.assertIs(get_type_hints(barfoo, globals(), locals())['x'], AT)
+        CT = Callable[..., List[T]]
+        def barfoo2(x: CT): ...
+        self.assertIs(get_type_hints(barfoo2, globals(), locals())['x'], CT)
 
     def test_extended_generic_rules_subclassing(self):
         class T1(Tuple[T, KT]): ...
@@ -800,6 +815,8 @@
 
     def test_type_erasure_special(self):
         T = TypeVar('T')
+        # this is the only test that checks type caching
+        self.clear_caches()
         class MyTup(Tuple[T, T]): ...
         self.assertIs(MyTup[int]().__class__, MyTup)
         self.assertIs(MyTup[int]().__orig_class__, MyTup[int])
diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -1503,6 +1503,8 @@
     """
 
     def __instancecheck__(self, obj):
+        if _Protocol not in self.__bases__:
+            return super().__instancecheck__(obj)
         raise TypeError("Protocols cannot be used with isinstance().")
 
     def __subclasscheck__(self, cls):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 19 13:33:30 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Sat, 19 Nov 2016 18:33:30 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328556=3A_two_more_small_upstream_changes_by_Iva?=
 =?utf-8?q?n_Levkivskyi_=28=23329=2C_=23330=29?=
Message-ID: <20161119183328.5199.97737.EFB55440@psf.io>

https://hg.python.org/cpython/rev/1465baaccd84
changeset:   105190:1465baaccd84
parent:      105187:3248782c3176
parent:      105189:cdddf4ee0e00
user:        Guido van Rossum <guido at python.org>
date:        Sat Nov 19 10:32:44 2016 -0800
summary:
  Issue #28556: two more small upstream changes by Ivan Levkivskyi (#329, #330) (3.6->3.7)

files:
  Lib/test/test_typing.py |  21 +++++++++++++++++++--
  Lib/typing.py           |   2 ++
  2 files changed, 21 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -45,6 +45,10 @@
                 message += ' : %s' % msg
             raise self.failureException(message)
 
+    def clear_caches(self):
+        for f in typing._cleanups:
+            f()
+
 
 class Employee:
     pass
@@ -509,6 +513,13 @@
     def test_protocol_instance_type_error(self):
         with self.assertRaises(TypeError):
             isinstance(0, typing.SupportsAbs)
+        class C1(typing.SupportsInt):
+            def __int__(self) -> int:
+                return 42
+        class C2(C1):
+            pass
+        c = C2()
+        self.assertIsInstance(c, C1)
 
 
 class GenericTests(BaseTestCase):
@@ -748,8 +759,12 @@
         class CC: ...
         self.assertEqual(get_type_hints(foobar, globals(), locals()), {'x': List[List[CC]]})
         T = TypeVar('T')
-        def barfoo(x: Tuple[T, ...]): ...
-        self.assertIs(get_type_hints(barfoo, globals(), locals())['x'], Tuple[T, ...])
+        AT = Tuple[T, ...]
+        def barfoo(x: AT): ...
+        self.assertIs(get_type_hints(barfoo, globals(), locals())['x'], AT)
+        CT = Callable[..., List[T]]
+        def barfoo2(x: CT): ...
+        self.assertIs(get_type_hints(barfoo2, globals(), locals())['x'], CT)
 
     def test_extended_generic_rules_subclassing(self):
         class T1(Tuple[T, KT]): ...
@@ -800,6 +815,8 @@
 
     def test_type_erasure_special(self):
         T = TypeVar('T')
+        # this is the only test that checks type caching
+        self.clear_caches()
         class MyTup(Tuple[T, T]): ...
         self.assertIs(MyTup[int]().__class__, MyTup)
         self.assertIs(MyTup[int]().__orig_class__, MyTup[int])
diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -1503,6 +1503,8 @@
     """
 
     def __instancecheck__(self, obj):
+        if _Protocol not in self.__bases__:
+            return super().__instancecheck__(obj)
         raise TypeError("Protocols cannot be used with isinstance().")
 
     def __subclasscheck__(self, cls):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 19 21:42:05 2016
From: python-checkins at python.org (steve.dower)
Date: Sun, 20 Nov 2016 02:42:05 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_from_3=2E5_and_fix_a_few_other_functions_missing_IPH_han?=
 =?utf-8?q?dling=2E?=
Message-ID: <20161120024205.12760.86609.D22E011E@psf.io>

https://hg.python.org/cpython/rev/b3a6cc610ee4
changeset:   105192:b3a6cc610ee4
branch:      3.6
parent:      105189:cdddf4ee0e00
parent:      105191:02f416441def
user:        Steve Dower <steve.dower at microsoft.com>
date:        Sat Nov 19 18:41:16 2016 -0800
summary:
  Merge from 3.5 and fix a few other functions missing IPH handling.

files:
  Misc/NEWS             |  2 ++
  Modules/posixmodule.c |  9 +++++++--
  2 files changed, 9 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -635,6 +635,8 @@
 Library
 -------
 
+- Issue #28732: Fix crash in os.spawnv() with no elements in args
+
 - Issue #28485: Always raise ValueError for negative
   compileall.compile_dir(workers=...) parameter, even when multithreading is
   unavailable.
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -3347,8 +3347,7 @@
 
 #ifdef MS_WINDOWS
     Py_BEGIN_ALLOW_THREADS
-    if (src->wide)
-        result = CreateHardLinkW(dst->wide, src->wide, NULL);
+    result = CreateHardLinkW(dst->wide, src->wide, NULL);
     Py_END_ALLOW_THREADS
 
     if (!result)
@@ -4111,7 +4110,9 @@
 {
     long result;
     Py_BEGIN_ALLOW_THREADS
+    _Py_BEGIN_SUPPRESS_IPH
     result = _wsystem(command);
+    _Py_END_SUPPRESS_IPH
     Py_END_ALLOW_THREADS
     return result;
 }
@@ -6889,7 +6890,9 @@
 
     do {
         Py_BEGIN_ALLOW_THREADS
+        _Py_BEGIN_SUPPRESS_IPH
         res = _cwait(&status, pid, options);
+        _Py_END_SUPPRESS_IPH
         Py_END_ALLOW_THREADS
     } while (res < 0 && errno == EINTR && !(async_err = PyErr_CheckSignals()));
     if (res < 0)
@@ -8256,6 +8259,7 @@
     attr.bInheritHandle = FALSE;
 
     Py_BEGIN_ALLOW_THREADS
+    _Py_BEGIN_SUPPRESS_IPH
     ok = CreatePipe(&read, &write, &attr, 0);
     if (ok) {
         fds[0] = _open_osfhandle((intptr_t)read, _O_RDONLY);
@@ -8266,6 +8270,7 @@
             ok = 0;
         }
     }
+    _Py_END_SUPPRESS_IPH
     Py_END_ALLOW_THREADS
 
     if (!ok)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 19 21:42:06 2016
From: python-checkins at python.org (steve.dower)
Date: Sun, 20 Nov 2016 02:42:06 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_from_3=2E6?=
Message-ID: <20161120024205.7475.53570.520C2180@psf.io>

https://hg.python.org/cpython/rev/22dee3a9cad9
changeset:   105193:22dee3a9cad9
parent:      105190:1465baaccd84
parent:      105192:b3a6cc610ee4
user:        Steve Dower <steve.dower at microsoft.com>
date:        Sat Nov 19 18:41:31 2016 -0800
summary:
  Merge from 3.6

files:
  Misc/NEWS             |  2 ++
  Modules/posixmodule.c |  9 +++++++--
  2 files changed, 9 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -622,6 +622,8 @@
 Library
 -------
 
+- Issue #28732: Fix crash in os.spawnv() with no elements in args
+
 - Issue #28485: Always raise ValueError for negative
   compileall.compile_dir(workers=...) parameter, even when multithreading is
   unavailable.
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -3347,8 +3347,7 @@
 
 #ifdef MS_WINDOWS
     Py_BEGIN_ALLOW_THREADS
-    if (src->wide)
-        result = CreateHardLinkW(dst->wide, src->wide, NULL);
+    result = CreateHardLinkW(dst->wide, src->wide, NULL);
     Py_END_ALLOW_THREADS
 
     if (!result)
@@ -4111,7 +4110,9 @@
 {
     long result;
     Py_BEGIN_ALLOW_THREADS
+    _Py_BEGIN_SUPPRESS_IPH
     result = _wsystem(command);
+    _Py_END_SUPPRESS_IPH
     Py_END_ALLOW_THREADS
     return result;
 }
@@ -6889,7 +6890,9 @@
 
     do {
         Py_BEGIN_ALLOW_THREADS
+        _Py_BEGIN_SUPPRESS_IPH
         res = _cwait(&status, pid, options);
+        _Py_END_SUPPRESS_IPH
         Py_END_ALLOW_THREADS
     } while (res < 0 && errno == EINTR && !(async_err = PyErr_CheckSignals()));
     if (res < 0)
@@ -8256,6 +8259,7 @@
     attr.bInheritHandle = FALSE;
 
     Py_BEGIN_ALLOW_THREADS
+    _Py_BEGIN_SUPPRESS_IPH
     ok = CreatePipe(&read, &write, &attr, 0);
     if (ok) {
         fds[0] = _open_osfhandle((intptr_t)read, _O_RDONLY);
@@ -8266,6 +8270,7 @@
             ok = 0;
         }
     }
+    _Py_END_SUPPRESS_IPH
     Py_END_ALLOW_THREADS
 
     if (!ok)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 19 21:42:05 2016
From: python-checkins at python.org (steve.dower)
Date: Sun, 20 Nov 2016 02:42:05 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NzMy?=
 =?utf-8?q?=3A_Fix_crash_in_os=2Espawnv=28=29_with_no_elements_in_args?=
Message-ID: <20161120024205.5421.82878.02402411@psf.io>

https://hg.python.org/cpython/rev/02f416441def
changeset:   105191:02f416441def
branch:      3.5
parent:      105188:1e49abb03e0f
user:        Steve Dower <steve.dower at microsoft.com>
date:        Sat Nov 19 18:33:39 2016 -0800
summary:
  Issue #28732: Fix crash in os.spawnv() with no elements in args
Prevents crashes in some other posixmodule.c functions

files:
  Misc/NEWS             |   2 ++
  Modules/posixmodule.c |  16 ++++++++++++++++
  2 files changed, 18 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -121,6 +121,8 @@
 Library
 -------
 
+- Issue #28732: Fix crash in os.spawnv() with no elements in args
+
 - Issue #28485: Always raise ValueError for negative
   compileall.compile_dir(workers=...) parameter, even when multithreading is
   unavailable.
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -5186,6 +5186,16 @@
                         "spawnv() arg 2 must be a tuple or list");
         return NULL;
     }
+#ifdef MS_WINDOWS
+    /* Avoid changing behavior in maintenance release, but
+       the previous Windows behavior was to crash, so this
+       is a "compatible" improvement. */
+    if (argc == 0) {
+        PyErr_SetString(PyExc_ValueError,
+                        "spawnv() arg 2 cannot be empty");
+        return NULL;
+    }
+#endif
 
     argvlist = PyMem_NEW(char *, argc+1);
     if (argvlist == NULL) {
@@ -5207,7 +5217,9 @@
         mode = _P_OVERLAY;
 
     Py_BEGIN_ALLOW_THREADS
+    _Py_BEGIN_SUPPRESS_IPH
     spawnval = _spawnv(mode, path_char, argvlist);
+    _Py_END_SUPPRESS_IPH
     Py_END_ALLOW_THREADS
 
     free_string_array(argvlist, argc);
@@ -5297,7 +5309,9 @@
         mode = _P_OVERLAY;
 
     Py_BEGIN_ALLOW_THREADS
+    _Py_BEGIN_SUPPRESS_IPH
     spawnval = _spawnve(mode, path_char, argvlist, envlist);
+    _Py_END_SUPPRESS_IPH
     Py_END_ALLOW_THREADS
 
     if (spawnval == -1)
@@ -7022,7 +7036,9 @@
 
     do {
         Py_BEGIN_ALLOW_THREADS
+        _Py_BEGIN_SUPPRESS_IPH
         res = _cwait(&status, pid, options);
+        _Py_END_SUPPRESS_IPH
         Py_END_ALLOW_THREADS
     } while (res < 0 && errno == EINTR && !(async_err = PyErr_CheckSignals()));
     if (res < 0)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 19 21:53:45 2016
From: python-checkins at python.org (steve.dower)
Date: Sun, 20 Nov 2016 02:53:45 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328732=3A_Raise_ValueError_when_os=2Espawn*=28?=
 =?utf-8?q?=29_is_passed_an_empty_tuple_of?=
Message-ID: <20161120025344.14486.22574.A6E01711@psf.io>

https://hg.python.org/cpython/rev/75824899f0dd
changeset:   105195:75824899f0dd
parent:      105193:22dee3a9cad9
parent:      105194:1a9e4b465497
user:        Steve Dower <steve.dower at microsoft.com>
date:        Sat Nov 19 18:53:36 2016 -0800
summary:
  Issue #28732: Raise ValueError when os.spawn*() is passed an empty tuple of arguments

files:
  Lib/test/test_os.py   |  21 +++++++++++++++++++++
  Modules/posixmodule.c |  10 ++++++++++
  2 files changed, 31 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
--- a/Lib/test/test_os.py
+++ b/Lib/test/test_os.py
@@ -2321,6 +2321,27 @@
         exitcode = os.spawnve(os.P_WAIT, args[0], args, self.env)
         self.assertEqual(exitcode, self.exitcode)
 
+    @requires_os_func('spawnl')
+    def test_spawnl_noargs(self):
+        args = self.create_args()
+        self.assertRaises(ValueError, os.spawnl, os.P_NOWAIT, args[0])
+
+    @requires_os_func('spawnle')
+    def test_spawnl_noargs(self):
+        args = self.create_args()
+        self.assertRaises(ValueError, os.spawnle, os.P_NOWAIT, args[0], {})
+
+    @requires_os_func('spawnv')
+    def test_spawnv_noargs(self):
+        args = self.create_args()
+        self.assertRaises(ValueError, os.spawnv, os.P_NOWAIT, args[0], ())
+        self.assertRaises(ValueError, os.spawnv, os.P_NOWAIT, args[0], [])
+
+    @requires_os_func('spawnve')
+    def test_spawnv_noargs(self):
+        args = self.create_args()
+        self.assertRaises(ValueError, os.spawnve, os.P_NOWAIT, args[0], (), {})
+        self.assertRaises(ValueError, os.spawnve, os.P_NOWAIT, args[0], [], {})
 
 # The introduction of this TestCase caused at least two different errors on
 # *nix buildbots. Temporarily skip this to let the buildbots move along.
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -5042,6 +5042,11 @@
                         "spawnv() arg 2 must be a tuple or list");
         return NULL;
     }
+    if (argc == 0) {
+        PyErr_SetString(PyExc_ValueError,
+            "spawnv() arg 2 cannot be empty");
+        return NULL;
+    }
 
     argvlist = PyMem_NEW(EXECV_CHAR *, argc+1);
     if (argvlist == NULL) {
@@ -5127,6 +5132,11 @@
                         "spawnve() arg 2 must be a tuple or list");
         goto fail_0;
     }
+    if (argc == 0) {
+        PyErr_SetString(PyExc_ValueError,
+            "spawnve() arg 2 cannot be empty");
+        goto fail_0;
+    }
     if (!PyMapping_Check(env)) {
         PyErr_SetString(PyExc_TypeError,
                         "spawnve() arg 3 must be a mapping object");

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 19 21:53:44 2016
From: python-checkins at python.org (steve.dower)
Date: Sun, 20 Nov 2016 02:53:44 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NzMy?=
 =?utf-8?q?=3A_Raise_ValueError_when_os=2Espawn*=28=29_is_passed_an_empty_?=
 =?utf-8?q?tuple_of?=
Message-ID: <20161120025344.12122.61607.BCEF1859@psf.io>

https://hg.python.org/cpython/rev/1a9e4b465497
changeset:   105194:1a9e4b465497
branch:      3.6
parent:      105192:b3a6cc610ee4
user:        Steve Dower <steve.dower at microsoft.com>
date:        Sat Nov 19 18:53:19 2016 -0800
summary:
  Issue #28732: Raise ValueError when os.spawn*() is passed an empty tuple of arguments

files:
  Lib/test/test_os.py   |  21 +++++++++++++++++++++
  Modules/posixmodule.c |  10 ++++++++++
  2 files changed, 31 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
--- a/Lib/test/test_os.py
+++ b/Lib/test/test_os.py
@@ -2321,6 +2321,27 @@
         exitcode = os.spawnve(os.P_WAIT, args[0], args, self.env)
         self.assertEqual(exitcode, self.exitcode)
 
+    @requires_os_func('spawnl')
+    def test_spawnl_noargs(self):
+        args = self.create_args()
+        self.assertRaises(ValueError, os.spawnl, os.P_NOWAIT, args[0])
+
+    @requires_os_func('spawnle')
+    def test_spawnl_noargs(self):
+        args = self.create_args()
+        self.assertRaises(ValueError, os.spawnle, os.P_NOWAIT, args[0], {})
+
+    @requires_os_func('spawnv')
+    def test_spawnv_noargs(self):
+        args = self.create_args()
+        self.assertRaises(ValueError, os.spawnv, os.P_NOWAIT, args[0], ())
+        self.assertRaises(ValueError, os.spawnv, os.P_NOWAIT, args[0], [])
+
+    @requires_os_func('spawnve')
+    def test_spawnv_noargs(self):
+        args = self.create_args()
+        self.assertRaises(ValueError, os.spawnve, os.P_NOWAIT, args[0], (), {})
+        self.assertRaises(ValueError, os.spawnve, os.P_NOWAIT, args[0], [], {})
 
 # The introduction of this TestCase caused at least two different errors on
 # *nix buildbots. Temporarily skip this to let the buildbots move along.
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -5042,6 +5042,11 @@
                         "spawnv() arg 2 must be a tuple or list");
         return NULL;
     }
+    if (argc == 0) {
+        PyErr_SetString(PyExc_ValueError,
+            "spawnv() arg 2 cannot be empty");
+        return NULL;
+    }
 
     argvlist = PyMem_NEW(EXECV_CHAR *, argc+1);
     if (argvlist == NULL) {
@@ -5127,6 +5132,11 @@
                         "spawnve() arg 2 must be a tuple or list");
         goto fail_0;
     }
+    if (argc == 0) {
+        PyErr_SetString(PyExc_ValueError,
+            "spawnve() arg 2 cannot be empty");
+        goto fail_0;
+    }
     if (!PyMapping_Check(env)) {
         PyErr_SetString(PyExc_TypeError,
                         "spawnve() arg 3 must be a mapping object");

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 19 22:17:57 2016
From: python-checkins at python.org (steve.dower)
Date: Sun, 20 Nov 2016 03:17:57 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328732=3A_Raise_ValueError_when_argv=5B0=5D_is_e?=
 =?utf-8?q?mpty?=
Message-ID: <20161120031757.45580.55402.9A4B91A0@psf.io>

https://hg.python.org/cpython/rev/fc6f757e53de
changeset:   105198:fc6f757e53de
parent:      105195:75824899f0dd
parent:      105197:af78b33704af
user:        Steve Dower <steve.dower at microsoft.com>
date:        Sat Nov 19 19:17:46 2016 -0800
summary:
  Issue #28732: Raise ValueError when argv[0] is empty

files:
  Lib/test/test_os.py   |  18 +++++++++++++-
  Modules/posixmodule.c |  36 ++++++++++++++++++++++++++++++-
  2 files changed, 51 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
--- a/Lib/test/test_os.py
+++ b/Lib/test/test_os.py
@@ -1481,8 +1481,16 @@
         self.assertRaises(OSError, os.execvpe, 'no such app-',
                           ['no such app-'], None)
 
+    def test_execv_with_bad_arglist(self):
+        self.assertRaises(ValueError, os.execv, 'notepad', ())
+        self.assertRaises(ValueError, os.execv, 'notepad', [])
+        self.assertRaises(ValueError, os.execv, 'notepad', ('',))
+        self.assertRaises(ValueError, os.execv, 'notepad', [''])
+
     def test_execvpe_with_bad_arglist(self):
         self.assertRaises(ValueError, os.execvpe, 'notepad', [], None)
+        self.assertRaises(ValueError, os.execvpe, 'notepad', [], {})
+        self.assertRaises(ValueError, os.execvpe, 'notepad', [''], {})
 
     @unittest.skipUnless(hasattr(os, '_execvpe'),
                          "No internal os._execvpe function to test.")
@@ -2325,23 +2333,29 @@
     def test_spawnl_noargs(self):
         args = self.create_args()
         self.assertRaises(ValueError, os.spawnl, os.P_NOWAIT, args[0])
+        self.assertRaises(ValueError, os.spawnl, os.P_NOWAIT, args[0], '')
 
     @requires_os_func('spawnle')
-    def test_spawnl_noargs(self):
+    def test_spawnle_noargs(self):
         args = self.create_args()
         self.assertRaises(ValueError, os.spawnle, os.P_NOWAIT, args[0], {})
+        self.assertRaises(ValueError, os.spawnle, os.P_NOWAIT, args[0], '', {})
 
     @requires_os_func('spawnv')
     def test_spawnv_noargs(self):
         args = self.create_args()
         self.assertRaises(ValueError, os.spawnv, os.P_NOWAIT, args[0], ())
         self.assertRaises(ValueError, os.spawnv, os.P_NOWAIT, args[0], [])
+        self.assertRaises(ValueError, os.spawnv, os.P_NOWAIT, args[0], ('',))
+        self.assertRaises(ValueError, os.spawnv, os.P_NOWAIT, args[0], [''])
 
     @requires_os_func('spawnve')
-    def test_spawnv_noargs(self):
+    def test_spawnve_noargs(self):
         args = self.create_args()
         self.assertRaises(ValueError, os.spawnve, os.P_NOWAIT, args[0], (), {})
         self.assertRaises(ValueError, os.spawnve, os.P_NOWAIT, args[0], [], {})
+        self.assertRaises(ValueError, os.spawnve, os.P_NOWAIT, args[0], ('',), {})
+        self.assertRaises(ValueError, os.spawnve, os.P_NOWAIT, args[0], [''], {})
 
 # The introduction of this TestCase caused at least two different errors on
 # *nix buildbots. Temporarily skip this to let the buildbots move along.
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -4916,12 +4916,20 @@
     if (argvlist == NULL) {
         return NULL;
     }
-
+    if (!argvlist[0][0]) {
+        PyErr_SetString(PyExc_ValueError,
+            "execv() arg 2 first element cannot be empty");
+        free_string_array(argvlist, argc);
+        return NULL;
+    }
+
+    _Py_BEGIN_SUPPRESS_IPH
 #ifdef HAVE_WEXECV
     _wexecv(path->wide, argvlist);
 #else
     execv(path->narrow, argvlist);
 #endif
+    _Py_END_SUPPRESS_IPH
 
     /* If we get here it's definitely an error */
 
@@ -4961,6 +4969,11 @@
         goto fail;
     }
     argc = PySequence_Size(argv);
+    if (argc < 1) {
+        PyErr_SetString(PyExc_ValueError, "execve: argv must not be empty");
+        return NULL;
+    }
+
     if (!PyMapping_Check(env)) {
         PyErr_SetString(PyExc_TypeError,
                         "execve: environment must be a mapping object");
@@ -4971,11 +4984,17 @@
     if (argvlist == NULL) {
         goto fail;
     }
+    if (!argvlist[0][0]) {
+        PyErr_SetString(PyExc_ValueError,
+            "execve: argv first element cannot be empty");
+        goto fail;
+    }
 
     envlist = parse_envlist(env, &envc);
     if (envlist == NULL)
         goto fail;
 
+    _Py_BEGIN_SUPPRESS_IPH
 #ifdef HAVE_FEXECVE
     if (path->fd > -1)
         fexecve(path->fd, argvlist, envlist);
@@ -4986,6 +5005,7 @@
 #else
         execve(path->narrow, argvlist, envlist);
 #endif
+    _Py_END_SUPPRESS_IPH
 
     /* If we get here it's definitely an error */
 
@@ -5061,6 +5081,13 @@
                 "spawnv() arg 2 must contain only strings");
             return NULL;
         }
+        if (i == 0 && !argvlist[0][0]) {
+            free_string_array(argvlist, i);
+            PyErr_SetString(
+                PyExc_ValueError,
+                "spawnv() arg 2 first element cannot be empty");
+            return NULL;
+        }
     }
     argvlist[argc] = NULL;
 
@@ -5155,6 +5182,13 @@
             lastarg = i;
             goto fail_1;
         }
+        if (i == 0 && !argvlist[0][0]) {
+            lastarg = i;
+            PyErr_SetString(
+                PyExc_ValueError,
+                "spawnv() arg 2 first element cannot be empty");
+            goto fail_1;
+        }
     }
     lastarg = argc;
     argvlist[argc] = NULL;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 19 22:17:57 2016
From: python-checkins at python.org (steve.dower)
Date: Sun, 20 Nov 2016 03:17:57 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NzMy?=
 =?utf-8?q?=3A_Raise_ValueError_when_argv=5B0=5D_is_empty=2E?=
Message-ID: <20161120031757.7167.21143.63EC5C7A@psf.io>

https://hg.python.org/cpython/rev/e076ace7b0ff
changeset:   105196:e076ace7b0ff
branch:      3.5
parent:      105191:02f416441def
user:        Steve Dower <steve.dower at microsoft.com>
date:        Sat Nov 19 19:03:54 2016 -0800
summary:
  Issue #28732: Raise ValueError when argv[0] is empty.

files:
  Modules/posixmodule.c |  9 +++++++++
  1 files changed, 9 insertions(+), 0 deletions(-)


diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -5210,6 +5210,15 @@
                 "spawnv() arg 2 must contain only strings");
             return NULL;
         }
+#ifdef MS_WINDOWS
+        if (i == 0 && !argvlist[0][0]) {
+            free_string_array(argvlist, i);
+            PyErr_SetString(
+                PyExc_ValueError,
+                "spawnv() arg 2 first element cannot be empty");
+            return NULL;
+        }
+#endif
     }
     argvlist[argc] = NULL;
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 19 22:17:57 2016
From: python-checkins at python.org (steve.dower)
Date: Sun, 20 Nov 2016 03:17:57 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328732=3A_Raise_ValueError_when_argv=5B0=5D_is_empty?=
Message-ID: <20161120031757.7964.38739.2EECD392@psf.io>

https://hg.python.org/cpython/rev/af78b33704af
changeset:   105197:af78b33704af
branch:      3.6
parent:      105194:1a9e4b465497
parent:      105196:e076ace7b0ff
user:        Steve Dower <steve.dower at microsoft.com>
date:        Sat Nov 19 19:17:26 2016 -0800
summary:
  Issue #28732: Raise ValueError when argv[0] is empty

files:
  Lib/test/test_os.py   |  18 +++++++++++++-
  Modules/posixmodule.c |  36 ++++++++++++++++++++++++++++++-
  2 files changed, 51 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
--- a/Lib/test/test_os.py
+++ b/Lib/test/test_os.py
@@ -1481,8 +1481,16 @@
         self.assertRaises(OSError, os.execvpe, 'no such app-',
                           ['no such app-'], None)
 
+    def test_execv_with_bad_arglist(self):
+        self.assertRaises(ValueError, os.execv, 'notepad', ())
+        self.assertRaises(ValueError, os.execv, 'notepad', [])
+        self.assertRaises(ValueError, os.execv, 'notepad', ('',))
+        self.assertRaises(ValueError, os.execv, 'notepad', [''])
+
     def test_execvpe_with_bad_arglist(self):
         self.assertRaises(ValueError, os.execvpe, 'notepad', [], None)
+        self.assertRaises(ValueError, os.execvpe, 'notepad', [], {})
+        self.assertRaises(ValueError, os.execvpe, 'notepad', [''], {})
 
     @unittest.skipUnless(hasattr(os, '_execvpe'),
                          "No internal os._execvpe function to test.")
@@ -2325,23 +2333,29 @@
     def test_spawnl_noargs(self):
         args = self.create_args()
         self.assertRaises(ValueError, os.spawnl, os.P_NOWAIT, args[0])
+        self.assertRaises(ValueError, os.spawnl, os.P_NOWAIT, args[0], '')
 
     @requires_os_func('spawnle')
-    def test_spawnl_noargs(self):
+    def test_spawnle_noargs(self):
         args = self.create_args()
         self.assertRaises(ValueError, os.spawnle, os.P_NOWAIT, args[0], {})
+        self.assertRaises(ValueError, os.spawnle, os.P_NOWAIT, args[0], '', {})
 
     @requires_os_func('spawnv')
     def test_spawnv_noargs(self):
         args = self.create_args()
         self.assertRaises(ValueError, os.spawnv, os.P_NOWAIT, args[0], ())
         self.assertRaises(ValueError, os.spawnv, os.P_NOWAIT, args[0], [])
+        self.assertRaises(ValueError, os.spawnv, os.P_NOWAIT, args[0], ('',))
+        self.assertRaises(ValueError, os.spawnv, os.P_NOWAIT, args[0], [''])
 
     @requires_os_func('spawnve')
-    def test_spawnv_noargs(self):
+    def test_spawnve_noargs(self):
         args = self.create_args()
         self.assertRaises(ValueError, os.spawnve, os.P_NOWAIT, args[0], (), {})
         self.assertRaises(ValueError, os.spawnve, os.P_NOWAIT, args[0], [], {})
+        self.assertRaises(ValueError, os.spawnve, os.P_NOWAIT, args[0], ('',), {})
+        self.assertRaises(ValueError, os.spawnve, os.P_NOWAIT, args[0], [''], {})
 
 # The introduction of this TestCase caused at least two different errors on
 # *nix buildbots. Temporarily skip this to let the buildbots move along.
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -4916,12 +4916,20 @@
     if (argvlist == NULL) {
         return NULL;
     }
-
+    if (!argvlist[0][0]) {
+        PyErr_SetString(PyExc_ValueError,
+            "execv() arg 2 first element cannot be empty");
+        free_string_array(argvlist, argc);
+        return NULL;
+    }
+
+    _Py_BEGIN_SUPPRESS_IPH
 #ifdef HAVE_WEXECV
     _wexecv(path->wide, argvlist);
 #else
     execv(path->narrow, argvlist);
 #endif
+    _Py_END_SUPPRESS_IPH
 
     /* If we get here it's definitely an error */
 
@@ -4961,6 +4969,11 @@
         goto fail;
     }
     argc = PySequence_Size(argv);
+    if (argc < 1) {
+        PyErr_SetString(PyExc_ValueError, "execve: argv must not be empty");
+        return NULL;
+    }
+
     if (!PyMapping_Check(env)) {
         PyErr_SetString(PyExc_TypeError,
                         "execve: environment must be a mapping object");
@@ -4971,11 +4984,17 @@
     if (argvlist == NULL) {
         goto fail;
     }
+    if (!argvlist[0][0]) {
+        PyErr_SetString(PyExc_ValueError,
+            "execve: argv first element cannot be empty");
+        goto fail;
+    }
 
     envlist = parse_envlist(env, &envc);
     if (envlist == NULL)
         goto fail;
 
+    _Py_BEGIN_SUPPRESS_IPH
 #ifdef HAVE_FEXECVE
     if (path->fd > -1)
         fexecve(path->fd, argvlist, envlist);
@@ -4986,6 +5005,7 @@
 #else
         execve(path->narrow, argvlist, envlist);
 #endif
+    _Py_END_SUPPRESS_IPH
 
     /* If we get here it's definitely an error */
 
@@ -5061,6 +5081,13 @@
                 "spawnv() arg 2 must contain only strings");
             return NULL;
         }
+        if (i == 0 && !argvlist[0][0]) {
+            free_string_array(argvlist, i);
+            PyErr_SetString(
+                PyExc_ValueError,
+                "spawnv() arg 2 first element cannot be empty");
+            return NULL;
+        }
     }
     argvlist[argc] = NULL;
 
@@ -5155,6 +5182,13 @@
             lastarg = i;
             goto fail_1;
         }
+        if (i == 0 && !argvlist[0][0]) {
+            lastarg = i;
+            PyErr_SetString(
+                PyExc_ValueError,
+                "spawnv() arg 2 first element cannot be empty");
+            goto fail_1;
+        }
     }
     lastarg = argc;
     argvlist[argc] = NULL;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 19 23:12:20 2016
From: python-checkins at python.org (steve.dower)
Date: Sun, 20 Nov 2016 04:12:20 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328732=3A_Adds_new_errors_to_spawnv_emulation_fo?=
 =?utf-8?q?r_platforms_that_only_have?=
Message-ID: <20161120041220.41115.93956.EB53B422@psf.io>

https://hg.python.org/cpython/rev/ac6de11fbd50
changeset:   105200:ac6de11fbd50
parent:      105198:fc6f757e53de
parent:      105199:2e1fb851dfb4
user:        Steve Dower <steve.dower at microsoft.com>
date:        Sat Nov 19 20:12:08 2016 -0800
summary:
  Issue #28732: Adds new errors to spawnv emulation for platforms that only have fork and execv

files:
  Lib/os.py |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Lib/os.py b/Lib/os.py
--- a/Lib/os.py
+++ b/Lib/os.py
@@ -832,6 +832,10 @@
 
     def _spawnvef(mode, file, args, env, func):
         # Internal helper; func is the exec*() function to use
+        if not isinstance(args, (tuple, list)):
+            raise TypeError('argv must be a tuple or a list')
+        if not args[0]:
+            raise ValueError('argv first element cannot be empty')
         pid = fork()
         if not pid:
             # Child

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 19 23:12:20 2016
From: python-checkins at python.org (steve.dower)
Date: Sun, 20 Nov 2016 04:12:20 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NzMy?=
 =?utf-8?q?=3A_Adds_new_errors_to_spawnv_emulation_for_platforms_that_only?=
 =?utf-8?q?_have?=
Message-ID: <20161120041219.7167.58568.360AA6B3@psf.io>

https://hg.python.org/cpython/rev/2e1fb851dfb4
changeset:   105199:2e1fb851dfb4
branch:      3.6
parent:      105197:af78b33704af
user:        Steve Dower <steve.dower at microsoft.com>
date:        Sat Nov 19 20:11:56 2016 -0800
summary:
  Issue #28732: Adds new errors to spawnv emulation for platforms that only have fork and execv

files:
  Lib/os.py |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Lib/os.py b/Lib/os.py
--- a/Lib/os.py
+++ b/Lib/os.py
@@ -832,6 +832,10 @@
 
     def _spawnvef(mode, file, args, env, func):
         # Internal helper; func is the exec*() function to use
+        if not isinstance(args, (tuple, list)):
+            raise TypeError('argv must be a tuple or a list')
+        if not args[0]:
+            raise ValueError('argv first element cannot be empty')
         pid = fork()
         if not pid:
             # Child

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 00:39:57 2016
From: python-checkins at python.org (steve.dower)
Date: Sun, 20 Nov 2016 05:39:57 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_from_3=2E6?=
Message-ID: <20161120051450.41421.43716.2154199A@psf.io>

https://hg.python.org/cpython/rev/08ca8d3ec707
changeset:   105202:08ca8d3ec707
parent:      105200:ac6de11fbd50
parent:      105201:1923556eb13f
user:        Steve Dower <steve.dower at microsoft.com>
date:        Sat Nov 19 21:14:37 2016 -0800
summary:
  Merge from 3.6

files:
  Lib/os.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/os.py b/Lib/os.py
--- a/Lib/os.py
+++ b/Lib/os.py
@@ -834,7 +834,7 @@
         # Internal helper; func is the exec*() function to use
         if not isinstance(args, (tuple, list)):
             raise TypeError('argv must be a tuple or a list')
-        if not args[0]:
+        if not args or not args[0]:
             raise ValueError('argv first element cannot be empty')
         pid = fork()
         if not pid:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 00:40:09 2016
From: python-checkins at python.org (steve.dower)
Date: Sun, 20 Nov 2016 05:40:09 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Fixes_empty_tu?=
 =?utf-8?q?ple_case=2E?=
Message-ID: <20161120051450.5241.60487.281ACFA6@psf.io>

https://hg.python.org/cpython/rev/1923556eb13f
changeset:   105201:1923556eb13f
branch:      3.6
parent:      105199:2e1fb851dfb4
user:        Steve Dower <steve.dower at microsoft.com>
date:        Sat Nov 19 21:14:27 2016 -0800
summary:
  Fixes empty tuple case.

files:
  Lib/os.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/os.py b/Lib/os.py
--- a/Lib/os.py
+++ b/Lib/os.py
@@ -834,7 +834,7 @@
         # Internal helper; func is the exec*() function to use
         if not isinstance(args, (tuple, list)):
             raise TypeError('argv must be a tuple or a list')
-        if not args[0]:
+        if not args or not args[0]:
             raise ValueError('argv first element cannot be empty')
         pid = fork()
         if not pid:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 01:25:29 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 06:25:29 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2327998=3A_Documented_bytes_paths_support_on_Wind?=
 =?utf-8?q?ows=2E?=
Message-ID: <20161120062529.45076.69534.54CEA123@psf.io>

https://hg.python.org/cpython/rev/26195e07fcc5
changeset:   105204:26195e07fcc5
parent:      105202:08ca8d3ec707
parent:      105203:ac63c70635db
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 08:25:07 2016 +0200
summary:
  Issue #27998: Documented bytes paths support on Windows.

files:
  Doc/library/os.rst   |  23 +++++++++++------------
  Doc/whatsnew/3.6.rst |   2 ++
  2 files changed, 13 insertions(+), 12 deletions(-)


diff --git a/Doc/library/os.rst b/Doc/library/os.rst
--- a/Doc/library/os.rst
+++ b/Doc/library/os.rst
@@ -2016,13 +2016,11 @@
    always requires a system call on Unix but only requires one for
    symbolic links on Windows.
 
-   On Unix, *path* can be of type :class:`str` or :class:`bytes` (either
-   directly or indirectly through the :class:`PathLike` interface; use
-   :func:`~os.fsencode` and :func:`~os.fsdecode` to encode and decode
-   :class:`bytes` paths). On Windows, *path* must be of type :class:`str`.
-   On both systems, the type of the :attr:`~os.DirEntry.name` and
-   :attr:`~os.DirEntry.path` attributes of each :class:`os.DirEntry` will be of
-   the same type as *path*.
+   *path* may be a :term:`path-like object`.  If *path* is of type ``bytes``
+   (directly or indirectly through the :class:`PathLike` interface),
+   the type of the :attr:`~os.DirEntry.name` and :attr:`~os.DirEntry.path`
+   attributes of each :class:`os.DirEntry` will be ``bytes``; in all other
+   circumstances, they will be of type ``str``.
 
    The :func:`scandir` iterator supports the :term:`context manager` protocol
    and has the following method:
@@ -2100,8 +2098,8 @@
       The entry's base filename, relative to the :func:`scandir` *path*
       argument.
 
-      The :attr:`name` attribute will be of the same type (``str`` or
-      ``bytes``) as the :func:`scandir` *path* argument. Use
+      The :attr:`name` attribute will be ``bytes`` if the :func:`scandir`
+      *path* argument is of type ``bytes`` and ``str`` otherwise.  Use
       :func:`~os.fsdecode` to decode byte filenames.
 
    .. attribute:: path
@@ -2111,8 +2109,8 @@
       argument.  The path is only absolute if the :func:`scandir` *path*
       argument was absolute.
 
-      The :attr:`path` attribute will be of the same type (``str`` or
-      ``bytes``) as the :func:`scandir` *path* argument. Use
+      The :attr:`path` attribute will be ``bytes`` if the :func:`scandir`
+      *path* argument is of type ``bytes`` and ``str`` otherwise.  Use
       :func:`~os.fsdecode` to decode byte filenames.
 
    .. method:: inode()
@@ -2207,7 +2205,8 @@
    .. versionadded:: 3.5
 
    .. versionchanged:: 3.6
-      Added support for the :class:`~os.PathLike` interface.
+      Added support for the :class:`~os.PathLike` interface.  Added support
+      for :class:`bytes` paths on Windows.
 
 
 .. function:: stat(path, \*, dir_fd=None, follow_symlinks=True)
diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -1196,6 +1196,8 @@
 :mod:`os` and :mod:`os.path` modules now support
 :term:`path-like objects <path-like object>`.
 
+:func:`~os.scandir` now supports :class:`bytes` paths on Windows.
+
 A new :meth:`~os.scandir.close` method allows explicitly closing a
 :func:`~os.scandir` iterator.  The :func:`~os.scandir` iterator now
 supports the :term:`context manager` protocol.  If a :func:`scandir`

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 01:25:29 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 06:25:29 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI3OTk4?=
 =?utf-8?q?=3A_Documented_bytes_paths_support_on_Windows=2E?=
Message-ID: <20161120062529.46126.56741.5DC7932B@psf.io>

https://hg.python.org/cpython/rev/ac63c70635db
changeset:   105203:ac63c70635db
branch:      3.6
parent:      105201:1923556eb13f
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 08:23:07 2016 +0200
summary:
  Issue #27998: Documented bytes paths support on Windows.

files:
  Doc/library/os.rst   |  23 +++++++++++------------
  Doc/whatsnew/3.6.rst |   2 ++
  2 files changed, 13 insertions(+), 12 deletions(-)


diff --git a/Doc/library/os.rst b/Doc/library/os.rst
--- a/Doc/library/os.rst
+++ b/Doc/library/os.rst
@@ -2016,13 +2016,11 @@
    always requires a system call on Unix but only requires one for
    symbolic links on Windows.
 
-   On Unix, *path* can be of type :class:`str` or :class:`bytes` (either
-   directly or indirectly through the :class:`PathLike` interface; use
-   :func:`~os.fsencode` and :func:`~os.fsdecode` to encode and decode
-   :class:`bytes` paths). On Windows, *path* must be of type :class:`str`.
-   On both systems, the type of the :attr:`~os.DirEntry.name` and
-   :attr:`~os.DirEntry.path` attributes of each :class:`os.DirEntry` will be of
-   the same type as *path*.
+   *path* may be a :term:`path-like object`.  If *path* is of type ``bytes``
+   (directly or indirectly through the :class:`PathLike` interface),
+   the type of the :attr:`~os.DirEntry.name` and :attr:`~os.DirEntry.path`
+   attributes of each :class:`os.DirEntry` will be ``bytes``; in all other
+   circumstances, they will be of type ``str``.
 
    The :func:`scandir` iterator supports the :term:`context manager` protocol
    and has the following method:
@@ -2100,8 +2098,8 @@
       The entry's base filename, relative to the :func:`scandir` *path*
       argument.
 
-      The :attr:`name` attribute will be of the same type (``str`` or
-      ``bytes``) as the :func:`scandir` *path* argument. Use
+      The :attr:`name` attribute will be ``bytes`` if the :func:`scandir`
+      *path* argument is of type ``bytes`` and ``str`` otherwise.  Use
       :func:`~os.fsdecode` to decode byte filenames.
 
    .. attribute:: path
@@ -2111,8 +2109,8 @@
       argument.  The path is only absolute if the :func:`scandir` *path*
       argument was absolute.
 
-      The :attr:`path` attribute will be of the same type (``str`` or
-      ``bytes``) as the :func:`scandir` *path* argument. Use
+      The :attr:`path` attribute will be ``bytes`` if the :func:`scandir`
+      *path* argument is of type ``bytes`` and ``str`` otherwise.  Use
       :func:`~os.fsdecode` to decode byte filenames.
 
    .. method:: inode()
@@ -2207,7 +2205,8 @@
    .. versionadded:: 3.5
 
    .. versionchanged:: 3.6
-      Added support for the :class:`~os.PathLike` interface.
+      Added support for the :class:`~os.PathLike` interface.  Added support
+      for :class:`bytes` paths on Windows.
 
 
 .. function:: stat(path, \*, dir_fd=None, follow_symlinks=True)
diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -1196,6 +1196,8 @@
 :mod:`os` and :mod:`os.path` modules now support
 :term:`path-like objects <path-like object>`.
 
+:func:`~os.scandir` now supports :class:`bytes` paths on Windows.
+
 A new :meth:`~os.scandir.close` method allows explicitly closing a
 :func:`~os.scandir` iterator.  The :func:`~os.scandir` iterator now
 supports the :term:`context manager` protocol.  If a :func:`scandir`

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 01:48:46 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 06:48:46 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NzE1?=
 =?utf-8?q?=3A_Added_error_checks_for_PyUnicode=5FAsUTF8=28=29=2E?=
Message-ID: <20161120064846.9962.45187.A9647C81@psf.io>

https://hg.python.org/cpython/rev/38f321a6be41
changeset:   105205:38f321a6be41
branch:      3.5
parent:      105196:e076ace7b0ff
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 08:47:21 2016 +0200
summary:
  Issue #28715: Added error checks for PyUnicode_AsUTF8().

files:
  Modules/_ctypes/_ctypes.c  |  5 ++---
  Modules/_ctypes/callproc.c |  4 +++-
  Modules/ossaudiodev.c      |  9 ++++++---
  Python/ast.c               |  8 +++++---
  Python/importdl.c          |  4 ++++
  5 files changed, 20 insertions(+), 10 deletions(-)


diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c
--- a/Modules/_ctypes/_ctypes.c
+++ b/Modules/_ctypes/_ctypes.c
@@ -734,8 +734,7 @@
         return -1;
 
     if (value && PyUnicode_Check(key) &&
-        /* XXX struni _PyUnicode_AsString can fail (also in other places)! */
-        0 == strcmp(_PyUnicode_AsString(key), "_fields_"))
+        _PyUnicode_EqualToASCIIString(key, "_fields_"))
         return PyCStructUnionType_update_stgdict(self, value, 1);
     return 0;
 }
@@ -749,7 +748,7 @@
         return -1;
 
     if (PyUnicode_Check(key) &&
-        0 == strcmp(_PyUnicode_AsString(key), "_fields_"))
+        _PyUnicode_EqualToASCIIString(key, "_fields_"))
         return PyCStructUnionType_update_stgdict(self, value, 0);
     return 0;
 }
diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c
--- a/Modules/_ctypes/callproc.c
+++ b/Modules/_ctypes/callproc.c
@@ -1670,7 +1670,9 @@
         return result;
     }
     if (PyUnicode_CheckExact(cls)) {
-        char *name = _PyUnicode_AsString(cls);
+        const char *name = PyUnicode_AsUTF8(cls);
+        if (name == NULL)
+            return NULL;
         buf = PyMem_Malloc(strlen(name) + 3 + 1);
         if (buf == NULL)
             return PyErr_NoMemory();
diff --git a/Modules/ossaudiodev.c b/Modules/ossaudiodev.c
--- a/Modules/ossaudiodev.c
+++ b/Modules/ossaudiodev.c
@@ -925,11 +925,14 @@
 static PyObject *
 oss_getattro(oss_audio_t *self, PyObject *nameobj)
 {
-    char *name = "";
+    const char *name = "";
     PyObject * rval = NULL;
 
-    if (PyUnicode_Check(nameobj))
-        name = _PyUnicode_AsString(nameobj);
+    if (PyUnicode_Check(nameobj)) {
+        name = PyUnicode_AsUTF8(nameobj);
+        if (name == NULL)
+            return NULL;
+    }
 
     if (strcmp(name, "closed") == 0) {
         rval = (self->fd == -1) ? Py_True : Py_False;
diff --git a/Python/ast.c b/Python/ast.c
--- a/Python/ast.c
+++ b/Python/ast.c
@@ -2024,16 +2024,18 @@
                 errtype = "value error";
             if (errtype) {
                 char buf[128];
+                const char *s = NULL;
                 PyObject *type, *value, *tback, *errstr;
                 PyErr_Fetch(&type, &value, &tback);
                 errstr = PyObject_Str(value);
-                if (errstr) {
-                    char *s = _PyUnicode_AsString(errstr);
+                if (errstr)
+                    s = PyUnicode_AsUTF8(errstr);
+                if (s) {
                     PyOS_snprintf(buf, sizeof(buf), "(%s) %s", errtype, s);
-                    Py_DECREF(errstr);
                 } else {
                     PyOS_snprintf(buf, sizeof(buf), "(%s) unknown error", errtype);
                 }
+                Py_XDECREF(errstr);
                 ast_error(c, n, buf);
                 Py_DECREF(type);
                 Py_XDECREF(value);
diff --git a/Python/importdl.c b/Python/importdl.c
--- a/Python/importdl.c
+++ b/Python/importdl.c
@@ -147,6 +147,10 @@
     /* Package context is needed for single-phase init */
     oldcontext = _Py_PackageContext;
     _Py_PackageContext = PyUnicode_AsUTF8(name_unicode);
+    if (_Py_PackageContext == NULL) {
+        _Py_PackageContext = oldcontext;
+        goto error;
+    }
     m = p0();
     _Py_PackageContext = oldcontext;
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 01:48:47 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 06:48:47 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328715=3A_Added_error_checks_for_PyUnicode=5FAsUTF8=28?=
 =?utf-8?b?KS4=?=
Message-ID: <20161120064846.8083.57704.295B93D0@psf.io>

https://hg.python.org/cpython/rev/0bb8ab158042
changeset:   105206:0bb8ab158042
branch:      3.6
parent:      105203:ac63c70635db
parent:      105205:38f321a6be41
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 08:48:07 2016 +0200
summary:
  Issue #28715: Added error checks for PyUnicode_AsUTF8().

files:
  Modules/_ctypes/_ctypes.c  |  5 ++---
  Modules/_ctypes/callproc.c |  4 +++-
  Modules/ossaudiodev.c      |  9 ++++++---
  Python/ast.c               |  8 +++++---
  Python/importdl.c          |  4 ++++
  5 files changed, 20 insertions(+), 10 deletions(-)


diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c
--- a/Modules/_ctypes/_ctypes.c
+++ b/Modules/_ctypes/_ctypes.c
@@ -728,8 +728,7 @@
         return -1;
 
     if (value && PyUnicode_Check(key) &&
-        /* XXX struni _PyUnicode_AsString can fail (also in other places)! */
-        0 == strcmp(_PyUnicode_AsString(key), "_fields_"))
+        _PyUnicode_EqualToASCIIString(key, "_fields_"))
         return PyCStructUnionType_update_stgdict(self, value, 1);
     return 0;
 }
@@ -743,7 +742,7 @@
         return -1;
 
     if (PyUnicode_Check(key) &&
-        0 == strcmp(_PyUnicode_AsString(key), "_fields_"))
+        _PyUnicode_EqualToASCIIString(key, "_fields_"))
         return PyCStructUnionType_update_stgdict(self, value, 0);
     return 0;
 }
diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c
--- a/Modules/_ctypes/callproc.c
+++ b/Modules/_ctypes/callproc.c
@@ -1666,7 +1666,9 @@
         return result;
     }
     if (PyUnicode_CheckExact(cls)) {
-        char *name = _PyUnicode_AsString(cls);
+        const char *name = PyUnicode_AsUTF8(cls);
+        if (name == NULL)
+            return NULL;
         buf = PyMem_Malloc(strlen(name) + 3 + 1);
         if (buf == NULL)
             return PyErr_NoMemory();
diff --git a/Modules/ossaudiodev.c b/Modules/ossaudiodev.c
--- a/Modules/ossaudiodev.c
+++ b/Modules/ossaudiodev.c
@@ -929,11 +929,14 @@
 static PyObject *
 oss_getattro(oss_audio_t *self, PyObject *nameobj)
 {
-    char *name = "";
+    const char *name = "";
     PyObject * rval = NULL;
 
-    if (PyUnicode_Check(nameobj))
-        name = _PyUnicode_AsString(nameobj);
+    if (PyUnicode_Check(nameobj)) {
+        name = PyUnicode_AsUTF8(nameobj);
+        if (name == NULL)
+            return NULL;
+    }
 
     if (strcmp(name, "closed") == 0) {
         rval = (self->fd == -1) ? Py_True : Py_False;
diff --git a/Python/ast.c b/Python/ast.c
--- a/Python/ast.c
+++ b/Python/ast.c
@@ -2118,17 +2118,19 @@
                 errtype = "value error";
             if (errtype) {
                 char buf[128];
+                const char *s = NULL;
                 PyObject *type, *value, *tback, *errstr;
                 PyErr_Fetch(&type, &value, &tback);
                 errstr = PyObject_Str(value);
-                if (errstr) {
-                    char *s = _PyUnicode_AsString(errstr);
+                if (errstr)
+                    s = PyUnicode_AsUTF8(errstr);
+                if (s) {
                     PyOS_snprintf(buf, sizeof(buf), "(%s) %s", errtype, s);
-                    Py_DECREF(errstr);
                 } else {
                     PyErr_Clear();
                     PyOS_snprintf(buf, sizeof(buf), "(%s) unknown error", errtype);
                 }
+                Py_XDECREF(errstr);
                 ast_error(c, n, buf);
                 Py_DECREF(type);
                 Py_XDECREF(value);
diff --git a/Python/importdl.c b/Python/importdl.c
--- a/Python/importdl.c
+++ b/Python/importdl.c
@@ -147,6 +147,10 @@
     /* Package context is needed for single-phase init */
     oldcontext = _Py_PackageContext;
     _Py_PackageContext = PyUnicode_AsUTF8(name_unicode);
+    if (_Py_PackageContext == NULL) {
+        _Py_PackageContext = oldcontext;
+        goto error;
+    }
     m = p0();
     _Py_PackageContext = oldcontext;
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 01:48:47 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 06:48:47 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328715=3A_Added_error_checks_for_PyUnicode=5FAsU?=
 =?utf-8?b?VEY4KCku?=
Message-ID: <20161120064847.8131.20892.2956B1E8@psf.io>

https://hg.python.org/cpython/rev/9cd42ed64bdb
changeset:   105207:9cd42ed64bdb
parent:      105204:26195e07fcc5
parent:      105206:0bb8ab158042
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 08:48:30 2016 +0200
summary:
  Issue #28715: Added error checks for PyUnicode_AsUTF8().

files:
  Modules/_ctypes/_ctypes.c  |  5 ++---
  Modules/_ctypes/callproc.c |  4 +++-
  Modules/ossaudiodev.c      |  9 ++++++---
  Python/ast.c               |  8 +++++---
  Python/importdl.c          |  4 ++++
  5 files changed, 20 insertions(+), 10 deletions(-)


diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c
--- a/Modules/_ctypes/_ctypes.c
+++ b/Modules/_ctypes/_ctypes.c
@@ -728,8 +728,7 @@
         return -1;
 
     if (value && PyUnicode_Check(key) &&
-        /* XXX struni _PyUnicode_AsString can fail (also in other places)! */
-        0 == strcmp(_PyUnicode_AsString(key), "_fields_"))
+        _PyUnicode_EqualToASCIIString(key, "_fields_"))
         return PyCStructUnionType_update_stgdict(self, value, 1);
     return 0;
 }
@@ -743,7 +742,7 @@
         return -1;
 
     if (PyUnicode_Check(key) &&
-        0 == strcmp(_PyUnicode_AsString(key), "_fields_"))
+        _PyUnicode_EqualToASCIIString(key, "_fields_"))
         return PyCStructUnionType_update_stgdict(self, value, 0);
     return 0;
 }
diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c
--- a/Modules/_ctypes/callproc.c
+++ b/Modules/_ctypes/callproc.c
@@ -1666,7 +1666,9 @@
         return result;
     }
     if (PyUnicode_CheckExact(cls)) {
-        char *name = _PyUnicode_AsString(cls);
+        const char *name = PyUnicode_AsUTF8(cls);
+        if (name == NULL)
+            return NULL;
         buf = PyMem_Malloc(strlen(name) + 3 + 1);
         if (buf == NULL)
             return PyErr_NoMemory();
diff --git a/Modules/ossaudiodev.c b/Modules/ossaudiodev.c
--- a/Modules/ossaudiodev.c
+++ b/Modules/ossaudiodev.c
@@ -929,11 +929,14 @@
 static PyObject *
 oss_getattro(oss_audio_t *self, PyObject *nameobj)
 {
-    char *name = "";
+    const char *name = "";
     PyObject * rval = NULL;
 
-    if (PyUnicode_Check(nameobj))
-        name = _PyUnicode_AsString(nameobj);
+    if (PyUnicode_Check(nameobj)) {
+        name = PyUnicode_AsUTF8(nameobj);
+        if (name == NULL)
+            return NULL;
+    }
 
     if (strcmp(name, "closed") == 0) {
         rval = (self->fd == -1) ? Py_True : Py_False;
diff --git a/Python/ast.c b/Python/ast.c
--- a/Python/ast.c
+++ b/Python/ast.c
@@ -2118,17 +2118,19 @@
                 errtype = "value error";
             if (errtype) {
                 char buf[128];
+                const char *s = NULL;
                 PyObject *type, *value, *tback, *errstr;
                 PyErr_Fetch(&type, &value, &tback);
                 errstr = PyObject_Str(value);
-                if (errstr) {
-                    char *s = _PyUnicode_AsString(errstr);
+                if (errstr)
+                    s = PyUnicode_AsUTF8(errstr);
+                if (s) {
                     PyOS_snprintf(buf, sizeof(buf), "(%s) %s", errtype, s);
-                    Py_DECREF(errstr);
                 } else {
                     PyErr_Clear();
                     PyOS_snprintf(buf, sizeof(buf), "(%s) unknown error", errtype);
                 }
+                Py_XDECREF(errstr);
                 ast_error(c, n, buf);
                 Py_DECREF(type);
                 Py_XDECREF(value);
diff --git a/Python/importdl.c b/Python/importdl.c
--- a/Python/importdl.c
+++ b/Python/importdl.c
@@ -147,6 +147,10 @@
     /* Package context is needed for single-phase init */
     oldcontext = _Py_PackageContext;
     _Py_PackageContext = PyUnicode_AsUTF8(name_unicode);
+    if (_Py_PackageContext == NULL) {
+        _Py_PackageContext = oldcontext;
+        goto error;
+    }
     m = p0();
     _Py_PackageContext = oldcontext;
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 03:17:22 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 08:17:22 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Replaced_outda?=
 =?utf-8?q?ted_macros_=5FPyUnicode=5FAsString_and_=5FPyUnicode=5FAsStringA?=
 =?utf-8?q?ndSize?=
Message-ID: <20161120081722.7770.77672.B6F5AA8C@psf.io>

https://hg.python.org/cpython/rev/39f0e0cf49fd
changeset:   105208:39f0e0cf49fd
branch:      3.6
parent:      105206:0bb8ab158042
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 09:13:07 2016 +0200
summary:
  Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.

files:
  Modules/_ctypes/_ctypes.c          |  12 ++++++------
  Modules/_ctypes/stgdict.c          |   2 +-
  Modules/_datetimemodule.c          |   4 ++--
  Modules/_io/stringio.c             |   2 +-
  Modules/_io/textio.c               |   4 ++--
  Modules/_pickle.c                  |   2 +-
  Modules/_sqlite/connection.c       |   4 ++--
  Modules/_sqlite/cursor.c           |   4 ++--
  Modules/_sqlite/row.c              |   4 ++--
  Modules/_sqlite/statement.c        |   6 +++---
  Modules/cjkcodecs/cjkcodecs.h      |   2 +-
  Modules/cjkcodecs/multibytecodec.c |   4 ++--
  Modules/parsermodule.c             |   4 ++--
  Modules/posixmodule.c              |   2 +-
  Modules/socketmodule.c             |   2 +-
  Modules/syslogmodule.c             |   4 ++--
  Modules/timemodule.c               |   2 +-
  Objects/floatobject.c              |   4 ++--
  Objects/moduleobject.c             |   8 ++++----
  Objects/object.c                   |   4 ++--
  Objects/setobject.c                |   2 +-
  Objects/structseq.c                |   2 +-
  Objects/typeobject.c               |   6 +++---
  Parser/tokenizer.c                 |   2 +-
  Python/bltinmodule.c               |   8 ++++----
  Python/ceval.c                     |   4 ++--
  Python/future.c                    |   2 +-
  Python/pylifecycle.c               |   4 ++--
  Python/pythonrun.c                 |   8 ++++----
  Python/structmember.c              |   2 +-
  Python/sysmodule.c                 |   4 ++--
  31 files changed, 62 insertions(+), 62 deletions(-)


diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c
--- a/Modules/_ctypes/_ctypes.c
+++ b/Modules/_ctypes/_ctypes.c
@@ -1723,7 +1723,7 @@
     if (stgd && CDataObject_Check(value) && stgd->proto && PyUnicode_Check(stgd->proto)) {
         PyCArgObject *parg;
 
-        switch (_PyUnicode_AsString(stgd->proto)[0]) {
+        switch (PyUnicode_AsUTF8(stgd->proto)[0]) {
         case 'z': /* c_char_p */
         case 'Z': /* c_wchar_p */
             parg = PyCArgObject_new();
@@ -1835,7 +1835,7 @@
 
     dict = PyObject_stgdict((PyObject *)self);
     assert(dict); /* Cannot be NULL for CDataObject instances */
-    fmt = _PyUnicode_AsString(dict->proto);
+    fmt = PyUnicode_AsUTF8(dict->proto);
     assert(fmt);
 
     fd = _ctypes_get_fielddesc(fmt);
@@ -2059,7 +2059,7 @@
     assert(dict);
 
     /* I think we can rely on this being a one-character string */
-    fmt = _PyUnicode_AsString(dict->proto);
+    fmt = PyUnicode_AsUTF8(dict->proto);
     assert(fmt);
 
     fd = _ctypes_get_fielddesc(fmt);
@@ -3128,7 +3128,7 @@
         /* simple pointer types, c_void_p, c_wchar_p, BSTR, ... */
         && PyUnicode_Check(dict->proto)
 /* We only allow c_void_p, c_char_p and c_wchar_p as a simple output parameter type */
-        && (strchr("PzZ", _PyUnicode_AsString(dict->proto)[0]))) {
+        && (strchr("PzZ", PyUnicode_AsUTF8(dict->proto)[0]))) {
         return 1;
     }
 
@@ -3218,7 +3218,7 @@
         return *pname ? 1 : 0;
     }
     if (PyUnicode_Check(obj)) {
-        *pname = _PyUnicode_AsString(obj);
+        *pname = PyUnicode_AsUTF8(obj);
         return *pname ? 1 : 0;
     }
     PyErr_SetString(PyExc_TypeError,
@@ -5233,7 +5233,7 @@
     dict = PyType_stgdict(arg);
     if (dict) {
         if (PyUnicode_Check(dict->proto)
-            && (strchr("sPzUZXO", _PyUnicode_AsString(dict->proto)[0]))) {
+            && (strchr("sPzUZXO", PyUnicode_AsUTF8(dict->proto)[0]))) {
             /* simple pointer types, c_void_p, c_wchar_p, BSTR, ... */
             return 1;
         }
diff --git a/Modules/_ctypes/stgdict.c b/Modules/_ctypes/stgdict.c
--- a/Modules/_ctypes/stgdict.c
+++ b/Modules/_ctypes/stgdict.c
@@ -489,7 +489,7 @@
 
         if (isStruct && !isPacked) {
             char *fieldfmt = dict->format ? dict->format : "B";
-            char *fieldname = _PyUnicode_AsString(name);
+            char *fieldname = PyUnicode_AsUTF8(name);
             char *ptr;
             Py_ssize_t len;
             char *buf;
diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c
--- a/Modules/_datetimemodule.c
+++ b/Modules/_datetimemodule.c
@@ -1219,7 +1219,7 @@
     assert(object && format && timetuple);
     assert(PyUnicode_Check(format));
     /* Convert the input format to a C string and size */
-    pin = _PyUnicode_AsStringAndSize(format, &flen);
+    pin = PyUnicode_AsUTF8AndSize(format, &flen);
     if (!pin)
         return NULL;
 
@@ -1287,7 +1287,7 @@
             }
             assert(Zreplacement != NULL);
             assert(PyUnicode_Check(Zreplacement));
-            ptoappend = _PyUnicode_AsStringAndSize(Zreplacement,
+            ptoappend = PyUnicode_AsUTF8AndSize(Zreplacement,
                                                   &ntoappend);
             if (ptoappend == NULL)
                 goto Done;
diff --git a/Modules/_io/stringio.c b/Modules/_io/stringio.c
--- a/Modules/_io/stringio.c
+++ b/Modules/_io/stringio.c
@@ -708,7 +708,7 @@
                          Py_TYPE(newline_obj)->tp_name);
             return -1;
         }
-        newline = _PyUnicode_AsString(newline_obj);
+        newline = PyUnicode_AsUTF8(newline_obj);
         if (newline == NULL)
             return -1;
     }
diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c
--- a/Modules/_io/textio.c
+++ b/Modules/_io/textio.c
@@ -921,7 +921,7 @@
             Py_CLEAR(self->encoding);
     }
     if (self->encoding != NULL) {
-        encoding = _PyUnicode_AsString(self->encoding);
+        encoding = PyUnicode_AsUTF8(self->encoding);
         if (encoding == NULL)
             goto error;
     }
@@ -964,7 +964,7 @@
     }
     self->writetranslate = (newline == NULL || newline[0] != '\0');
     if (!self->readuniversal && self->readnl) {
-        self->writenl = _PyUnicode_AsString(self->readnl);
+        self->writenl = PyUnicode_AsUTF8(self->readnl);
         if (self->writenl == NULL)
             goto error;
         if (!strcmp(self->writenl, "\n"))
diff --git a/Modules/_pickle.c b/Modules/_pickle.c
--- a/Modules/_pickle.c
+++ b/Modules/_pickle.c
@@ -1951,7 +1951,7 @@
         if (repr == NULL)
             goto error;
 
-        string = _PyUnicode_AsStringAndSize(repr, &size);
+        string = PyUnicode_AsUTF8AndSize(repr, &size);
         if (string == NULL)
             goto error;
 
diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c
--- a/Modules/_sqlite/connection.c
+++ b/Modules/_sqlite/connection.c
@@ -505,7 +505,7 @@
     } else if (PyFloat_Check(py_val)) {
         sqlite3_result_double(context, PyFloat_AsDouble(py_val));
     } else if (PyUnicode_Check(py_val)) {
-        const char *str = _PyUnicode_AsString(py_val);
+        const char *str = PyUnicode_AsUTF8(py_val);
         if (str == NULL)
             return -1;
         sqlite3_result_text(context, str, -1, SQLITE_TRANSIENT);
@@ -1527,7 +1527,7 @@
         }
     }
 
-    uppercase_name_str = _PyUnicode_AsString(uppercase_name);
+    uppercase_name_str = PyUnicode_AsUTF8(uppercase_name);
     if (!uppercase_name_str)
         goto finally;
 
diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c
--- a/Modules/_sqlite/cursor.c
+++ b/Modules/_sqlite/cursor.c
@@ -465,7 +465,7 @@
         pysqlite_statement_reset(self->statement);
     }
 
-    operation_cstr = _PyUnicode_AsStringAndSize(operation, &operation_len);
+    operation_cstr = PyUnicode_AsUTF8AndSize(operation, &operation_len);
     if (operation_cstr == NULL)
         goto error;
 
@@ -689,7 +689,7 @@
     self->reset = 0;
 
     if (PyUnicode_Check(script_obj)) {
-        script_cstr = _PyUnicode_AsString(script_obj);
+        script_cstr = PyUnicode_AsUTF8(script_obj);
         if (!script_cstr) {
             return NULL;
         }
diff --git a/Modules/_sqlite/row.c b/Modules/_sqlite/row.c
--- a/Modules/_sqlite/row.c
+++ b/Modules/_sqlite/row.c
@@ -98,7 +98,7 @@
         Py_XINCREF(item);
         return item;
     } else if (PyUnicode_Check(idx)) {
-        key = _PyUnicode_AsString(idx);
+        key = PyUnicode_AsUTF8(idx);
         if (key == NULL)
             return NULL;
 
@@ -108,7 +108,7 @@
             PyObject *obj;
             obj = PyTuple_GET_ITEM(self->description, i);
             obj = PyTuple_GET_ITEM(obj, 0);
-            compare_key = _PyUnicode_AsString(obj);
+            compare_key = PyUnicode_AsUTF8(obj);
             if (!compare_key) {
                 return NULL;
             }
diff --git a/Modules/_sqlite/statement.c b/Modules/_sqlite/statement.c
--- a/Modules/_sqlite/statement.c
+++ b/Modules/_sqlite/statement.c
@@ -59,7 +59,7 @@
     self->st = NULL;
     self->in_use = 0;
 
-    sql_cstr = _PyUnicode_AsStringAndSize(sql, &sql_cstr_len);
+    sql_cstr = PyUnicode_AsUTF8AndSize(sql, &sql_cstr_len);
     if (sql_cstr == NULL) {
         rc = PYSQLITE_SQL_WRONG_TYPE;
         return rc;
@@ -152,7 +152,7 @@
             rc = sqlite3_bind_double(self->st, pos, PyFloat_AsDouble(parameter));
             break;
         case TYPE_UNICODE:
-            string = _PyUnicode_AsStringAndSize(parameter, &buflen);
+            string = PyUnicode_AsUTF8AndSize(parameter, &buflen);
             if (string == NULL)
                 return -1;
             if (buflen > INT_MAX) {
@@ -325,7 +325,7 @@
     Py_ssize_t sql_len;
     sqlite3_stmt* new_st;
 
-    sql_cstr = _PyUnicode_AsStringAndSize(self->sql, &sql_len);
+    sql_cstr = PyUnicode_AsUTF8AndSize(self->sql, &sql_len);
     if (sql_cstr == NULL) {
         rc = PYSQLITE_SQL_WRONG_TYPE;
         return rc;
diff --git a/Modules/cjkcodecs/cjkcodecs.h b/Modules/cjkcodecs/cjkcodecs.h
--- a/Modules/cjkcodecs/cjkcodecs.h
+++ b/Modules/cjkcodecs/cjkcodecs.h
@@ -267,7 +267,7 @@
                         "encoding name must be a string.");
         return NULL;
     }
-    enc = _PyUnicode_AsString(encoding);
+    enc = PyUnicode_AsUTF8(encoding);
     if (enc == NULL)
         return NULL;
 
diff --git a/Modules/cjkcodecs/multibytecodec.c b/Modules/cjkcodecs/multibytecodec.c
--- a/Modules/cjkcodecs/multibytecodec.c
+++ b/Modules/cjkcodecs/multibytecodec.c
@@ -85,7 +85,7 @@
     const char *str;
 
     assert(PyUnicode_Check(errors));
-    str = _PyUnicode_AsString(errors);
+    str = PyUnicode_AsUTF8(errors);
     if (str == NULL)
         return NULL;
     cb = PyCodec_LookupError(str);
@@ -138,7 +138,7 @@
         return -1;
     }
 
-    str = _PyUnicode_AsString(value);
+    str = PyUnicode_AsUTF8(value);
     if (str == NULL)
         return -1;
 
diff --git a/Modules/parsermodule.c b/Modules/parsermodule.c
--- a/Modules/parsermodule.c
+++ b/Modules/parsermodule.c
@@ -912,7 +912,7 @@
                     Py_DECREF(o);
                 }
             }
-            temp_str = _PyUnicode_AsStringAndSize(temp, &len);
+            temp_str = PyUnicode_AsUTF8AndSize(temp, &len);
             if (temp_str == NULL) {
                 Py_DECREF(temp);
                 Py_XDECREF(elem);
@@ -1013,7 +1013,7 @@
             if (res && encoding) {
                 Py_ssize_t len;
                 const char *temp;
-                temp = _PyUnicode_AsStringAndSize(encoding, &len);
+                temp = PyUnicode_AsUTF8AndSize(encoding, &len);
                 if (temp == NULL) {
                     Py_DECREF(res);
                     Py_DECREF(encoding);
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -9325,7 +9325,7 @@
                 "configuration names must be strings or integers");
             return 0;
         }
-        confname = _PyUnicode_AsString(arg);
+        confname = PyUnicode_AsUTF8(arg);
         if (confname == NULL)
             return 0;
         while (lo < hi) {
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -5989,7 +5989,7 @@
         PyOS_snprintf(pbuf, sizeof(pbuf), "%ld", value);
         pptr = pbuf;
     } else if (PyUnicode_Check(pobj)) {
-        pptr = _PyUnicode_AsString(pobj);
+        pptr = PyUnicode_AsUTF8(pobj);
         if (pptr == NULL)
             goto err;
     } else if (PyBytes_Check(pobj)) {
diff --git a/Modules/syslogmodule.c b/Modules/syslogmodule.c
--- a/Modules/syslogmodule.c
+++ b/Modules/syslogmodule.c
@@ -139,7 +139,7 @@
      * If NULL, just let openlog figure it out (probably using C argv[0]).
      */
     if (S_ident_o) {
-        ident = _PyUnicode_AsString(S_ident_o);
+        ident = PyUnicode_AsUTF8(S_ident_o);
         if (ident == NULL)
             return NULL;
     }
@@ -167,7 +167,7 @@
             return NULL;
     }
 
-    message = _PyUnicode_AsString(message_object);
+    message = PyUnicode_AsUTF8(message_object);
     if (message == NULL)
         return NULL;
 
diff --git a/Modules/timemodule.c b/Modules/timemodule.c
--- a/Modules/timemodule.c
+++ b/Modules/timemodule.c
@@ -441,7 +441,7 @@
     if (Py_TYPE(args) == &StructTimeType) {
         PyObject *item;
         item = PyTuple_GET_ITEM(args, 9);
-        p->tm_zone = item == Py_None ? NULL : _PyUnicode_AsString(item);
+        p->tm_zone = item == Py_None ? NULL : PyUnicode_AsUTF8(item);
         item = PyTuple_GET_ITEM(args, 10);
         p->tm_gmtoff = item == Py_None ? 0 : PyLong_AsLong(item);
         if (PyErr_Occurred())
diff --git a/Objects/floatobject.c b/Objects/floatobject.c
--- a/Objects/floatobject.c
+++ b/Objects/floatobject.c
@@ -1274,7 +1274,7 @@
      * exp+4*ndigits and exp-4*ndigits are within the range of a long.
      */
 
-    s = _PyUnicode_AsStringAndSize(arg, &length);
+    s = PyUnicode_AsUTF8AndSize(arg, &length);
     if (s == NULL)
         return NULL;
     s_end = s + length;
@@ -1628,7 +1628,7 @@
                          Py_TYPE(arg)->tp_name);
         return NULL;
     }
-    s = _PyUnicode_AsString(arg);
+    s = PyUnicode_AsUTF8(arg);
     if (s == NULL)
         return NULL;
     if (strcmp(s, "double") == 0) {
diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c
--- a/Objects/moduleobject.c
+++ b/Objects/moduleobject.c
@@ -483,7 +483,7 @@
     if (name == NULL)
         return NULL;
     Py_DECREF(name);   /* module dict has still a reference */
-    return _PyUnicode_AsString(name);
+    return PyUnicode_AsUTF8(name);
 }
 
 PyObject*
@@ -516,7 +516,7 @@
     fileobj = PyModule_GetFilenameObject(m);
     if (fileobj == NULL)
         return NULL;
-    utf8 = _PyUnicode_AsString(fileobj);
+    utf8 = PyUnicode_AsUTF8(fileobj);
     Py_DECREF(fileobj);   /* module dict has still a reference */
     return utf8;
 }
@@ -569,7 +569,7 @@
             if (PyUnicode_READ_CHAR(key, 0) == '_' &&
                 PyUnicode_READ_CHAR(key, 1) != '_') {
                 if (Py_VerboseFlag > 1) {
-                    const char *s = _PyUnicode_AsString(key);
+                    const char *s = PyUnicode_AsUTF8(key);
                     if (s != NULL)
                         PySys_WriteStderr("#   clear[1] %s\n", s);
                     else
@@ -589,7 +589,7 @@
                 !_PyUnicode_EqualToASCIIString(key, "__builtins__"))
             {
                 if (Py_VerboseFlag > 1) {
-                    const char *s = _PyUnicode_AsString(key);
+                    const char *s = PyUnicode_AsUTF8(key);
                     if (s != NULL)
                         PySys_WriteStderr("#   clear[2] %s\n", s);
                     else
diff --git a/Objects/object.c b/Objects/object.c
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -890,7 +890,7 @@
     if (tp->tp_getattro != NULL)
         return (*tp->tp_getattro)(v, name);
     if (tp->tp_getattr != NULL) {
-        char *name_str = _PyUnicode_AsString(name);
+        char *name_str = PyUnicode_AsUTF8(name);
         if (name_str == NULL)
             return NULL;
         return (*tp->tp_getattr)(v, name_str);
@@ -934,7 +934,7 @@
         return err;
     }
     if (tp->tp_setattr != NULL) {
-        char *name_str = _PyUnicode_AsString(name);
+        char *name_str = PyUnicode_AsUTF8(name);
         if (name_str == NULL)
             return -1;
         err = (*tp->tp_setattr)(v, name_str, value);
diff --git a/Objects/setobject.c b/Objects/setobject.c
--- a/Objects/setobject.c
+++ b/Objects/setobject.c
@@ -2466,7 +2466,7 @@
     /* Exercise direct iteration */
     i = 0, count = 0;
     while (_PySet_NextEntry((PyObject *)dup, &i, &x, &hash)) {
-        s = _PyUnicode_AsString(x);
+        s = PyUnicode_AsUTF8(x);
         assert(s && (s[0] == 'a' || s[0] == 'b' || s[0] == 'c'));
         count++;
     }
diff --git a/Objects/structseq.c b/Objects/structseq.c
--- a/Objects/structseq.c
+++ b/Objects/structseq.c
@@ -194,7 +194,7 @@
         repr = PyObject_Repr(val);
         if (repr == NULL)
             return NULL;
-        crepr = _PyUnicode_AsString(repr);
+        crepr = PyUnicode_AsUTF8(repr);
         if (crepr == NULL) {
             Py_DECREF(repr);
             return NULL;
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -1626,7 +1626,7 @@
         PyObject *name = class_name(k);
         char *name_str;
         if (name != NULL) {
-            name_str = _PyUnicode_AsString(name);
+            name_str = PyUnicode_AsUTF8(name);
             if (name_str == NULL)
                 name_str = "?";
         } else
@@ -2575,7 +2575,7 @@
             char *doc_str;
             char *tp_doc;
 
-            doc_str = _PyUnicode_AsString(doc);
+            doc_str = PyUnicode_AsUTF8(doc);
             if (doc_str == NULL)
                 goto error;
             /* Silently truncate the docstring if it contains null bytes. */
@@ -2623,7 +2623,7 @@
     slotoffset = base->tp_basicsize;
     if (et->ht_slots != NULL) {
         for (i = 0; i < nslots; i++, mp++) {
-            mp->name = _PyUnicode_AsString(
+            mp->name = PyUnicode_AsUTF8(
                 PyTuple_GET_ITEM(et->ht_slots, i));
             if (mp->name == NULL)
                 goto error;
diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c
--- a/Parser/tokenizer.c
+++ b/Parser/tokenizer.c
@@ -446,7 +446,7 @@
     }
     if (PyUnicode_CheckExact(bufobj))
     {
-        buf = _PyUnicode_AsStringAndSize(bufobj, &buflen);
+        buf = PyUnicode_AsUTF8AndSize(bufobj, &buflen);
         if (buf == NULL) {
             goto error;
         }
diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c
--- a/Python/bltinmodule.c
+++ b/Python/bltinmodule.c
@@ -1903,8 +1903,8 @@
             /* stdin is a text stream, so it must have an
                encoding. */
             goto _readline_errors;
-        stdin_encoding_str = _PyUnicode_AsString(stdin_encoding);
-        stdin_errors_str = _PyUnicode_AsString(stdin_errors);
+        stdin_encoding_str = PyUnicode_AsUTF8(stdin_encoding);
+        stdin_errors_str = PyUnicode_AsUTF8(stdin_errors);
         if (!stdin_encoding_str || !stdin_errors_str)
             goto _readline_errors;
         tmp = _PyObject_CallMethodId(fout, &PyId_flush, NULL);
@@ -1920,8 +1920,8 @@
             stdout_errors = _PyObject_GetAttrId(fout, &PyId_errors);
             if (!stdout_encoding || !stdout_errors)
                 goto _readline_errors;
-            stdout_encoding_str = _PyUnicode_AsString(stdout_encoding);
-            stdout_errors_str = _PyUnicode_AsString(stdout_errors);
+            stdout_encoding_str = PyUnicode_AsUTF8(stdout_encoding);
+            stdout_errors_str = PyUnicode_AsUTF8(stdout_errors);
             if (!stdout_encoding_str || !stdout_errors_str)
                 goto _readline_errors;
             stringpo = PyObject_Str(prompt);
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -4712,7 +4712,7 @@
     if (PyMethod_Check(func))
         return PyEval_GetFuncName(PyMethod_GET_FUNCTION(func));
     else if (PyFunction_Check(func))
-        return _PyUnicode_AsString(((PyFunctionObject*)func)->func_name);
+        return PyUnicode_AsUTF8(((PyFunctionObject*)func)->func_name);
     else if (PyCFunction_Check(func))
         return ((PyCFunctionObject*)func)->m_ml->ml_name;
     else
@@ -5286,7 +5286,7 @@
     if (!obj)
         return;
 
-    obj_str = _PyUnicode_AsString(obj);
+    obj_str = PyUnicode_AsUTF8(obj);
     if (!obj_str)
         return;
 
diff --git a/Python/future.c b/Python/future.c
--- a/Python/future.c
+++ b/Python/future.c
@@ -21,7 +21,7 @@
     names = s->v.ImportFrom.names;
     for (i = 0; i < asdl_seq_LEN(names); i++) {
         alias_ty name = (alias_ty)asdl_seq_GET(names, i);
-        const char *feature = _PyUnicode_AsString(name->name);
+        const char *feature = PyUnicode_AsUTF8(name->name);
         if (!feature)
             return 0;
         if (strcmp(feature, FUTURE_NESTED_SCOPES) == 0) {
diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c
--- a/Python/pylifecycle.c
+++ b/Python/pylifecycle.c
@@ -205,7 +205,7 @@
     if (!name)
         goto error;
 
-    name_utf8 = _PyUnicode_AsString(name);
+    name_utf8 = PyUnicode_AsUTF8(name);
     if (name_utf8 == NULL)
         goto error;
     name_str = _PyMem_RawStrdup(name_utf8);
@@ -1285,7 +1285,7 @@
     encoding_attr = PyObject_GetAttrString(std, "encoding");
     if (encoding_attr != NULL) {
         const char * std_encoding;
-        std_encoding = _PyUnicode_AsString(encoding_attr);
+        std_encoding = PyUnicode_AsUTF8(encoding_attr);
         if (std_encoding != NULL) {
             PyObject *codec_info = _PyCodec_Lookup(std_encoding);
             Py_XDECREF(codec_info);
diff --git a/Python/pythonrun.c b/Python/pythonrun.c
--- a/Python/pythonrun.c
+++ b/Python/pythonrun.c
@@ -171,7 +171,7 @@
         if (v && v != Py_None) {
             oenc = _PyObject_GetAttrId(v, &PyId_encoding);
             if (oenc)
-                enc = _PyUnicode_AsString(oenc);
+                enc = PyUnicode_AsUTF8(oenc);
             if (!enc)
                 PyErr_Clear();
         }
@@ -182,7 +182,7 @@
         if (v == NULL)
             PyErr_Clear();
         else if (PyUnicode_Check(v)) {
-            ps1 = _PyUnicode_AsString(v);
+            ps1 = PyUnicode_AsUTF8(v);
             if (ps1 == NULL) {
                 PyErr_Clear();
                 ps1 = "";
@@ -195,7 +195,7 @@
         if (w == NULL)
             PyErr_Clear();
         else if (PyUnicode_Check(w)) {
-            ps2 = _PyUnicode_AsString(w);
+            ps2 = PyUnicode_AsUTF8(w);
             if (ps2 == NULL) {
                 PyErr_Clear();
                 ps2 = "";
@@ -514,7 +514,7 @@
     char *text;
     char *nl;
 
-    text = _PyUnicode_AsString(text_obj);
+    text = PyUnicode_AsUTF8(text_obj);
     if (text == NULL)
         return;
 
diff --git a/Python/structmember.c b/Python/structmember.c
--- a/Python/structmember.c
+++ b/Python/structmember.c
@@ -252,7 +252,7 @@
         char *string;
         Py_ssize_t len;
 
-        string = _PyUnicode_AsStringAndSize(v, &len);
+        string = PyUnicode_AsUTF8AndSize(v, &len);
         if (string == NULL || len != 1) {
             PyErr_BadArgument();
             return -1;
diff --git a/Python/sysmodule.c b/Python/sysmodule.c
--- a/Python/sysmodule.c
+++ b/Python/sysmodule.c
@@ -114,7 +114,7 @@
     stdout_encoding = _PyObject_GetAttrId(outf, &PyId_encoding);
     if (stdout_encoding == NULL)
         goto error;
-    stdout_encoding_str = _PyUnicode_AsString(stdout_encoding);
+    stdout_encoding_str = PyUnicode_AsUTF8(stdout_encoding);
     if (stdout_encoding_str == NULL)
         goto error;
 
@@ -2411,7 +2411,7 @@
     if (message != NULL) {
         if (sys_pyfile_write_unicode(message, file) != 0) {
             PyErr_Clear();
-            utf8 = _PyUnicode_AsString(message);
+            utf8 = PyUnicode_AsUTF8(message);
             if (utf8 != NULL)
                 fputs(utf8, fp);
         }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 03:17:23 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 08:17:23 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Replaced_outdated_macros_=5FPyUnicode=5FAsString_and_=5F?=
 =?utf-8?q?PyUnicode=5FAsStringAndSize?=
Message-ID: <20161120081722.45206.25072.E77743D5@psf.io>

https://hg.python.org/cpython/rev/c9cd4b2de74f
changeset:   105209:c9cd4b2de74f
parent:      105207:9cd42ed64bdb
parent:      105208:39f0e0cf49fd
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 09:13:40 2016 +0200
summary:
  Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.

files:
  Modules/_ctypes/_ctypes.c          |  12 ++++++------
  Modules/_ctypes/stgdict.c          |   2 +-
  Modules/_datetimemodule.c          |   4 ++--
  Modules/_io/stringio.c             |   2 +-
  Modules/_io/textio.c               |   4 ++--
  Modules/_pickle.c                  |   2 +-
  Modules/_sqlite/connection.c       |   4 ++--
  Modules/_sqlite/cursor.c           |   4 ++--
  Modules/_sqlite/row.c              |   4 ++--
  Modules/_sqlite/statement.c        |   6 +++---
  Modules/cjkcodecs/cjkcodecs.h      |   2 +-
  Modules/cjkcodecs/multibytecodec.c |   4 ++--
  Modules/parsermodule.c             |   4 ++--
  Modules/posixmodule.c              |   2 +-
  Modules/socketmodule.c             |   2 +-
  Modules/syslogmodule.c             |   4 ++--
  Modules/timemodule.c               |   2 +-
  Objects/floatobject.c              |   4 ++--
  Objects/moduleobject.c             |   8 ++++----
  Objects/object.c                   |   4 ++--
  Objects/setobject.c                |   2 +-
  Objects/structseq.c                |   2 +-
  Objects/typeobject.c               |   6 +++---
  Parser/tokenizer.c                 |   2 +-
  Python/bltinmodule.c               |   8 ++++----
  Python/ceval.c                     |   4 ++--
  Python/future.c                    |   2 +-
  Python/pylifecycle.c               |   4 ++--
  Python/pythonrun.c                 |   8 ++++----
  Python/structmember.c              |   2 +-
  Python/sysmodule.c                 |   4 ++--
  31 files changed, 62 insertions(+), 62 deletions(-)


diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c
--- a/Modules/_ctypes/_ctypes.c
+++ b/Modules/_ctypes/_ctypes.c
@@ -1723,7 +1723,7 @@
     if (stgd && CDataObject_Check(value) && stgd->proto && PyUnicode_Check(stgd->proto)) {
         PyCArgObject *parg;
 
-        switch (_PyUnicode_AsString(stgd->proto)[0]) {
+        switch (PyUnicode_AsUTF8(stgd->proto)[0]) {
         case 'z': /* c_char_p */
         case 'Z': /* c_wchar_p */
             parg = PyCArgObject_new();
@@ -1835,7 +1835,7 @@
 
     dict = PyObject_stgdict((PyObject *)self);
     assert(dict); /* Cannot be NULL for CDataObject instances */
-    fmt = _PyUnicode_AsString(dict->proto);
+    fmt = PyUnicode_AsUTF8(dict->proto);
     assert(fmt);
 
     fd = _ctypes_get_fielddesc(fmt);
@@ -2059,7 +2059,7 @@
     assert(dict);
 
     /* I think we can rely on this being a one-character string */
-    fmt = _PyUnicode_AsString(dict->proto);
+    fmt = PyUnicode_AsUTF8(dict->proto);
     assert(fmt);
 
     fd = _ctypes_get_fielddesc(fmt);
@@ -3128,7 +3128,7 @@
         /* simple pointer types, c_void_p, c_wchar_p, BSTR, ... */
         && PyUnicode_Check(dict->proto)
 /* We only allow c_void_p, c_char_p and c_wchar_p as a simple output parameter type */
-        && (strchr("PzZ", _PyUnicode_AsString(dict->proto)[0]))) {
+        && (strchr("PzZ", PyUnicode_AsUTF8(dict->proto)[0]))) {
         return 1;
     }
 
@@ -3218,7 +3218,7 @@
         return *pname ? 1 : 0;
     }
     if (PyUnicode_Check(obj)) {
-        *pname = _PyUnicode_AsString(obj);
+        *pname = PyUnicode_AsUTF8(obj);
         return *pname ? 1 : 0;
     }
     PyErr_SetString(PyExc_TypeError,
@@ -5233,7 +5233,7 @@
     dict = PyType_stgdict(arg);
     if (dict) {
         if (PyUnicode_Check(dict->proto)
-            && (strchr("sPzUZXO", _PyUnicode_AsString(dict->proto)[0]))) {
+            && (strchr("sPzUZXO", PyUnicode_AsUTF8(dict->proto)[0]))) {
             /* simple pointer types, c_void_p, c_wchar_p, BSTR, ... */
             return 1;
         }
diff --git a/Modules/_ctypes/stgdict.c b/Modules/_ctypes/stgdict.c
--- a/Modules/_ctypes/stgdict.c
+++ b/Modules/_ctypes/stgdict.c
@@ -489,7 +489,7 @@
 
         if (isStruct && !isPacked) {
             char *fieldfmt = dict->format ? dict->format : "B";
-            char *fieldname = _PyUnicode_AsString(name);
+            char *fieldname = PyUnicode_AsUTF8(name);
             char *ptr;
             Py_ssize_t len;
             char *buf;
diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c
--- a/Modules/_datetimemodule.c
+++ b/Modules/_datetimemodule.c
@@ -1219,7 +1219,7 @@
     assert(object && format && timetuple);
     assert(PyUnicode_Check(format));
     /* Convert the input format to a C string and size */
-    pin = _PyUnicode_AsStringAndSize(format, &flen);
+    pin = PyUnicode_AsUTF8AndSize(format, &flen);
     if (!pin)
         return NULL;
 
@@ -1287,7 +1287,7 @@
             }
             assert(Zreplacement != NULL);
             assert(PyUnicode_Check(Zreplacement));
-            ptoappend = _PyUnicode_AsStringAndSize(Zreplacement,
+            ptoappend = PyUnicode_AsUTF8AndSize(Zreplacement,
                                                   &ntoappend);
             if (ptoappend == NULL)
                 goto Done;
diff --git a/Modules/_io/stringio.c b/Modules/_io/stringio.c
--- a/Modules/_io/stringio.c
+++ b/Modules/_io/stringio.c
@@ -708,7 +708,7 @@
                          Py_TYPE(newline_obj)->tp_name);
             return -1;
         }
-        newline = _PyUnicode_AsString(newline_obj);
+        newline = PyUnicode_AsUTF8(newline_obj);
         if (newline == NULL)
             return -1;
     }
diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c
--- a/Modules/_io/textio.c
+++ b/Modules/_io/textio.c
@@ -921,7 +921,7 @@
             Py_CLEAR(self->encoding);
     }
     if (self->encoding != NULL) {
-        encoding = _PyUnicode_AsString(self->encoding);
+        encoding = PyUnicode_AsUTF8(self->encoding);
         if (encoding == NULL)
             goto error;
     }
@@ -964,7 +964,7 @@
     }
     self->writetranslate = (newline == NULL || newline[0] != '\0');
     if (!self->readuniversal && self->readnl) {
-        self->writenl = _PyUnicode_AsString(self->readnl);
+        self->writenl = PyUnicode_AsUTF8(self->readnl);
         if (self->writenl == NULL)
             goto error;
         if (!strcmp(self->writenl, "\n"))
diff --git a/Modules/_pickle.c b/Modules/_pickle.c
--- a/Modules/_pickle.c
+++ b/Modules/_pickle.c
@@ -1951,7 +1951,7 @@
         if (repr == NULL)
             goto error;
 
-        string = _PyUnicode_AsStringAndSize(repr, &size);
+        string = PyUnicode_AsUTF8AndSize(repr, &size);
         if (string == NULL)
             goto error;
 
diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c
--- a/Modules/_sqlite/connection.c
+++ b/Modules/_sqlite/connection.c
@@ -505,7 +505,7 @@
     } else if (PyFloat_Check(py_val)) {
         sqlite3_result_double(context, PyFloat_AsDouble(py_val));
     } else if (PyUnicode_Check(py_val)) {
-        const char *str = _PyUnicode_AsString(py_val);
+        const char *str = PyUnicode_AsUTF8(py_val);
         if (str == NULL)
             return -1;
         sqlite3_result_text(context, str, -1, SQLITE_TRANSIENT);
@@ -1527,7 +1527,7 @@
         }
     }
 
-    uppercase_name_str = _PyUnicode_AsString(uppercase_name);
+    uppercase_name_str = PyUnicode_AsUTF8(uppercase_name);
     if (!uppercase_name_str)
         goto finally;
 
diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c
--- a/Modules/_sqlite/cursor.c
+++ b/Modules/_sqlite/cursor.c
@@ -465,7 +465,7 @@
         pysqlite_statement_reset(self->statement);
     }
 
-    operation_cstr = _PyUnicode_AsStringAndSize(operation, &operation_len);
+    operation_cstr = PyUnicode_AsUTF8AndSize(operation, &operation_len);
     if (operation_cstr == NULL)
         goto error;
 
@@ -689,7 +689,7 @@
     self->reset = 0;
 
     if (PyUnicode_Check(script_obj)) {
-        script_cstr = _PyUnicode_AsString(script_obj);
+        script_cstr = PyUnicode_AsUTF8(script_obj);
         if (!script_cstr) {
             return NULL;
         }
diff --git a/Modules/_sqlite/row.c b/Modules/_sqlite/row.c
--- a/Modules/_sqlite/row.c
+++ b/Modules/_sqlite/row.c
@@ -98,7 +98,7 @@
         Py_XINCREF(item);
         return item;
     } else if (PyUnicode_Check(idx)) {
-        key = _PyUnicode_AsString(idx);
+        key = PyUnicode_AsUTF8(idx);
         if (key == NULL)
             return NULL;
 
@@ -108,7 +108,7 @@
             PyObject *obj;
             obj = PyTuple_GET_ITEM(self->description, i);
             obj = PyTuple_GET_ITEM(obj, 0);
-            compare_key = _PyUnicode_AsString(obj);
+            compare_key = PyUnicode_AsUTF8(obj);
             if (!compare_key) {
                 return NULL;
             }
diff --git a/Modules/_sqlite/statement.c b/Modules/_sqlite/statement.c
--- a/Modules/_sqlite/statement.c
+++ b/Modules/_sqlite/statement.c
@@ -59,7 +59,7 @@
     self->st = NULL;
     self->in_use = 0;
 
-    sql_cstr = _PyUnicode_AsStringAndSize(sql, &sql_cstr_len);
+    sql_cstr = PyUnicode_AsUTF8AndSize(sql, &sql_cstr_len);
     if (sql_cstr == NULL) {
         rc = PYSQLITE_SQL_WRONG_TYPE;
         return rc;
@@ -152,7 +152,7 @@
             rc = sqlite3_bind_double(self->st, pos, PyFloat_AsDouble(parameter));
             break;
         case TYPE_UNICODE:
-            string = _PyUnicode_AsStringAndSize(parameter, &buflen);
+            string = PyUnicode_AsUTF8AndSize(parameter, &buflen);
             if (string == NULL)
                 return -1;
             if (buflen > INT_MAX) {
@@ -325,7 +325,7 @@
     Py_ssize_t sql_len;
     sqlite3_stmt* new_st;
 
-    sql_cstr = _PyUnicode_AsStringAndSize(self->sql, &sql_len);
+    sql_cstr = PyUnicode_AsUTF8AndSize(self->sql, &sql_len);
     if (sql_cstr == NULL) {
         rc = PYSQLITE_SQL_WRONG_TYPE;
         return rc;
diff --git a/Modules/cjkcodecs/cjkcodecs.h b/Modules/cjkcodecs/cjkcodecs.h
--- a/Modules/cjkcodecs/cjkcodecs.h
+++ b/Modules/cjkcodecs/cjkcodecs.h
@@ -267,7 +267,7 @@
                         "encoding name must be a string.");
         return NULL;
     }
-    enc = _PyUnicode_AsString(encoding);
+    enc = PyUnicode_AsUTF8(encoding);
     if (enc == NULL)
         return NULL;
 
diff --git a/Modules/cjkcodecs/multibytecodec.c b/Modules/cjkcodecs/multibytecodec.c
--- a/Modules/cjkcodecs/multibytecodec.c
+++ b/Modules/cjkcodecs/multibytecodec.c
@@ -85,7 +85,7 @@
     const char *str;
 
     assert(PyUnicode_Check(errors));
-    str = _PyUnicode_AsString(errors);
+    str = PyUnicode_AsUTF8(errors);
     if (str == NULL)
         return NULL;
     cb = PyCodec_LookupError(str);
@@ -138,7 +138,7 @@
         return -1;
     }
 
-    str = _PyUnicode_AsString(value);
+    str = PyUnicode_AsUTF8(value);
     if (str == NULL)
         return -1;
 
diff --git a/Modules/parsermodule.c b/Modules/parsermodule.c
--- a/Modules/parsermodule.c
+++ b/Modules/parsermodule.c
@@ -912,7 +912,7 @@
                     Py_DECREF(o);
                 }
             }
-            temp_str = _PyUnicode_AsStringAndSize(temp, &len);
+            temp_str = PyUnicode_AsUTF8AndSize(temp, &len);
             if (temp_str == NULL) {
                 Py_DECREF(temp);
                 Py_XDECREF(elem);
@@ -1013,7 +1013,7 @@
             if (res && encoding) {
                 Py_ssize_t len;
                 const char *temp;
-                temp = _PyUnicode_AsStringAndSize(encoding, &len);
+                temp = PyUnicode_AsUTF8AndSize(encoding, &len);
                 if (temp == NULL) {
                     Py_DECREF(res);
                     Py_DECREF(encoding);
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -9325,7 +9325,7 @@
                 "configuration names must be strings or integers");
             return 0;
         }
-        confname = _PyUnicode_AsString(arg);
+        confname = PyUnicode_AsUTF8(arg);
         if (confname == NULL)
             return 0;
         while (lo < hi) {
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -6017,7 +6017,7 @@
         PyOS_snprintf(pbuf, sizeof(pbuf), "%ld", value);
         pptr = pbuf;
     } else if (PyUnicode_Check(pobj)) {
-        pptr = _PyUnicode_AsString(pobj);
+        pptr = PyUnicode_AsUTF8(pobj);
         if (pptr == NULL)
             goto err;
     } else if (PyBytes_Check(pobj)) {
diff --git a/Modules/syslogmodule.c b/Modules/syslogmodule.c
--- a/Modules/syslogmodule.c
+++ b/Modules/syslogmodule.c
@@ -139,7 +139,7 @@
      * If NULL, just let openlog figure it out (probably using C argv[0]).
      */
     if (S_ident_o) {
-        ident = _PyUnicode_AsString(S_ident_o);
+        ident = PyUnicode_AsUTF8(S_ident_o);
         if (ident == NULL)
             return NULL;
     }
@@ -167,7 +167,7 @@
             return NULL;
     }
 
-    message = _PyUnicode_AsString(message_object);
+    message = PyUnicode_AsUTF8(message_object);
     if (message == NULL)
         return NULL;
 
diff --git a/Modules/timemodule.c b/Modules/timemodule.c
--- a/Modules/timemodule.c
+++ b/Modules/timemodule.c
@@ -441,7 +441,7 @@
     if (Py_TYPE(args) == &StructTimeType) {
         PyObject *item;
         item = PyTuple_GET_ITEM(args, 9);
-        p->tm_zone = item == Py_None ? NULL : _PyUnicode_AsString(item);
+        p->tm_zone = item == Py_None ? NULL : PyUnicode_AsUTF8(item);
         item = PyTuple_GET_ITEM(args, 10);
         p->tm_gmtoff = item == Py_None ? 0 : PyLong_AsLong(item);
         if (PyErr_Occurred())
diff --git a/Objects/floatobject.c b/Objects/floatobject.c
--- a/Objects/floatobject.c
+++ b/Objects/floatobject.c
@@ -1274,7 +1274,7 @@
      * exp+4*ndigits and exp-4*ndigits are within the range of a long.
      */
 
-    s = _PyUnicode_AsStringAndSize(arg, &length);
+    s = PyUnicode_AsUTF8AndSize(arg, &length);
     if (s == NULL)
         return NULL;
     s_end = s + length;
@@ -1628,7 +1628,7 @@
                          Py_TYPE(arg)->tp_name);
         return NULL;
     }
-    s = _PyUnicode_AsString(arg);
+    s = PyUnicode_AsUTF8(arg);
     if (s == NULL)
         return NULL;
     if (strcmp(s, "double") == 0) {
diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c
--- a/Objects/moduleobject.c
+++ b/Objects/moduleobject.c
@@ -483,7 +483,7 @@
     if (name == NULL)
         return NULL;
     Py_DECREF(name);   /* module dict has still a reference */
-    return _PyUnicode_AsString(name);
+    return PyUnicode_AsUTF8(name);
 }
 
 PyObject*
@@ -516,7 +516,7 @@
     fileobj = PyModule_GetFilenameObject(m);
     if (fileobj == NULL)
         return NULL;
-    utf8 = _PyUnicode_AsString(fileobj);
+    utf8 = PyUnicode_AsUTF8(fileobj);
     Py_DECREF(fileobj);   /* module dict has still a reference */
     return utf8;
 }
@@ -569,7 +569,7 @@
             if (PyUnicode_READ_CHAR(key, 0) == '_' &&
                 PyUnicode_READ_CHAR(key, 1) != '_') {
                 if (Py_VerboseFlag > 1) {
-                    const char *s = _PyUnicode_AsString(key);
+                    const char *s = PyUnicode_AsUTF8(key);
                     if (s != NULL)
                         PySys_WriteStderr("#   clear[1] %s\n", s);
                     else
@@ -589,7 +589,7 @@
                 !_PyUnicode_EqualToASCIIString(key, "__builtins__"))
             {
                 if (Py_VerboseFlag > 1) {
-                    const char *s = _PyUnicode_AsString(key);
+                    const char *s = PyUnicode_AsUTF8(key);
                     if (s != NULL)
                         PySys_WriteStderr("#   clear[2] %s\n", s);
                     else
diff --git a/Objects/object.c b/Objects/object.c
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -890,7 +890,7 @@
     if (tp->tp_getattro != NULL)
         return (*tp->tp_getattro)(v, name);
     if (tp->tp_getattr != NULL) {
-        char *name_str = _PyUnicode_AsString(name);
+        char *name_str = PyUnicode_AsUTF8(name);
         if (name_str == NULL)
             return NULL;
         return (*tp->tp_getattr)(v, name_str);
@@ -934,7 +934,7 @@
         return err;
     }
     if (tp->tp_setattr != NULL) {
-        char *name_str = _PyUnicode_AsString(name);
+        char *name_str = PyUnicode_AsUTF8(name);
         if (name_str == NULL)
             return -1;
         err = (*tp->tp_setattr)(v, name_str, value);
diff --git a/Objects/setobject.c b/Objects/setobject.c
--- a/Objects/setobject.c
+++ b/Objects/setobject.c
@@ -2466,7 +2466,7 @@
     /* Exercise direct iteration */
     i = 0, count = 0;
     while (_PySet_NextEntry((PyObject *)dup, &i, &x, &hash)) {
-        s = _PyUnicode_AsString(x);
+        s = PyUnicode_AsUTF8(x);
         assert(s && (s[0] == 'a' || s[0] == 'b' || s[0] == 'c'));
         count++;
     }
diff --git a/Objects/structseq.c b/Objects/structseq.c
--- a/Objects/structseq.c
+++ b/Objects/structseq.c
@@ -194,7 +194,7 @@
         repr = PyObject_Repr(val);
         if (repr == NULL)
             return NULL;
-        crepr = _PyUnicode_AsString(repr);
+        crepr = PyUnicode_AsUTF8(repr);
         if (crepr == NULL) {
             Py_DECREF(repr);
             return NULL;
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -1626,7 +1626,7 @@
         PyObject *name = class_name(k);
         char *name_str;
         if (name != NULL) {
-            name_str = _PyUnicode_AsString(name);
+            name_str = PyUnicode_AsUTF8(name);
             if (name_str == NULL)
                 name_str = "?";
         } else
@@ -2575,7 +2575,7 @@
             char *doc_str;
             char *tp_doc;
 
-            doc_str = _PyUnicode_AsString(doc);
+            doc_str = PyUnicode_AsUTF8(doc);
             if (doc_str == NULL)
                 goto error;
             /* Silently truncate the docstring if it contains null bytes. */
@@ -2623,7 +2623,7 @@
     slotoffset = base->tp_basicsize;
     if (et->ht_slots != NULL) {
         for (i = 0; i < nslots; i++, mp++) {
-            mp->name = _PyUnicode_AsString(
+            mp->name = PyUnicode_AsUTF8(
                 PyTuple_GET_ITEM(et->ht_slots, i));
             if (mp->name == NULL)
                 goto error;
diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c
--- a/Parser/tokenizer.c
+++ b/Parser/tokenizer.c
@@ -446,7 +446,7 @@
     }
     if (PyUnicode_CheckExact(bufobj))
     {
-        buf = _PyUnicode_AsStringAndSize(bufobj, &buflen);
+        buf = PyUnicode_AsUTF8AndSize(bufobj, &buflen);
         if (buf == NULL) {
             goto error;
         }
diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c
--- a/Python/bltinmodule.c
+++ b/Python/bltinmodule.c
@@ -1903,8 +1903,8 @@
             /* stdin is a text stream, so it must have an
                encoding. */
             goto _readline_errors;
-        stdin_encoding_str = _PyUnicode_AsString(stdin_encoding);
-        stdin_errors_str = _PyUnicode_AsString(stdin_errors);
+        stdin_encoding_str = PyUnicode_AsUTF8(stdin_encoding);
+        stdin_errors_str = PyUnicode_AsUTF8(stdin_errors);
         if (!stdin_encoding_str || !stdin_errors_str)
             goto _readline_errors;
         tmp = _PyObject_CallMethodId(fout, &PyId_flush, NULL);
@@ -1920,8 +1920,8 @@
             stdout_errors = _PyObject_GetAttrId(fout, &PyId_errors);
             if (!stdout_encoding || !stdout_errors)
                 goto _readline_errors;
-            stdout_encoding_str = _PyUnicode_AsString(stdout_encoding);
-            stdout_errors_str = _PyUnicode_AsString(stdout_errors);
+            stdout_encoding_str = PyUnicode_AsUTF8(stdout_encoding);
+            stdout_errors_str = PyUnicode_AsUTF8(stdout_errors);
             if (!stdout_encoding_str || !stdout_errors_str)
                 goto _readline_errors;
             stringpo = PyObject_Str(prompt);
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -4723,7 +4723,7 @@
     if (PyMethod_Check(func))
         return PyEval_GetFuncName(PyMethod_GET_FUNCTION(func));
     else if (PyFunction_Check(func))
-        return _PyUnicode_AsString(((PyFunctionObject*)func)->func_name);
+        return PyUnicode_AsUTF8(((PyFunctionObject*)func)->func_name);
     else if (PyCFunction_Check(func))
         return ((PyCFunctionObject*)func)->m_ml->ml_name;
     else
@@ -5297,7 +5297,7 @@
     if (!obj)
         return;
 
-    obj_str = _PyUnicode_AsString(obj);
+    obj_str = PyUnicode_AsUTF8(obj);
     if (!obj_str)
         return;
 
diff --git a/Python/future.c b/Python/future.c
--- a/Python/future.c
+++ b/Python/future.c
@@ -21,7 +21,7 @@
     names = s->v.ImportFrom.names;
     for (i = 0; i < asdl_seq_LEN(names); i++) {
         alias_ty name = (alias_ty)asdl_seq_GET(names, i);
-        const char *feature = _PyUnicode_AsString(name->name);
+        const char *feature = PyUnicode_AsUTF8(name->name);
         if (!feature)
             return 0;
         if (strcmp(feature, FUTURE_NESTED_SCOPES) == 0) {
diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c
--- a/Python/pylifecycle.c
+++ b/Python/pylifecycle.c
@@ -205,7 +205,7 @@
     if (!name)
         goto error;
 
-    name_utf8 = _PyUnicode_AsString(name);
+    name_utf8 = PyUnicode_AsUTF8(name);
     if (name_utf8 == NULL)
         goto error;
     name_str = _PyMem_RawStrdup(name_utf8);
@@ -1285,7 +1285,7 @@
     encoding_attr = PyObject_GetAttrString(std, "encoding");
     if (encoding_attr != NULL) {
         const char * std_encoding;
-        std_encoding = _PyUnicode_AsString(encoding_attr);
+        std_encoding = PyUnicode_AsUTF8(encoding_attr);
         if (std_encoding != NULL) {
             PyObject *codec_info = _PyCodec_Lookup(std_encoding);
             Py_XDECREF(codec_info);
diff --git a/Python/pythonrun.c b/Python/pythonrun.c
--- a/Python/pythonrun.c
+++ b/Python/pythonrun.c
@@ -171,7 +171,7 @@
         if (v && v != Py_None) {
             oenc = _PyObject_GetAttrId(v, &PyId_encoding);
             if (oenc)
-                enc = _PyUnicode_AsString(oenc);
+                enc = PyUnicode_AsUTF8(oenc);
             if (!enc)
                 PyErr_Clear();
         }
@@ -182,7 +182,7 @@
         if (v == NULL)
             PyErr_Clear();
         else if (PyUnicode_Check(v)) {
-            ps1 = _PyUnicode_AsString(v);
+            ps1 = PyUnicode_AsUTF8(v);
             if (ps1 == NULL) {
                 PyErr_Clear();
                 ps1 = "";
@@ -195,7 +195,7 @@
         if (w == NULL)
             PyErr_Clear();
         else if (PyUnicode_Check(w)) {
-            ps2 = _PyUnicode_AsString(w);
+            ps2 = PyUnicode_AsUTF8(w);
             if (ps2 == NULL) {
                 PyErr_Clear();
                 ps2 = "";
@@ -514,7 +514,7 @@
     char *text;
     char *nl;
 
-    text = _PyUnicode_AsString(text_obj);
+    text = PyUnicode_AsUTF8(text_obj);
     if (text == NULL)
         return;
 
diff --git a/Python/structmember.c b/Python/structmember.c
--- a/Python/structmember.c
+++ b/Python/structmember.c
@@ -252,7 +252,7 @@
         char *string;
         Py_ssize_t len;
 
-        string = _PyUnicode_AsStringAndSize(v, &len);
+        string = PyUnicode_AsUTF8AndSize(v, &len);
         if (string == NULL || len != 1) {
             PyErr_BadArgument();
             return -1;
diff --git a/Python/sysmodule.c b/Python/sysmodule.c
--- a/Python/sysmodule.c
+++ b/Python/sysmodule.c
@@ -114,7 +114,7 @@
     stdout_encoding = _PyObject_GetAttrId(outf, &PyId_encoding);
     if (stdout_encoding == NULL)
         goto error;
-    stdout_encoding_str = _PyUnicode_AsString(stdout_encoding);
+    stdout_encoding_str = PyUnicode_AsUTF8(stdout_encoding);
     if (stdout_encoding_str == NULL)
         goto error;
 
@@ -2412,7 +2412,7 @@
     if (message != NULL) {
         if (sys_pyfile_write_unicode(message, file) != 0) {
             PyErr_Clear();
-            utf8 = _PyUnicode_AsString(message);
+            utf8 = PyUnicode_AsUTF8(message);
             if (utf8 != NULL)
                 fputs(utf8, fp);
         }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 03:17:34 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 08:17:34 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Added_the_const_qualifier_?=
 =?utf-8?q?to_char*_variables_that_refer_to_readonly_internal?=
Message-ID: <20161120081723.9962.5810.F5D99409@psf.io>

https://hg.python.org/cpython/rev/6e5404da46b1
changeset:   105210:6e5404da46b1
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 10:16:47 2016 +0200
summary:
  Added the const qualifier to char* variables that refer to readonly internal
UTF-8 represenatation of Unicode objects.

files:
  Modules/_ctypes/_ctypes.c    |   4 ++--
  Modules/_ctypes/stgdict.c    |   4 ++--
  Modules/_elementtree.c       |   2 +-
  Modules/_io/bufferedio.c     |   2 +-
  Modules/_io/stringio.c       |   2 +-
  Modules/_pickle.c            |   4 ++--
  Modules/_sqlite/connection.c |   2 +-
  Modules/_sqlite/row.c        |   8 ++++----
  Modules/_sqlite/statement.c  |   2 +-
  Modules/_tkinter.c           |   6 +++---
  Modules/socketmodule.c       |   2 +-
  Modules/syslogmodule.c       |   2 +-
  Objects/fileobject.c         |   8 +++-----
  Objects/floatobject.c        |   6 +++---
  Objects/longobject.c         |   3 ++-
  Objects/moduleobject.c       |   4 ++--
  Objects/setobject.c          |   2 +-
  Objects/structseq.c          |   2 +-
  Objects/typeobject.c         |   4 ++--
  Python/bltinmodule.c         |   4 ++--
  Python/ceval.c               |  10 +++++-----
  Python/codecs.c              |   2 +-
  Python/compile.c             |   2 +-
  Python/getargs.c             |  14 +++++++-------
  Python/import.c              |   2 +-
  Python/pylifecycle.c         |   6 +++---
  Python/pythonrun.c           |   6 +++---
  Python/structmember.c        |   2 +-
  Python/sysmodule.c           |   4 ++--
  29 files changed, 60 insertions(+), 61 deletions(-)


diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c
--- a/Modules/_ctypes/_ctypes.c
+++ b/Modules/_ctypes/_ctypes.c
@@ -1829,7 +1829,7 @@
 PyCSimpleType_paramfunc(CDataObject *self)
 {
     StgDictObject *dict;
-    char *fmt;
+    const char *fmt;
     PyCArgObject *parg;
     struct fielddesc *fd;
 
@@ -2039,7 +2039,7 @@
 PyCSimpleType_from_param(PyObject *type, PyObject *value)
 {
     StgDictObject *dict;
-    char *fmt;
+    const char *fmt;
     PyCArgObject *parg;
     struct fielddesc *fd;
     PyObject *as_parameter;
diff --git a/Modules/_ctypes/stgdict.c b/Modules/_ctypes/stgdict.c
--- a/Modules/_ctypes/stgdict.c
+++ b/Modules/_ctypes/stgdict.c
@@ -488,8 +488,8 @@
             bitsize = 0;
 
         if (isStruct && !isPacked) {
-            char *fieldfmt = dict->format ? dict->format : "B";
-            char *fieldname = PyUnicode_AsUTF8(name);
+            const char *fieldfmt = dict->format ? dict->format : "B";
+            const char *fieldname = PyUnicode_AsUTF8(name);
             char *ptr;
             Py_ssize_t len;
             char *buf;
diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c
--- a/Modules/_elementtree.c
+++ b/Modules/_elementtree.c
@@ -3613,7 +3613,7 @@
 
     for (i = 0; i < PySequence_Size(events_seq); ++i) {
         PyObject *event_name_obj = PySequence_Fast_GET_ITEM(events_seq, i);
-        char *event_name = NULL;
+        const char *event_name = NULL;
         if (PyUnicode_Check(event_name_obj)) {
             event_name = PyUnicode_AsUTF8(event_name_obj);
         } else if (PyBytes_Check(event_name_obj)) {
diff --git a/Modules/_io/bufferedio.c b/Modules/_io/bufferedio.c
--- a/Modules/_io/bufferedio.c
+++ b/Modules/_io/bufferedio.c
@@ -305,7 +305,7 @@
             "could not acquire lock for %A at interpreter "
             "shutdown, possibly due to daemon threads",
             (PyObject *) self);
-        char *msg = PyUnicode_AsUTF8(msgobj);
+        const char *msg = PyUnicode_AsUTF8(msgobj);
         Py_FatalError(msg);
     }
     return 1;
diff --git a/Modules/_io/stringio.c b/Modules/_io/stringio.c
--- a/Modules/_io/stringio.c
+++ b/Modules/_io/stringio.c
@@ -693,7 +693,7 @@
                            PyObject *newline_obj)
 /*[clinic end generated code: output=a421ea023b22ef4e input=cee2d9181b2577a3]*/
 {
-    char *newline = "\n";
+    const char *newline = "\n";
     Py_ssize_t value_len;
 
     /* Parse the newline argument. We only want to allow unicode objects or
diff --git a/Modules/_pickle.c b/Modules/_pickle.c
--- a/Modules/_pickle.c
+++ b/Modules/_pickle.c
@@ -1941,7 +1941,7 @@
             goto error;
     }
     else {
-        char *string;
+        const char *string;
 
         /* proto < 2: write the repr and newline.  This is quadratic-time (in
            the number of digits), in both directions.  We add a trailing 'L'
@@ -2218,7 +2218,7 @@
 {
     PyObject *encoded = NULL;
     Py_ssize_t size;
-    char *data;
+    const char *data;
     int r;
 
     if (PyUnicode_READY(obj))
diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c
--- a/Modules/_sqlite/connection.c
+++ b/Modules/_sqlite/connection.c
@@ -1489,7 +1489,7 @@
     PyObject* retval;
     Py_ssize_t i, len;
     _Py_IDENTIFIER(upper);
-    char *uppercase_name_str;
+    const char *uppercase_name_str;
     int rc;
     unsigned int kind;
     void *data;
diff --git a/Modules/_sqlite/row.c b/Modules/_sqlite/row.c
--- a/Modules/_sqlite/row.c
+++ b/Modules/_sqlite/row.c
@@ -79,12 +79,12 @@
 PyObject* pysqlite_row_subscript(pysqlite_Row* self, PyObject* idx)
 {
     Py_ssize_t _idx;
-    char* key;
+    const char *key;
     Py_ssize_t nitems, i;
-    char* compare_key;
+    const char *compare_key;
 
-    char* p1;
-    char* p2;
+    const char *p1;
+    const char *p2;
 
     PyObject* item;
 
diff --git a/Modules/_sqlite/statement.c b/Modules/_sqlite/statement.c
--- a/Modules/_sqlite/statement.c
+++ b/Modules/_sqlite/statement.c
@@ -112,7 +112,7 @@
 int pysqlite_statement_bind_parameter(pysqlite_Statement* self, int pos, PyObject* parameter)
 {
     int rc = SQLITE_OK;
-    char* string;
+    const char *string;
     Py_ssize_t buflen;
     parameter_type paramtype;
 
diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c
--- a/Modules/_tkinter.c
+++ b/Modules/_tkinter.c
@@ -536,7 +536,7 @@
     else if (PyUnicode_Check(arg)) {
         int argc;
         const char **argv;
-        char *list = PyUnicode_AsUTF8(arg);
+        const char *list = PyUnicode_AsUTF8(arg);
 
         if (list == NULL ||
             Tcl_SplitList((Tcl_Interp *)NULL, list, &argc, &argv) != TCL_OK) {
@@ -993,7 +993,7 @@
     Tcl_Obj *result;
     int neg;
     PyObject *hexstr;
-    char *hexchars;
+    const char *hexchars;
     mp_int bigValue;
 
     neg = Py_SIZE(value) < 0;
@@ -1723,7 +1723,7 @@
 static int
 varname_converter(PyObject *in, void *_out)
 {
-    char *s;
+    const char *s;
     const char **out = (const char**)_out;
     if (PyBytes_Check(in)) {
         if (PyBytes_GET_SIZE(in) > INT_MAX) {
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -5982,7 +5982,7 @@
     PyObject *hobj = NULL;
     PyObject *pobj = (PyObject *)NULL;
     char pbuf[30];
-    char *hptr, *pptr;
+    const char *hptr, *pptr;
     int family, socktype, protocol, flags;
     int error;
     PyObject *all = (PyObject *)NULL;
diff --git a/Modules/syslogmodule.c b/Modules/syslogmodule.c
--- a/Modules/syslogmodule.c
+++ b/Modules/syslogmodule.c
@@ -116,7 +116,7 @@
     long facility = LOG_USER;
     PyObject *new_S_ident_o = NULL;
     static char *keywords[] = {"ident", "logoption", "facility", 0};
-    char *ident = NULL;
+    const char *ident = NULL;
 
     if (!PyArg_ParseTupleAndKeywords(args, kwds,
                           "|Ull:openlog", keywords, &new_S_ident_o, &logopt, &facility))
diff --git a/Objects/fileobject.c b/Objects/fileobject.c
--- a/Objects/fileobject.c
+++ b/Objects/fileobject.c
@@ -367,7 +367,7 @@
 {
     PyObject *unicode;
     PyObject *bytes = NULL;
-    char *str;
+    const char *str;
     Py_ssize_t n;
     int err;
 
@@ -389,10 +389,8 @@
         bytes = _PyUnicode_AsUTF8String(unicode, "backslashreplace");
         if (bytes == NULL)
             return NULL;
-        if (PyBytes_AsStringAndSize(bytes, &str, &n) < 0) {
-            Py_DECREF(bytes);
-            return NULL;
-        }
+        str = PyBytes_AS_STRING(bytes);
+        n = PyBytes_GET_SIZE(bytes);
     }
 
     n = _Py_write(self->fd, str, n);
diff --git a/Objects/floatobject.c b/Objects/floatobject.c
--- a/Objects/floatobject.c
+++ b/Objects/floatobject.c
@@ -1225,7 +1225,7 @@
     PyObject *result;
     double x;
     long exp, top_exp, lsb, key_digit;
-    char *s, *coeff_start, *s_store, *coeff_end, *exp_start, *s_end;
+    const char *s, *coeff_start, *s_store, *coeff_end, *exp_start, *s_end;
     int half_eps, digit, round_up, negate=0;
     Py_ssize_t length, ndigits, fdigits, i;
 
@@ -1288,7 +1288,7 @@
         s++;
 
     /* infinities and nans */
-    x = _Py_parse_inf_or_nan(s, &coeff_end);
+    x = _Py_parse_inf_or_nan(s, (char **)&coeff_end);
     if (coeff_end != s) {
         s = coeff_end;
         goto finished;
@@ -1619,7 +1619,7 @@
 static PyObject *
 float_getformat(PyTypeObject *v, PyObject* arg)
 {
-    char* s;
+    const char *s;
     float_format_type r;
 
     if (!PyUnicode_Check(arg)) {
diff --git a/Objects/longobject.c b/Objects/longobject.c
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -2491,7 +2491,8 @@
 PyLong_FromUnicodeObject(PyObject *u, int base)
 {
     PyObject *result, *asciidig;
-    char *buffer, *end = NULL;
+    const char *buffer;
+    char *end = NULL;
     Py_ssize_t buflen;
 
     asciidig = _PyUnicode_TransformDecimalAndSpaceToASCII(u);
diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c
--- a/Objects/moduleobject.c
+++ b/Objects/moduleobject.c
@@ -231,7 +231,7 @@
     PyObject *nameobj;
     PyObject *m = NULL;
     int has_execution_slots = 0;
-    char *name;
+    const char *name;
     int ret;
 
     PyModuleDef_Init(def);
@@ -512,7 +512,7 @@
 PyModule_GetFilename(PyObject *m)
 {
     PyObject *fileobj;
-    char *utf8;
+    const char *utf8;
     fileobj = PyModule_GetFilenameObject(m);
     if (fileobj == NULL)
         return NULL;
diff --git a/Objects/setobject.c b/Objects/setobject.c
--- a/Objects/setobject.c
+++ b/Objects/setobject.c
@@ -2398,7 +2398,7 @@
 test_c_api(PySetObject *so)
 {
     Py_ssize_t count;
-    char *s;
+    const char *s;
     Py_ssize_t i;
     PyObject *elem=NULL, *dup=NULL, *t, *f, *dup2, *x=NULL;
     PyObject *ob = (PyObject *)so;
diff --git a/Objects/structseq.c b/Objects/structseq.c
--- a/Objects/structseq.c
+++ b/Objects/structseq.c
@@ -182,7 +182,7 @@
 
     for (i=0; i < VISIBLE_SIZE(obj); i++) {
         PyObject *val, *repr;
-        char *cname, *crepr;
+        const char *cname, *crepr;
 
         cname = typ->tp_members[i].name;
         if (cname == NULL) {
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -1624,7 +1624,7 @@
     i = 0;
     while (PyDict_Next(set, &i, &k, &v) && (size_t)off < sizeof(buf)) {
         PyObject *name = class_name(k);
-        char *name_str;
+        const char *name_str;
         if (name != NULL) {
             name_str = PyUnicode_AsUTF8(name);
             if (name_str == NULL)
@@ -2572,7 +2572,7 @@
         PyObject *doc = _PyDict_GetItemId(dict, &PyId___doc__);
         if (doc != NULL && PyUnicode_Check(doc)) {
             Py_ssize_t len;
-            char *doc_str;
+            const char *doc_str;
             char *tp_doc;
 
             doc_str = PyUnicode_AsUTF8(doc);
diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c
--- a/Python/bltinmodule.c
+++ b/Python/bltinmodule.c
@@ -1893,7 +1893,7 @@
         char *s = NULL;
         PyObject *stdin_encoding = NULL, *stdin_errors = NULL;
         PyObject *stdout_encoding = NULL, *stdout_errors = NULL;
-        char *stdin_encoding_str, *stdin_errors_str;
+        const char *stdin_encoding_str, *stdin_errors_str;
         PyObject *result;
         size_t len;
 
@@ -1914,7 +1914,7 @@
             Py_DECREF(tmp);
         if (prompt != NULL) {
             /* We have a prompt, encode it as stdout would */
-            char *stdout_encoding_str, *stdout_errors_str;
+            const char *stdout_encoding_str, *stdout_errors_str;
             PyObject *stringpo;
             stdout_encoding = _PyObject_GetAttrId(fout, &PyId_encoding);
             stdout_errors = _PyObject_GetAttrId(fout, &PyId_errors);
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -5440,8 +5440,8 @@
 static void
 dtrace_function_entry(PyFrameObject *f)
 {
-    char* filename;
-    char* funcname;
+    const char *filename;
+    const char *funcname;
     int lineno;
 
     filename = PyUnicode_AsUTF8(f->f_code->co_filename);
@@ -5454,8 +5454,8 @@
 static void
 dtrace_function_return(PyFrameObject *f)
 {
-    char* filename;
-    char* funcname;
+    const char *filename;
+    const char *funcname;
     int lineno;
 
     filename = PyUnicode_AsUTF8(f->f_code->co_filename);
@@ -5471,7 +5471,7 @@
                   int *instr_lb, int *instr_ub, int *instr_prev)
 {
     int line = frame->f_lineno;
-    char *co_filename, *co_name;
+    const char *co_filename, *co_name;
 
     /* If the last instruction executed isn't in the current
        instruction window, reset the window.
diff --git a/Python/codecs.c b/Python/codecs.c
--- a/Python/codecs.c
+++ b/Python/codecs.c
@@ -1131,7 +1131,7 @@
     PyObject *restuple;
     PyObject *object;
     PyObject *encode;
-    char *encoding;
+    const char *encoding;
     int code;
     int bytelength;
     Py_ssize_t i;
diff --git a/Python/compile.c b/Python/compile.c
--- a/Python/compile.c
+++ b/Python/compile.c
@@ -4039,7 +4039,7 @@
 static int
 expr_constant(struct compiler *c, expr_ty e)
 {
-    char *id;
+    const char *id;
     switch (e->kind) {
     case Ellipsis_kind:
         return 1;
diff --git a/Python/getargs.c b/Python/getargs.c
--- a/Python/getargs.c
+++ b/Python/getargs.c
@@ -74,7 +74,7 @@
                                 int *, char *, size_t, int, freelist_t *);
 static const char *convertsimple(PyObject *, const char **, va_list *, int,
                                  char *, size_t, freelist_t *);
-static Py_ssize_t convertbuffer(PyObject *, void **p, const char **);
+static Py_ssize_t convertbuffer(PyObject *, const void **p, const char **);
 static int getbuffer(PyObject *, Py_buffer *, const char**);
 
 static int vgetargskeywords(PyObject *, PyObject *,
@@ -625,7 +625,7 @@
 
     const char *format = *p_format;
     char c = *format++;
-    char *sarg;
+    const char *sarg;
 
     switch (c) {
 
@@ -897,7 +897,7 @@
             }
             break;
         }
-        count = convertbuffer(arg, p, &buf);
+        count = convertbuffer(arg, (const void **)p, &buf);
         if (count < 0)
             return converterr(buf, arg, msgbuf, bufsize);
         if (*format == '#') {
@@ -928,7 +928,7 @@
                 if (sarg == NULL)
                     return converterr(CONV_UNICODE,
                                       arg, msgbuf, bufsize);
-                PyBuffer_FillInfo(p, arg, sarg, len, 1, 0);
+                PyBuffer_FillInfo(p, arg, (void *)sarg, len, 1, 0);
             }
             else { /* any bytes-like object */
                 const char *buf;
@@ -943,7 +943,7 @@
             format++;
         } else if (*format == '#') { /* a string or read-only bytes-like object */
             /* "s#" or "z#" */
-            void **p = (void **)va_arg(*p_va, char **);
+            const void **p = (const void **)va_arg(*p_va, const char **);
             FETCH_SIZE;
 
             if (c == 'z' && arg == Py_None) {
@@ -970,7 +970,7 @@
             format++;
         } else {
             /* "s" or "z" */
-            char **p = va_arg(*p_va, char **);
+            const char **p = va_arg(*p_va, const char **);
             Py_ssize_t len;
             sarg = NULL;
 
@@ -1300,7 +1300,7 @@
 }
 
 static Py_ssize_t
-convertbuffer(PyObject *arg, void **p, const char **errmsg)
+convertbuffer(PyObject *arg, const void **p, const char **errmsg)
 {
     PyBufferProcs *pb = Py_TYPE(arg)->tp_as_buffer;
     Py_ssize_t count;
diff --git a/Python/import.c b/Python/import.c
--- a/Python/import.c
+++ b/Python/import.c
@@ -1035,7 +1035,7 @@
 {
     struct _inittab *p;
     PyObject *name;
-    char *namestr;
+    const char *namestr;
     PyObject *mod;
 
     name = PyObject_GetAttrString(spec, "name");
diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c
--- a/Python/pylifecycle.c
+++ b/Python/pylifecycle.c
@@ -193,7 +193,8 @@
 static char*
 get_codec_name(const char *encoding)
 {
-    char *name_utf8, *name_str;
+    const char *name_utf8;
+    char *name_str;
     PyObject *codec, *name = NULL;
 
     codec = _PyCodec_Lookup(encoding);
@@ -1284,8 +1285,7 @@
        when import.c tries to write to stderr in verbose mode. */
     encoding_attr = PyObject_GetAttrString(std, "encoding");
     if (encoding_attr != NULL) {
-        const char * std_encoding;
-        std_encoding = PyUnicode_AsUTF8(encoding_attr);
+        const char *std_encoding = PyUnicode_AsUTF8(encoding_attr);
         if (std_encoding != NULL) {
             PyObject *codec_info = _PyCodec_Lookup(std_encoding);
             Py_XDECREF(codec_info);
diff --git a/Python/pythonrun.c b/Python/pythonrun.c
--- a/Python/pythonrun.c
+++ b/Python/pythonrun.c
@@ -154,7 +154,7 @@
     PyObject *m, *d, *v, *w, *oenc = NULL, *mod_name;
     mod_ty mod;
     PyArena *arena;
-    char *ps1 = "", *ps2 = "", *enc = NULL;
+    const char *ps1 = "", *ps2 = "", *enc = NULL;
     int errcode = 0;
     _Py_IDENTIFIER(encoding);
     _Py_IDENTIFIER(__main__);
@@ -511,8 +511,8 @@
 static void
 print_error_text(PyObject *f, int offset, PyObject *text_obj)
 {
-    char *text;
-    char *nl;
+    const char *text;
+    const char *nl;
 
     text = PyUnicode_AsUTF8(text_obj);
     if (text == NULL)
diff --git a/Python/structmember.c b/Python/structmember.c
--- a/Python/structmember.c
+++ b/Python/structmember.c
@@ -249,7 +249,7 @@
         Py_XDECREF(oldv);
         break;
     case T_CHAR: {
-        char *string;
+        const char *string;
         Py_ssize_t len;
 
         string = PyUnicode_AsUTF8AndSize(v, &len);
diff --git a/Python/sysmodule.c b/Python/sysmodule.c
--- a/Python/sysmodule.c
+++ b/Python/sysmodule.c
@@ -108,7 +108,7 @@
 {
     PyObject *stdout_encoding = NULL;
     PyObject *encoded, *escaped_str, *repr_str, *buffer, *result;
-    char *stdout_encoding_str;
+    const char *stdout_encoding_str;
     int ret;
 
     stdout_encoding = _PyObject_GetAttrId(outf, &PyId_encoding);
@@ -2404,7 +2404,7 @@
 {
     PyObject *file, *message;
     PyObject *error_type, *error_value, *error_traceback;
-    char *utf8;
+    const char *utf8;
 
     PyErr_Fetch(&error_type, &error_value, &error_traceback);
     file = _PySys_GetObjectId(key);

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Sun Nov 20 04:06:01 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sun, 20 Nov 2016 09:06:01 +0000
Subject: [Python-checkins] Daily reference leaks (08ca8d3ec707): sum=4
Message-ID: <20161120090601.8131.31668.6567F862@psf.io>

results for 08ca8d3ec707 on branch "default"
--------------------------------------------

test_collections leaked [0, 7, -7] memory blocks, sum=0
test_functools leaked [0, 3, 1] memory blocks, sum=4


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogiPpknl', '--timeout', '7200']

From python-checkins at python.org  Sun Nov 20 05:17:04 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 10:17:04 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzE5NTY5?=
 =?utf-8?q?=3A_Suggested_more_appropriate_replacements_for_deprecated_Unic?=
 =?utf-8?q?ode?=
Message-ID: <20161120101704.40975.14797.9DE08B47@psf.io>

https://hg.python.org/cpython/rev/75fe67538905
changeset:   105211:75fe67538905
branch:      3.5
parent:      105205:38f321a6be41
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 12:13:44 2016 +0200
summary:
  Issue #19569: Suggested more appropriate replacements for deprecated Unicode
C API functions.

files:
  Doc/c-api/unicode.rst |  39 +++++++++++++++++-------------
  1 files changed, 22 insertions(+), 17 deletions(-)


diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -685,8 +685,8 @@
    string content has been filled before using any of the access macros such as
    :c:func:`PyUnicode_KIND`.
 
-   Please migrate to using :c:func:`PyUnicode_FromKindAndData` or
-   :c:func:`PyUnicode_New`.
+   Please migrate to using :c:func:`PyUnicode_FromKindAndData`,
+   :c:func:`PyUnicode_FromWideChar` or :c:func:`PyUnicode_New`.
 
 
 .. c:function:: Py_UNICODE* PyUnicode_AsUnicode(PyObject *unicode)
@@ -700,7 +700,7 @@
    used in most C functions.
 
    Please migrate to using :c:func:`PyUnicode_AsUCS4`,
-   :c:func:`PyUnicode_Substring`, :c:func:`PyUnicode_ReadChar` or similar new
+   :c:func:`PyUnicode_AsWideChar`, :c:func:`PyUnicode_ReadChar` or similar new
    APIs.
 
 
@@ -1086,7 +1086,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsUTF8String` or :c:func:`PyUnicode_AsUTF8AndSize`.
+      :c:func:`PyUnicode_AsUTF8String`, :c:func:`PyUnicode_AsUTF8AndSize` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 UTF-32 Codecs
@@ -1159,7 +1160,7 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsUTF32String`.
+      :c:func:`PyUnicode_AsUTF32String` or :c:func:`PyUnicode_AsEncodedString`.
 
 
 UTF-16 Codecs
@@ -1234,7 +1235,7 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsUTF16String`.
+      :c:func:`PyUnicode_AsUTF16String` or :c:func:`PyUnicode_AsEncodedString`.
 
 
 UTF-7 Codecs
@@ -1271,9 +1272,8 @@
    Python "utf-7" codec.
 
    .. deprecated-removed:: 3.3 4.0
-      Part of the old-style :c:type:`Py_UNICODE` API.
-
-   .. XXX replace with what?
+      Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 Unicode-Escape Codecs
@@ -1336,7 +1336,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsRawUnicodeEscapeString`.
+      :c:func:`PyUnicode_AsRawUnicodeEscapeString` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 Latin-1 Codecs
@@ -1367,7 +1368,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsLatin1String`.
+      :c:func:`PyUnicode_AsLatin1String` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 ASCII Codecs
@@ -1398,7 +1400,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsASCIIString`.
+      :c:func:`PyUnicode_AsASCIIString` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 Character Map Codecs
@@ -1462,9 +1465,9 @@
    :exc:`LookupError`) are left untouched and are copied as-is.
 
    .. deprecated-removed:: 3.3 4.0
-      Part of the old-style :c:type:`Py_UNICODE` API.
-
-   .. XXX replace with what?
+      Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
+      :c:func:`PyUnicode_Translate`. or :ref:`generic codec based API
+      <codec-registry>`
 
 
 .. c:function:: PyObject* PyUnicode_EncodeCharmap(const Py_UNICODE *s, Py_ssize_t size, \
@@ -1476,7 +1479,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsCharmapString`.
+      :c:func:`PyUnicode_AsCharmapString` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 MBCS codecs for Windows
@@ -1526,7 +1530,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsMBCSString` or :c:func:`PyUnicode_EncodeCodePage`.
+      :c:func:`PyUnicode_AsMBCSString`, :c:func:`PyUnicode_EncodeCodePage` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 Methods & Slots

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 05:17:04 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 10:17:04 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2319569=3A_Suggested_more_appropriate_replacement?=
 =?utf-8?q?s_for_deprecated_Unicode?=
Message-ID: <20161120101704.7287.71151.820D1172@psf.io>

https://hg.python.org/cpython/rev/2ae992bc2def
changeset:   105213:2ae992bc2def
parent:      105210:6e5404da46b1
parent:      105212:f26d3f9a958a
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 12:14:46 2016 +0200
summary:
  Issue #19569: Suggested more appropriate replacements for deprecated Unicode
C API functions.

files:
  Doc/c-api/unicode.rst |  39 +++++++++++++++++-------------
  1 files changed, 22 insertions(+), 17 deletions(-)


diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -679,8 +679,8 @@
    string content has been filled before using any of the access macros such as
    :c:func:`PyUnicode_KIND`.
 
-   Please migrate to using :c:func:`PyUnicode_FromKindAndData` or
-   :c:func:`PyUnicode_New`.
+   Please migrate to using :c:func:`PyUnicode_FromKindAndData`,
+   :c:func:`PyUnicode_FromWideChar` or :c:func:`PyUnicode_New`.
 
 
 .. c:function:: Py_UNICODE* PyUnicode_AsUnicode(PyObject *unicode)
@@ -694,7 +694,7 @@
    used in most C functions.
 
    Please migrate to using :c:func:`PyUnicode_AsUCS4`,
-   :c:func:`PyUnicode_Substring`, :c:func:`PyUnicode_ReadChar` or similar new
+   :c:func:`PyUnicode_AsWideChar`, :c:func:`PyUnicode_ReadChar` or similar new
    APIs.
 
 
@@ -1091,7 +1091,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsUTF8String` or :c:func:`PyUnicode_AsUTF8AndSize`.
+      :c:func:`PyUnicode_AsUTF8String`, :c:func:`PyUnicode_AsUTF8AndSize` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 UTF-32 Codecs
@@ -1164,7 +1165,7 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsUTF32String`.
+      :c:func:`PyUnicode_AsUTF32String` or :c:func:`PyUnicode_AsEncodedString`.
 
 
 UTF-16 Codecs
@@ -1239,7 +1240,7 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsUTF16String`.
+      :c:func:`PyUnicode_AsUTF16String` or :c:func:`PyUnicode_AsEncodedString`.
 
 
 UTF-7 Codecs
@@ -1276,9 +1277,8 @@
    Python "utf-7" codec.
 
    .. deprecated-removed:: 3.3 4.0
-      Part of the old-style :c:type:`Py_UNICODE` API.
-
-   .. XXX replace with what?
+      Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 Unicode-Escape Codecs
@@ -1341,7 +1341,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsRawUnicodeEscapeString`.
+      :c:func:`PyUnicode_AsRawUnicodeEscapeString` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 Latin-1 Codecs
@@ -1372,7 +1373,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsLatin1String`.
+      :c:func:`PyUnicode_AsLatin1String` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 ASCII Codecs
@@ -1403,7 +1405,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsASCIIString`.
+      :c:func:`PyUnicode_AsASCIIString` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 Character Map Codecs
@@ -1467,9 +1470,9 @@
    :exc:`LookupError`) are left untouched and are copied as-is.
 
    .. deprecated-removed:: 3.3 4.0
-      Part of the old-style :c:type:`Py_UNICODE` API.
-
-   .. XXX replace with what?
+      Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
+      :c:func:`PyUnicode_Translate`. or :ref:`generic codec based API
+      <codec-registry>`
 
 
 .. c:function:: PyObject* PyUnicode_EncodeCharmap(const Py_UNICODE *s, Py_ssize_t size, \
@@ -1481,7 +1484,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsCharmapString`.
+      :c:func:`PyUnicode_AsCharmapString` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 MBCS codecs for Windows
@@ -1531,7 +1535,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsMBCSString` or :c:func:`PyUnicode_EncodeCodePage`.
+      :c:func:`PyUnicode_AsMBCSString`, :c:func:`PyUnicode_EncodeCodePage` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 Methods & Slots

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 05:17:07 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 10:17:07 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2319569=3A_Suggested_more_appropriate_replacements_for_?=
 =?utf-8?q?deprecated_Unicode?=
Message-ID: <20161120101704.5220.39438.984873DC@psf.io>

https://hg.python.org/cpython/rev/f26d3f9a958a
changeset:   105212:f26d3f9a958a
branch:      3.6
parent:      105208:39f0e0cf49fd
parent:      105211:75fe67538905
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 12:14:08 2016 +0200
summary:
  Issue #19569: Suggested more appropriate replacements for deprecated Unicode
C API functions.

files:
  Doc/c-api/unicode.rst |  39 +++++++++++++++++-------------
  1 files changed, 22 insertions(+), 17 deletions(-)


diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -679,8 +679,8 @@
    string content has been filled before using any of the access macros such as
    :c:func:`PyUnicode_KIND`.
 
-   Please migrate to using :c:func:`PyUnicode_FromKindAndData` or
-   :c:func:`PyUnicode_New`.
+   Please migrate to using :c:func:`PyUnicode_FromKindAndData`,
+   :c:func:`PyUnicode_FromWideChar` or :c:func:`PyUnicode_New`.
 
 
 .. c:function:: Py_UNICODE* PyUnicode_AsUnicode(PyObject *unicode)
@@ -694,7 +694,7 @@
    used in most C functions.
 
    Please migrate to using :c:func:`PyUnicode_AsUCS4`,
-   :c:func:`PyUnicode_Substring`, :c:func:`PyUnicode_ReadChar` or similar new
+   :c:func:`PyUnicode_AsWideChar`, :c:func:`PyUnicode_ReadChar` or similar new
    APIs.
 
 
@@ -1091,7 +1091,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsUTF8String` or :c:func:`PyUnicode_AsUTF8AndSize`.
+      :c:func:`PyUnicode_AsUTF8String`, :c:func:`PyUnicode_AsUTF8AndSize` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 UTF-32 Codecs
@@ -1164,7 +1165,7 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsUTF32String`.
+      :c:func:`PyUnicode_AsUTF32String` or :c:func:`PyUnicode_AsEncodedString`.
 
 
 UTF-16 Codecs
@@ -1239,7 +1240,7 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsUTF16String`.
+      :c:func:`PyUnicode_AsUTF16String` or :c:func:`PyUnicode_AsEncodedString`.
 
 
 UTF-7 Codecs
@@ -1276,9 +1277,8 @@
    Python "utf-7" codec.
 
    .. deprecated-removed:: 3.3 4.0
-      Part of the old-style :c:type:`Py_UNICODE` API.
-
-   .. XXX replace with what?
+      Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 Unicode-Escape Codecs
@@ -1341,7 +1341,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsRawUnicodeEscapeString`.
+      :c:func:`PyUnicode_AsRawUnicodeEscapeString` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 Latin-1 Codecs
@@ -1372,7 +1373,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsLatin1String`.
+      :c:func:`PyUnicode_AsLatin1String` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 ASCII Codecs
@@ -1403,7 +1405,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsASCIIString`.
+      :c:func:`PyUnicode_AsASCIIString` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 Character Map Codecs
@@ -1467,9 +1470,9 @@
    :exc:`LookupError`) are left untouched and are copied as-is.
 
    .. deprecated-removed:: 3.3 4.0
-      Part of the old-style :c:type:`Py_UNICODE` API.
-
-   .. XXX replace with what?
+      Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
+      :c:func:`PyUnicode_Translate`. or :ref:`generic codec based API
+      <codec-registry>`
 
 
 .. c:function:: PyObject* PyUnicode_EncodeCharmap(const Py_UNICODE *s, Py_ssize_t size, \
@@ -1481,7 +1484,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsCharmapString`.
+      :c:func:`PyUnicode_AsCharmapString` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 MBCS codecs for Windows
@@ -1531,7 +1535,8 @@
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
-      :c:func:`PyUnicode_AsMBCSString` or :c:func:`PyUnicode_EncodeCodePage`.
+      :c:func:`PyUnicode_AsMBCSString`, :c:func:`PyUnicode_EncodeCodePage` or
+      :c:func:`PyUnicode_AsEncodedString`.
 
 
 Methods & Slots

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 05:17:07 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 10:17:07 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2319569=3A_Compiler?=
 =?utf-8?q?_warnings_are_now_emitted_if_use_most_of_deprecated?=
Message-ID: <20161120101704.12774.234.05AAD718@psf.io>

https://hg.python.org/cpython/rev/f692dafe6797
changeset:   105214:f692dafe6797
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 12:16:46 2016 +0200
summary:
  Issue #19569: Compiler warnings are now emitted if use most of deprecated
functions.

files:
  Include/abstract.h        |  12 ++-
  Include/ceval.h           |   4 +-
  Include/longobject.h      |   2 +-
  Include/moduleobject.h    |   2 +-
  Include/pyerrors.h        |  10 +-
  Include/unicodeobject.h   |  68 ++++++++++---------
  Misc/NEWS                 |   3 +
  Modules/_testcapimodule.c |  12 +--
  Modules/arraymodule.c     |   4 +-
  Objects/abstract.c        |  28 ++++---
  Objects/bytearrayobject.c |   5 +-
  Objects/bytesobject.c     |   5 +-
  Objects/codeobject.c      |   5 +-
  Objects/longobject.c      |   2 +-
  Objects/unicodeobject.c   |  94 ++++++++++++--------------
  Python/errors.c           |  12 +--
  Python/getargs.c          |  13 ++-
  Python/modsupport.c       |   4 +-
  18 files changed, 143 insertions(+), 142 deletions(-)


diff --git a/Include/abstract.h b/Include/abstract.h
--- a/Include/abstract.h
+++ b/Include/abstract.h
@@ -549,7 +549,8 @@
 
      PyAPI_FUNC(int) PyObject_AsCharBuffer(PyObject *obj,
                                            const char **buffer,
-                                           Py_ssize_t *buffer_len);
+                                           Py_ssize_t *buffer_len)
+                                           Py_DEPRECATED(3.0);
 
        /*
       Takes an arbitrary object which must support the (character,
@@ -562,7 +563,8 @@
       an exception set.
        */
 
-     PyAPI_FUNC(int) PyObject_CheckReadBuffer(PyObject *obj);
+     PyAPI_FUNC(int) PyObject_CheckReadBuffer(PyObject *obj)
+                                              Py_DEPRECATED(3.0);
 
       /*
       Checks whether an arbitrary object supports the (character,
@@ -572,7 +574,8 @@
 
      PyAPI_FUNC(int) PyObject_AsReadBuffer(PyObject *obj,
                                            const void **buffer,
-                                           Py_ssize_t *buffer_len);
+                                           Py_ssize_t *buffer_len)
+                                           Py_DEPRECATED(3.0);
 
        /*
       Same as PyObject_AsCharBuffer() except that this API expects
@@ -587,7 +590,8 @@
 
      PyAPI_FUNC(int) PyObject_AsWriteBuffer(PyObject *obj,
                                             void **buffer,
-                                            Py_ssize_t *buffer_len);
+                                            Py_ssize_t *buffer_len)
+                                            Py_DEPRECATED(3.0);
 
        /*
       Takes an arbitrary object which must support the (writable,
diff --git a/Include/ceval.h b/Include/ceval.h
--- a/Include/ceval.h
+++ b/Include/ceval.h
@@ -182,8 +182,8 @@
 #ifndef Py_LIMITED_API
 PyAPI_FUNC(void) _PyEval_FiniThreads(void);
 #endif /* !Py_LIMITED_API */
-PyAPI_FUNC(void) PyEval_AcquireLock(void);
-PyAPI_FUNC(void) PyEval_ReleaseLock(void);
+PyAPI_FUNC(void) PyEval_AcquireLock(void) Py_DEPRECATED(3.2);
+PyAPI_FUNC(void) PyEval_ReleaseLock(void) /* Py_DEPRECATED(3.2) */;
 PyAPI_FUNC(void) PyEval_AcquireThread(PyThreadState *tstate);
 PyAPI_FUNC(void) PyEval_ReleaseThread(PyThreadState *tstate);
 PyAPI_FUNC(void) PyEval_ReInitThreads(void);
diff --git a/Include/longobject.h b/Include/longobject.h
--- a/Include/longobject.h
+++ b/Include/longobject.h
@@ -94,7 +94,7 @@
 
 PyAPI_FUNC(PyObject *) PyLong_FromString(const char *, char **, int);
 #ifndef Py_LIMITED_API
-PyAPI_FUNC(PyObject *) PyLong_FromUnicode(Py_UNICODE*, Py_ssize_t, int);
+PyAPI_FUNC(PyObject *) PyLong_FromUnicode(Py_UNICODE*, Py_ssize_t, int) Py_DEPRECATED(3.3);
 PyAPI_FUNC(PyObject *) PyLong_FromUnicodeObject(PyObject *u, int base);
 PyAPI_FUNC(PyObject *) _PyLong_FromBytes(const char *, Py_ssize_t, int);
 #endif
diff --git a/Include/moduleobject.h b/Include/moduleobject.h
--- a/Include/moduleobject.h
+++ b/Include/moduleobject.h
@@ -21,7 +21,7 @@
 PyAPI_FUNC(PyObject *) PyModule_GetDict(PyObject *);
 PyAPI_FUNC(PyObject *) PyModule_GetNameObject(PyObject *);
 PyAPI_FUNC(const char *) PyModule_GetName(PyObject *);
-PyAPI_FUNC(const char *) PyModule_GetFilename(PyObject *);
+PyAPI_FUNC(const char *) PyModule_GetFilename(PyObject *) Py_DEPRECATED(3.2);
 PyAPI_FUNC(PyObject *) PyModule_GetFilenameObject(PyObject *);
 #ifndef Py_LIMITED_API
 PyAPI_FUNC(void) _PyModule_Clear(PyObject *);
diff --git a/Include/pyerrors.h b/Include/pyerrors.h
--- a/Include/pyerrors.h
+++ b/Include/pyerrors.h
@@ -240,7 +240,7 @@
     );
 #if defined(MS_WINDOWS) && !defined(Py_LIMITED_API)
 PyAPI_FUNC(PyObject *) PyErr_SetFromErrnoWithUnicodeFilename(
-    PyObject *, const Py_UNICODE *);
+    PyObject *, const Py_UNICODE *) Py_DEPRECATED(3.3);
 #endif /* MS_WINDOWS */
 
 PyAPI_FUNC(PyObject *) PyErr_Format(
@@ -274,7 +274,7 @@
 #ifndef Py_LIMITED_API
 /* XXX redeclare to use WSTRING */
 PyAPI_FUNC(PyObject *) PyErr_SetFromWindowsErrWithUnicodeFilename(
-    int, const Py_UNICODE *);
+    int, const Py_UNICODE *) Py_DEPRECATED(3.3);
 #endif
 PyAPI_FUNC(PyObject *) PyErr_SetFromWindowsErr(int);
 PyAPI_FUNC(PyObject *) PyErr_SetExcFromWindowsErrWithFilenameObject(
@@ -288,7 +288,7 @@
     );
 #ifndef Py_LIMITED_API
 PyAPI_FUNC(PyObject *) PyErr_SetExcFromWindowsErrWithUnicodeFilename(
-    PyObject *,int, const Py_UNICODE *);
+    PyObject *,int, const Py_UNICODE *) Py_DEPRECATED(3.3);
 #endif
 PyAPI_FUNC(PyObject *) PyErr_SetExcFromWindowsErr(PyObject *, int);
 #endif /* MS_WINDOWS */
@@ -391,7 +391,7 @@
     Py_ssize_t start,
     Py_ssize_t end,
     const char *reason          /* UTF-8 encoded string */
-    );
+    ) Py_DEPRECATED(3.3);
 #endif
 
 /* create a UnicodeTranslateError object */
@@ -402,7 +402,7 @@
     Py_ssize_t start,
     Py_ssize_t end,
     const char *reason          /* UTF-8 encoded string */
-    );
+    ) Py_DEPRECATED(3.3);
 PyAPI_FUNC(PyObject *) _PyUnicodeTranslateError_Create(
     PyObject *object,
     Py_ssize_t start,
diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -90,7 +90,7 @@
 
 #ifndef Py_LIMITED_API
 #define PY_UNICODE_TYPE wchar_t
-typedef wchar_t Py_UNICODE;
+typedef wchar_t Py_UNICODE /* Py_DEPRECATED(3.3) */;
 #endif
 
 /* If the compiler provides a wchar_t type we try to support it
@@ -387,9 +387,11 @@
       ((void)PyUnicode_AsUnicode((PyObject *)(op)),  \
        assert(((PyASCIIObject *)(op))->wstr),        \
        PyUnicode_WSTR_LENGTH(op)))
+    /* Py_DEPRECATED(3.3) */
 
 #define PyUnicode_GET_DATA_SIZE(op) \
     (PyUnicode_GET_SIZE(op) * Py_UNICODE_SIZE)
+    /* Py_DEPRECATED(3.3) */
 
 /* Alias for PyUnicode_AsUnicode().  This will create a wchar_t/Py_UNICODE
    representation on demand.  Using this macro is very inefficient now,
@@ -400,9 +402,11 @@
     (assert(PyUnicode_Check(op)), \
      (((PyASCIIObject *)(op))->wstr) ? (((PyASCIIObject *)(op))->wstr) : \
       PyUnicode_AsUnicode((PyObject *)(op)))
+    /* Py_DEPRECATED(3.3) */
 
 #define PyUnicode_AS_DATA(op) \
     ((const char *)(PyUnicode_AS_UNICODE(op)))
+    /* Py_DEPRECATED(3.3) */
 
 
 /* --- Flexible String Representation Helper Macros (PEP 393) -------------- */
@@ -688,7 +692,7 @@
 PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
     const Py_UNICODE *u,        /* Unicode buffer */
     Py_ssize_t size             /* size of buffer */
-    );
+    ) /* Py_DEPRECATED(3.3) */;
 #endif
 
 /* Similar to PyUnicode_FromUnicode(), but u points to UTF-8 encoded bytes */
@@ -756,7 +760,7 @@
 #ifndef Py_LIMITED_API
 PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
     PyObject *unicode           /* Unicode object */
-    );
+    ) /* Py_DEPRECATED(3.3) */;
 #endif
 
 /* Return a read-only pointer to the Unicode object's internal
@@ -768,7 +772,7 @@
 PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicodeAndSize(
     PyObject *unicode,          /* Unicode object */
     Py_ssize_t *size            /* location where to save the length */
-    );
+    )  /* Py_DEPRECATED(3.3) */;
 #endif
 
 /* Get the length of the Unicode object. */
@@ -782,7 +786,7 @@
 
 PyAPI_FUNC(Py_ssize_t) PyUnicode_GetSize(
     PyObject *unicode           /* Unicode object */
-    );
+    ) Py_DEPRECATED(3.3);
 
 /* Read a character from the string. */
 
@@ -804,7 +808,7 @@
 
 #ifndef Py_LIMITED_API
 /* Get the maximum ordinal for a Unicode character. */
-PyAPI_FUNC(Py_UNICODE) PyUnicode_GetMax(void);
+PyAPI_FUNC(Py_UNICODE) PyUnicode_GetMax(void) Py_DEPRECATED(3.3);
 #endif
 
 /* Resize a Unicode object. The length is the number of characters, except
@@ -1205,7 +1209,7 @@
     Py_ssize_t size,            /* number of Py_UNICODE chars to encode */
     const char *encoding,       /* encoding */
     const char *errors          /* error handling */
-    );
+    ) Py_DEPRECATED(3.3);
 #endif
 
 /* Encodes a Unicode object and returns the result as Python
@@ -1272,7 +1276,7 @@
     int base64SetO,             /* Encode RFC2152 Set O characters in base64 */
     int base64WhiteSpace,       /* Encode whitespace (sp, ht, nl, cr) in base64 */
     const char *errors          /* error handling */
-    );
+    ) Py_DEPRECATED(3.3);
 PyAPI_FUNC(PyObject*) _PyUnicode_EncodeUTF7(
     PyObject *unicode,          /* Unicode object */
     int base64SetO,             /* Encode RFC2152 Set O characters in base64 */
@@ -1309,7 +1313,7 @@
     const Py_UNICODE *data,     /* Unicode char buffer */
     Py_ssize_t length,          /* number of Py_UNICODE chars to encode */
     const char *errors          /* error handling */
-    );
+    ) Py_DEPRECATED(3.3);
 #endif
 
 /* --- UTF-32 Codecs ------------------------------------------------------ */
@@ -1385,7 +1389,7 @@
     Py_ssize_t length,          /* number of Py_UNICODE chars to encode */
     const char *errors,         /* error handling */
     int byteorder               /* byteorder to use 0=BOM+native;-1=LE,1=BE */
-    );
+    ) Py_DEPRECATED(3.3);
 PyAPI_FUNC(PyObject*) _PyUnicode_EncodeUTF32(
     PyObject *object,           /* Unicode object */
     const char *errors,         /* error handling */
@@ -1470,7 +1474,7 @@
     Py_ssize_t length,          /* number of Py_UNICODE chars to encode */
     const char *errors,         /* error handling */
     int byteorder               /* byteorder to use 0=BOM+native;-1=LE,1=BE */
-    );
+    ) Py_DEPRECATED(3.3);
 PyAPI_FUNC(PyObject*) _PyUnicode_EncodeUTF16(
     PyObject* unicode,          /* Unicode object */
     const char *errors,         /* error handling */
@@ -1505,7 +1509,7 @@
 PyAPI_FUNC(PyObject*) PyUnicode_EncodeUnicodeEscape(
     const Py_UNICODE *data,     /* Unicode char buffer */
     Py_ssize_t length           /* Number of Py_UNICODE chars to encode */
-    );
+    ) Py_DEPRECATED(3.3);
 #endif
 
 /* --- Raw-Unicode-Escape Codecs ------------------------------------------ */
@@ -1524,7 +1528,7 @@
 PyAPI_FUNC(PyObject*) PyUnicode_EncodeRawUnicodeEscape(
     const Py_UNICODE *data,     /* Unicode char buffer */
     Py_ssize_t length           /* Number of Py_UNICODE chars to encode */
-    );
+    ) Py_DEPRECATED(3.3);
 #endif
 
 /* --- Unicode Internal Codec ---------------------------------------------
@@ -1564,7 +1568,7 @@
     const Py_UNICODE *data,     /* Unicode char buffer */
     Py_ssize_t length,          /* Number of Py_UNICODE chars to encode */
     const char *errors          /* error handling */
-    );
+    ) Py_DEPRECATED(3.3);
 #endif
 
 /* --- ASCII Codecs -------------------------------------------------------
@@ -1592,7 +1596,7 @@
     const Py_UNICODE *data,     /* Unicode char buffer */
     Py_ssize_t length,          /* Number of Py_UNICODE chars to encode */
     const char *errors          /* error handling */
-    );
+    ) Py_DEPRECATED(3.3);
 #endif
 
 /* --- Character Map Codecs -----------------------------------------------
@@ -1638,7 +1642,7 @@
     PyObject *mapping,          /* character mapping
                                    (unicode ordinal -> char ordinal) */
     const char *errors          /* error handling */
-    );
+    ) Py_DEPRECATED(3.3);
 PyAPI_FUNC(PyObject*) _PyUnicode_EncodeCharmap(
     PyObject *unicode,          /* Unicode object */
     PyObject *mapping,          /* character mapping
@@ -1666,7 +1670,7 @@
     Py_ssize_t length,          /* Number of Py_UNICODE chars to encode */
     PyObject *table,            /* Translate table */
     const char *errors          /* error handling */
-    );
+    ) Py_DEPRECATED(3.3);
 #endif
 
 #ifdef MS_WINDOWS
@@ -1703,7 +1707,7 @@
     const Py_UNICODE *data,     /* Unicode char buffer */
     Py_ssize_t length,          /* number of Py_UNICODE chars to encode */
     const char *errors          /* error handling */
-    );
+    ) Py_DEPRECATED(3.3);
 #endif
 
 PyAPI_FUNC(PyObject*) PyUnicode_EncodeCodePage(
@@ -1744,7 +1748,7 @@
     Py_ssize_t length,          /* Number of Py_UNICODE chars to encode */
     char *output,               /* Output buffer; must have size >= length */
     const char *errors          /* error handling */
-    );
+    ) /* Py_DEPRECATED(3.3) */;
 #endif
 
 /* Transforms code points that have decimal digit property to the
@@ -1757,7 +1761,7 @@
 PyAPI_FUNC(PyObject*) PyUnicode_TransformDecimalToASCII(
     Py_UNICODE *s,              /* Unicode buffer */
     Py_ssize_t length           /* Number of Py_UNICODE chars to transform */
-    );
+    ) /* Py_DEPRECATED(3.3) */;
 #endif
 
 /* Similar to PyUnicode_TransformDecimalToASCII(), but takes a PyObject
@@ -2182,15 +2186,15 @@
 
 PyAPI_FUNC(Py_UCS4) _PyUnicode_ToLowercase(
     Py_UCS4 ch       /* Unicode character */
-    );
+    ) /* Py_DEPRECATED(3.3) */;
 
 PyAPI_FUNC(Py_UCS4) _PyUnicode_ToUppercase(
     Py_UCS4 ch       /* Unicode character */
-    );
+    ) /* Py_DEPRECATED(3.3) */;
 
 PyAPI_FUNC(Py_UCS4) _PyUnicode_ToTitlecase(
     Py_UCS4 ch       /* Unicode character */
-    );
+    ) Py_DEPRECATED(3.3);
 
 PyAPI_FUNC(int) _PyUnicode_ToLowerFull(
     Py_UCS4 ch,       /* Unicode character */
@@ -2254,40 +2258,40 @@
 
 PyAPI_FUNC(size_t) Py_UNICODE_strlen(
     const Py_UNICODE *u
-    );
+    ) Py_DEPRECATED(3.3);
 
 PyAPI_FUNC(Py_UNICODE*) Py_UNICODE_strcpy(
     Py_UNICODE *s1,
-    const Py_UNICODE *s2);
+    const Py_UNICODE *s2) Py_DEPRECATED(3.3);
 
 PyAPI_FUNC(Py_UNICODE*) Py_UNICODE_strcat(
-    Py_UNICODE *s1, const Py_UNICODE *s2);
+    Py_UNICODE *s1, const Py_UNICODE *s2) Py_DEPRECATED(3.3);
 
 PyAPI_FUNC(Py_UNICODE*) Py_UNICODE_strncpy(
     Py_UNICODE *s1,
     const Py_UNICODE *s2,
-    size_t n);
+    size_t n) Py_DEPRECATED(3.3);
 
 PyAPI_FUNC(int) Py_UNICODE_strcmp(
     const Py_UNICODE *s1,
     const Py_UNICODE *s2
-    );
+    ) Py_DEPRECATED(3.3);
 
 PyAPI_FUNC(int) Py_UNICODE_strncmp(
     const Py_UNICODE *s1,
     const Py_UNICODE *s2,
     size_t n
-    );
+    ) Py_DEPRECATED(3.3);
 
 PyAPI_FUNC(Py_UNICODE*) Py_UNICODE_strchr(
     const Py_UNICODE *s,
     Py_UNICODE c
-    );
+    ) Py_DEPRECATED(3.3);
 
 PyAPI_FUNC(Py_UNICODE*) Py_UNICODE_strrchr(
     const Py_UNICODE *s,
     Py_UNICODE c
-    );
+    ) Py_DEPRECATED(3.3);
 
 PyAPI_FUNC(PyObject*) _PyUnicode_FormatLong(PyObject *, int, int, int);
 
@@ -2297,7 +2301,7 @@
 
 PyAPI_FUNC(Py_UNICODE*) PyUnicode_AsUnicodeCopy(
     PyObject *unicode
-    );
+    ) Py_DEPRECATED(3.3);
 #endif /* Py_LIMITED_API */
 
 #if defined(Py_DEBUG) && !defined(Py_LIMITED_API)
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -410,6 +410,9 @@
 C API
 -----
 
+- Issue #19569: Compiler warnings are now emitted if use most of deprecated
+  functions.
+
 - Issue #28426: Deprecated undocumented functions PyUnicode_AsEncodedObject(),
   PyUnicode_AsDecodedObject(), PyUnicode_AsDecodedUnicode() and
   PyUnicode_AsEncodedUnicode().
diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c
--- a/Modules/_testcapimodule.c
+++ b/Modules/_testcapimodule.c
@@ -1398,11 +1398,9 @@
 getargs_u(PyObject *self, PyObject *args)
 {
     Py_UNICODE *str;
-    Py_ssize_t size;
     if (!PyArg_ParseTuple(args, "u", &str))
         return NULL;
-    size = Py_UNICODE_strlen(str);
-    return PyUnicode_FromUnicode(str, size);
+    return PyUnicode_FromWideChar(str, -1);
 }
 
 static PyObject *
@@ -1412,19 +1410,17 @@
     Py_ssize_t size;
     if (!PyArg_ParseTuple(args, "u#", &str, &size))
         return NULL;
-    return PyUnicode_FromUnicode(str, size);
+    return PyUnicode_FromWideChar(str, size);
 }
 
 static PyObject *
 getargs_Z(PyObject *self, PyObject *args)
 {
     Py_UNICODE *str;
-    Py_ssize_t size;
     if (!PyArg_ParseTuple(args, "Z", &str))
         return NULL;
     if (str != NULL) {
-        size = Py_UNICODE_strlen(str);
-        return PyUnicode_FromUnicode(str, size);
+        return PyUnicode_FromWideChar(str, -1);
     } else
         Py_RETURN_NONE;
 }
@@ -1437,7 +1433,7 @@
     if (!PyArg_ParseTuple(args, "Z#", &str, &size))
         return NULL;
     if (str != NULL)
-        return PyUnicode_FromUnicode(str, size);
+        return PyUnicode_FromWideChar(str, size);
     else
         Py_RETURN_NONE;
 }
diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c
--- a/Modules/arraymodule.c
+++ b/Modules/arraymodule.c
@@ -236,7 +236,7 @@
 static PyObject *
 u_getitem(arrayobject *ap, Py_ssize_t i)
 {
-    return PyUnicode_FromUnicode(&((Py_UNICODE *) ap->ob_item)[i], 1);
+    return PyUnicode_FromOrdinal(((Py_UNICODE *) ap->ob_item)[i]);
 }
 
 static int
@@ -1693,7 +1693,7 @@
              "tounicode() may only be called on unicode type arrays");
         return NULL;
     }
-    return PyUnicode_FromUnicode((Py_UNICODE *) self->ob_item, Py_SIZE(self));
+    return PyUnicode_FromWideChar((Py_UNICODE *) self->ob_item, Py_SIZE(self));
 }
 
 /*[clinic input]
diff --git a/Objects/abstract.c b/Objects/abstract.c
--- a/Objects/abstract.c
+++ b/Objects/abstract.c
@@ -252,14 +252,6 @@
    cause issues later on.  Don't use these functions in new code.
  */
 int
-PyObject_AsCharBuffer(PyObject *obj,
-                      const char **buffer,
-                      Py_ssize_t *buffer_len)
-{
-    return PyObject_AsReadBuffer(obj, (const void **)buffer, buffer_len);
-}
-
-int
 PyObject_CheckReadBuffer(PyObject *obj)
 {
     PyBufferProcs *pb = obj->ob_type->tp_as_buffer;
@@ -276,9 +268,8 @@
     return 1;
 }
 
-int PyObject_AsReadBuffer(PyObject *obj,
-                          const void **buffer,
-                          Py_ssize_t *buffer_len)
+static int
+as_read_buffer(PyObject *obj, const void **buffer, Py_ssize_t *buffer_len)
 {
     Py_buffer view;
 
@@ -295,6 +286,21 @@
     return 0;
 }
 
+int
+PyObject_AsCharBuffer(PyObject *obj,
+                      const char **buffer,
+                      Py_ssize_t *buffer_len)
+{
+    return as_read_buffer(obj, (const void **)buffer, buffer_len);
+}
+
+int PyObject_AsReadBuffer(PyObject *obj,
+                          const void **buffer,
+                          Py_ssize_t *buffer_len)
+{
+    return as_read_buffer(obj, buffer, buffer_len);
+}
+
 int PyObject_AsWriteBuffer(PyObject *obj,
                            void **buffer,
                            Py_ssize_t *buffer_len)
diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c
--- a/Objects/bytearrayobject.c
+++ b/Objects/bytearrayobject.c
@@ -2324,10 +2324,7 @@
         return Py_BuildValue("N(O)n", _PyObject_GetBuiltin("iter"),
                              it->it_seq, it->it_index);
     } else {
-        PyObject *u = PyUnicode_FromUnicode(NULL, 0);
-        if (u == NULL)
-            return NULL;
-        return Py_BuildValue("N(N)", _PyObject_GetBuiltin("iter"), u);
+        return Py_BuildValue("N(())", _PyObject_GetBuiltin("iter"));
     }
 }
 
diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c
--- a/Objects/bytesobject.c
+++ b/Objects/bytesobject.c
@@ -3048,10 +3048,7 @@
         return Py_BuildValue("N(O)n", _PyObject_GetBuiltin("iter"),
                              it->it_seq, it->it_index);
     } else {
-        PyObject *u = PyUnicode_FromUnicode(NULL, 0);
-        if (u == NULL)
-            return NULL;
-        return Py_BuildValue("N(N)", _PyObject_GetBuiltin("iter"), u);
+        return Py_BuildValue("N(())", _PyObject_GetBuiltin("iter"));
     }
 }
 
diff --git a/Objects/codeobject.c b/Objects/codeobject.c
--- a/Objects/codeobject.c
+++ b/Objects/codeobject.c
@@ -115,7 +115,7 @@
 
     /* Check argument types */
     if (argcount < 0 || kwonlyargcount < 0 || nlocals < 0 ||
-        code == NULL ||
+        code == NULL || !PyBytes_Check(code) ||
         consts == NULL || !PyTuple_Check(consts) ||
         names == NULL || !PyTuple_Check(names) ||
         varnames == NULL || !PyTuple_Check(varnames) ||
@@ -123,8 +123,7 @@
         cellvars == NULL || !PyTuple_Check(cellvars) ||
         name == NULL || !PyUnicode_Check(name) ||
         filename == NULL || !PyUnicode_Check(filename) ||
-        lnotab == NULL || !PyBytes_Check(lnotab) ||
-        !PyObject_CheckReadBuffer(code)) {
+        lnotab == NULL || !PyBytes_Check(lnotab)) {
         PyErr_BadInternalCall();
         return NULL;
     }
diff --git a/Objects/longobject.c b/Objects/longobject.c
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -2479,7 +2479,7 @@
 PyObject *
 PyLong_FromUnicode(Py_UNICODE *u, Py_ssize_t length, int base)
 {
-    PyObject *v, *unicode = PyUnicode_FromUnicode(u, length);
+    PyObject *v, *unicode = PyUnicode_FromWideChar(u, length);
     if (unicode == NULL)
         return NULL;
     v = PyLong_FromUnicodeObject(unicode, base);
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -1988,12 +1988,32 @@
 PyObject *
 PyUnicode_FromUnicode(const Py_UNICODE *u, Py_ssize_t size)
 {
+    if (u == NULL)
+        return (PyObject*)_PyUnicode_New(size);
+
+    if (size < 0) {
+        PyErr_BadInternalCall();
+        return NULL;
+    }
+
+    return PyUnicode_FromWideChar(u, size);
+}
+
+PyObject *
+PyUnicode_FromWideChar(const wchar_t *u, Py_ssize_t size)
+{
     PyObject *unicode;
     Py_UCS4 maxchar = 0;
     Py_ssize_t num_surrogates;
 
-    if (u == NULL)
-        return (PyObject*)_PyUnicode_New(size);
+    if (u == NULL && size != 0) {
+        PyErr_BadInternalCall();
+        return NULL;
+    }
+
+    if (size == -1) {
+        size = wcslen(u);
+    }
 
     /* If the Unicode data is known at construction time, we can apply
        some optimizations which share commonly used objects. */
@@ -2478,27 +2498,6 @@
     return as_ucs4(string, NULL, 0, 1);
 }
 
-#ifdef HAVE_WCHAR_H
-
-PyObject *
-PyUnicode_FromWideChar(const wchar_t *w, Py_ssize_t size)
-{
-    if (w == NULL) {
-        if (size == 0)
-            _Py_RETURN_UNICODE_EMPTY();
-        PyErr_BadInternalCall();
-        return NULL;
-    }
-
-    if (size == -1) {
-        size = wcslen(w);
-    }
-
-    return PyUnicode_FromUnicode(w, size);
-}
-
-#endif /* HAVE_WCHAR_H */
-
 /* maximum number of characters required for output of %lld or %p.
    We need at most ceil(log10(256)*SIZEOF_LONG_LONG) digits,
    plus 1 for the sign.  53/22 is an upper bound for log10(256). */
@@ -3296,7 +3295,7 @@
 {
     PyObject *v, *unicode;
 
-    unicode = PyUnicode_FromUnicode(s, size);
+    unicode = PyUnicode_FromWideChar(s, size);
     if (unicode == NULL)
         return NULL;
     v = PyUnicode_AsEncodedString(unicode, encoding, errors);
@@ -4129,7 +4128,11 @@
         PyErr_BadArgument();
         goto onError;
     }
-    return PyUnicode_GET_SIZE(unicode);
+    if (_PyUnicode_WSTR(unicode) == NULL) {
+        if (PyUnicode_AsUnicode(unicode) == NULL)
+            goto onError;
+    }
+    return PyUnicode_WSTR_LENGTH(unicode);
 
   onError:
     return -1;
@@ -4815,7 +4818,7 @@
                      const char *errors)
 {
     PyObject *result;
-    PyObject *tmp = PyUnicode_FromUnicode(s, size);
+    PyObject *tmp = PyUnicode_FromWideChar(s, size);
     if (tmp == NULL)
         return NULL;
     result = _PyUnicode_EncodeUTF7(tmp, base64SetO,
@@ -5171,7 +5174,7 @@
 {
     PyObject *v, *unicode;
 
-    unicode = PyUnicode_FromUnicode(s, size);
+    unicode = PyUnicode_FromWideChar(s, size);
     if (unicode == NULL)
         return NULL;
     v = _PyUnicode_AsUTF8String(unicode, errors);
@@ -5496,7 +5499,7 @@
                       int byteorder)
 {
     PyObject *result;
-    PyObject *tmp = PyUnicode_FromUnicode(s, size);
+    PyObject *tmp = PyUnicode_FromWideChar(s, size);
     if (tmp == NULL)
         return NULL;
     result = _PyUnicode_EncodeUTF32(tmp, errors, byteorder);
@@ -5849,7 +5852,7 @@
                       int byteorder)
 {
     PyObject *result;
-    PyObject *tmp = PyUnicode_FromUnicode(s, size);
+    PyObject *tmp = PyUnicode_FromWideChar(s, size);
     if (tmp == NULL)
         return NULL;
     result = _PyUnicode_EncodeUTF16(tmp, errors, byteorder);
@@ -6246,7 +6249,7 @@
                               Py_ssize_t size)
 {
     PyObject *result;
-    PyObject *tmp = PyUnicode_FromUnicode(s, size);
+    PyObject *tmp = PyUnicode_FromWideChar(s, size);
     if (tmp == NULL) {
         return NULL;
     }
@@ -6463,7 +6466,7 @@
                                  Py_ssize_t size)
 {
     PyObject *result;
-    PyObject *tmp = PyUnicode_FromUnicode(s, size);
+    PyObject *tmp = PyUnicode_FromWideChar(s, size);
     if (tmp == NULL)
         return NULL;
     result = PyUnicode_AsRawUnicodeEscapeString(tmp);
@@ -6874,7 +6877,7 @@
                        const char *errors)
 {
     PyObject *result;
-    PyObject *unicode = PyUnicode_FromUnicode(p, size);
+    PyObject *unicode = PyUnicode_FromWideChar(p, size);
     if (unicode == NULL)
         return NULL;
     result = unicode_encode_ucs1(unicode, errors, 256);
@@ -7015,7 +7018,7 @@
                       const char *errors)
 {
     PyObject *result;
-    PyObject *unicode = PyUnicode_FromUnicode(p, size);
+    PyObject *unicode = PyUnicode_FromWideChar(p, size);
     if (unicode == NULL)
         return NULL;
     result = unicode_encode_ucs1(unicode, errors, 128);
@@ -7741,7 +7744,7 @@
                      const char *errors)
 {
     PyObject *unicode, *res;
-    unicode = PyUnicode_FromUnicode(p, size);
+    unicode = PyUnicode_FromWideChar(p, size);
     if (unicode == NULL)
         return NULL;
     res = encode_code_page(CP_ACP, unicode, errors);
@@ -8589,7 +8592,7 @@
                         const char *errors)
 {
     PyObject *result;
-    PyObject *unicode = PyUnicode_FromUnicode(p, size);
+    PyObject *unicode = PyUnicode_FromWideChar(p, size);
     if (unicode == NULL)
         return NULL;
     result = _PyUnicode_EncodeCharmap(unicode, mapping, errors);
@@ -9029,7 +9032,7 @@
                            const char *errors)
 {
     PyObject *result;
-    PyObject *unicode = PyUnicode_FromUnicode(p, size);
+    PyObject *unicode = PyUnicode_FromWideChar(p, size);
     if (!unicode)
         return NULL;
     result = _PyUnicode_TranslateCharmap(unicode, mapping, errors);
@@ -9157,14 +9160,10 @@
         return -1;
     }
 
-    unicode = PyUnicode_FromUnicode(s, length);
+    unicode = PyUnicode_FromWideChar(s, length);
     if (unicode == NULL)
         return -1;
 
-    if (PyUnicode_READY(unicode) == -1) {
-        Py_DECREF(unicode);
-        return -1;
-    }
     kind = PyUnicode_KIND(unicode);
     data = PyUnicode_DATA(unicode);
 
@@ -15332,7 +15331,7 @@
         return Py_BuildValue("N(O)n", _PyObject_GetBuiltin("iter"),
                              it->it_seq, it->it_index);
     } else {
-        PyObject *u = PyUnicode_FromUnicode(NULL, 0);
+        PyObject *u = (PyObject *)_PyUnicode_New(0);
         if (u == NULL)
             return NULL;
         return Py_BuildValue("N(N)", _PyObject_GetBuiltin("iter"), u);
@@ -15427,10 +15426,7 @@
 size_t
 Py_UNICODE_strlen(const Py_UNICODE *u)
 {
-    int res = 0;
-    while(*u++)
-        res++;
-    return res;
+    return wcslen(u);
 }
 
 Py_UNICODE*
@@ -15455,8 +15451,8 @@
 Py_UNICODE_strcat(Py_UNICODE *s1, const Py_UNICODE *s2)
 {
     Py_UNICODE *u1 = s1;
-    u1 += Py_UNICODE_strlen(u1);
-    Py_UNICODE_strcpy(u1, s2);
+    u1 += wcslen(u1);
+    while ((*u1++ = *s2++));
     return s1;
 }
 
@@ -15505,7 +15501,7 @@
 Py_UNICODE_strrchr(const Py_UNICODE *s, Py_UNICODE c)
 {
     const Py_UNICODE *p;
-    p = s + Py_UNICODE_strlen(s);
+    p = s + wcslen(s);
     while (p != s) {
         p--;
         if (*p == c)
diff --git a/Python/errors.c b/Python/errors.c
--- a/Python/errors.c
+++ b/Python/errors.c
@@ -582,9 +582,7 @@
 PyObject *
 PyErr_SetFromErrnoWithUnicodeFilename(PyObject *exc, const Py_UNICODE *filename)
 {
-    PyObject *name = filename ?
-                     PyUnicode_FromUnicode(filename, wcslen(filename)) :
-             NULL;
+    PyObject *name = filename ? PyUnicode_FromWideChar(filename, -1) : NULL;
     PyObject *result = PyErr_SetFromErrnoWithFilenameObjects(exc, name, NULL);
     Py_XDECREF(name);
     return result;
@@ -691,9 +689,7 @@
     int ierr,
     const Py_UNICODE *filename)
 {
-    PyObject *name = filename ?
-                     PyUnicode_FromUnicode(filename, wcslen(filename)) :
-             NULL;
+    PyObject *name = filename ? PyUnicode_FromWideChar(filename, -1) : NULL;
     PyObject *ret = PyErr_SetExcFromWindowsErrWithFilenameObjects(exc,
                                                                  ierr,
                                                                  name,
@@ -729,9 +725,7 @@
     int ierr,
     const Py_UNICODE *filename)
 {
-    PyObject *name = filename ?
-                     PyUnicode_FromUnicode(filename, wcslen(filename)) :
-             NULL;
+    PyObject *name = filename ? PyUnicode_FromWideChar(filename, -1) : NULL;
     PyObject *result = PyErr_SetExcFromWindowsErrWithFilenameObjects(
                                                   PyExc_OSError,
                                                   ierr, name, NULL);
diff --git a/Python/getargs.c b/Python/getargs.c
--- a/Python/getargs.c
+++ b/Python/getargs.c
@@ -1027,7 +1027,7 @@
                 *p = PyUnicode_AsUnicodeAndSize(arg, &len);
                 if (*p == NULL)
                     RETURN_ERR_OCCURRED;
-                if (Py_UNICODE_strlen(*p) != (size_t)len) {
+                if (wcslen(*p) != (size_t)len) {
                     PyErr_SetString(PyExc_ValueError, "embedded null character");
                     RETURN_ERR_OCCURRED;
                 }
@@ -1074,9 +1074,14 @@
             (PyBytes_Check(arg) || PyByteArray_Check(arg))) {
             s = arg;
             Py_INCREF(s);
-            if (PyObject_AsCharBuffer(s, &ptr, &size) < 0)
-                return converterr("(AsCharBuffer failed)",
-                                  arg, msgbuf, bufsize);
+            if (PyBytes_Check(arg)) {
+                size = PyBytes_GET_SIZE(s);
+                ptr = PyBytes_AS_STRING(s);
+            }
+            else {
+                size = PyByteArray_GET_SIZE(s);
+                ptr = PyByteArray_AS_STRING(s);
+            }
         }
         else if (PyUnicode_Check(arg)) {
             /* Encode object; use default error handling */
diff --git a/Python/modsupport.c b/Python/modsupport.c
--- a/Python/modsupport.c
+++ b/Python/modsupport.c
@@ -286,8 +286,8 @@
             }
             else {
                 if (n < 0)
-                    n = Py_UNICODE_strlen(u);
-                v = PyUnicode_FromUnicode(u, n);
+                    n = wcslen(u);
+                v = PyUnicode_FromWideChar(u, n);
             }
             return v;
         }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 09:28:55 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 14:28:55 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328666=3A_Now_test=2Esupport=2Ermtree_is_able_to_remov?=
 =?utf-8?q?e_unwritable_or?=
Message-ID: <20161120142053.45777.78637.81A14735@psf.io>

https://hg.python.org/cpython/rev/efe2993b20e2
changeset:   105217:efe2993b20e2
branch:      3.6
parent:      105212:f26d3f9a958a
parent:      105216:c92f9be77b9b
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 16:19:20 2016 +0200
summary:
  Issue #28666: Now test.support.rmtree is able to remove unwritable or
unreadable directories.

files:
  Lib/test/support/__init__.py |  32 ++++++++++++++++++++++-
  Lib/test/test_support.py     |  33 ++++++++++++++++--------
  Misc/NEWS                    |   3 ++
  3 files changed, 56 insertions(+), 12 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -358,7 +358,37 @@
 else:
     _unlink = os.unlink
     _rmdir = os.rmdir
-    _rmtree = shutil.rmtree
+
+    def _rmtree(path):
+        try:
+            shutil.rmtree(path)
+            return
+        except OSError:
+            pass
+
+        def force_run(path, func, *args):
+            try:
+                return func(*args)
+            except OSError as err:
+                if verbose >= 2:
+                    print('%s: %s' % (err.__class__.__name__, err))
+                    print('re-run %s%r' % (func.__name__, args))
+                os.chmod(path, stat.S_IRWXU)
+                return func(*args)
+        def _rmtree_inner(path):
+            for name in force_run(path, os.listdir, path):
+                fullname = os.path.join(path, name)
+                try:
+                    mode = os.lstat(fullname).st_mode
+                except OSError:
+                    mode = 0
+                if stat.S_ISDIR(mode):
+                    _rmtree_inner(fullname)
+                    force_run(path, os.rmdir, fullname)
+                else:
+                    force_run(path, os.unlink, fullname)
+        _rmtree_inner(path)
+        os.rmdir(path)
 
 def unlink(filename):
     try:
diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py
--- a/Lib/test/test_support.py
+++ b/Lib/test/test_support.py
@@ -1,5 +1,6 @@
 import importlib
 import shutil
+import stat
 import sys
 import os
 import unittest
@@ -12,9 +13,6 @@
 
 
 class TestSupport(unittest.TestCase):
-    def setUp(self):
-        support.unlink(TESTFN)
-    tearDown = setUp
 
     def test_import_module(self):
         support.import_module("ftplib")
@@ -46,15 +44,28 @@
         support.unlink(TESTFN)
 
     def test_rmtree(self):
-        TESTDIRN = os.path.basename(tempfile.mkdtemp(dir='.'))
-        self.addCleanup(support.rmtree, TESTDIRN)
-        support.rmtree(TESTDIRN)
+        dirpath = support.TESTFN + 'd'
+        subdirpath = os.path.join(dirpath, 'subdir')
+        os.mkdir(dirpath)
+        os.mkdir(subdirpath)
+        support.rmtree(dirpath)
+        self.assertFalse(os.path.exists(dirpath))
+        with support.swap_attr(support, 'verbose', 0):
+            support.rmtree(dirpath)
 
-        os.mkdir(TESTDIRN)
-        os.mkdir(os.path.join(TESTDIRN, TESTDIRN))
-        support.rmtree(TESTDIRN)
-        self.assertFalse(os.path.exists(TESTDIRN))
-        support.rmtree(TESTDIRN)
+        os.mkdir(dirpath)
+        os.mkdir(subdirpath)
+        os.chmod(dirpath, stat.S_IRUSR|stat.S_IXUSR)
+        with support.swap_attr(support, 'verbose', 0):
+            support.rmtree(dirpath)
+        self.assertFalse(os.path.exists(dirpath))
+
+        os.mkdir(dirpath)
+        os.mkdir(subdirpath)
+        os.chmod(dirpath, 0)
+        with support.swap_attr(support, 'verbose', 0):
+            support.rmtree(dirpath)
+        self.assertFalse(os.path.exists(dirpath))
 
     def test_forget(self):
         mod_filename = TESTFN + '.py'
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -82,6 +82,9 @@
 Tests
 -----
 
+- Issue #28666: Now test.support.rmtree is able to remove unwritable or
+  unreadable directories.
+
 - Issue #23839: Various caches now are cleared before running every test file.
 
 Build

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 09:28:55 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 14:28:55 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328666=3A_Now_test=2Esupport=2Ermtree_is_able_to?=
 =?utf-8?q?_remove_unwritable_or?=
Message-ID: <20161120142053.46084.53988.525CB5AA@psf.io>

https://hg.python.org/cpython/rev/3a1e75ecc17d
changeset:   105218:3a1e75ecc17d
parent:      105214:f692dafe6797
parent:      105217:efe2993b20e2
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 16:20:20 2016 +0200
summary:
  Issue #28666: Now test.support.rmtree is able to remove unwritable or
unreadable directories.

files:
  Lib/test/support/__init__.py |  32 ++++++++++++++++++++++-
  Lib/test/test_support.py     |  33 ++++++++++++++++--------
  Misc/NEWS                    |   3 ++
  3 files changed, 56 insertions(+), 12 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -358,7 +358,37 @@
 else:
     _unlink = os.unlink
     _rmdir = os.rmdir
-    _rmtree = shutil.rmtree
+
+    def _rmtree(path):
+        try:
+            shutil.rmtree(path)
+            return
+        except OSError:
+            pass
+
+        def force_run(path, func, *args):
+            try:
+                return func(*args)
+            except OSError as err:
+                if verbose >= 2:
+                    print('%s: %s' % (err.__class__.__name__, err))
+                    print('re-run %s%r' % (func.__name__, args))
+                os.chmod(path, stat.S_IRWXU)
+                return func(*args)
+        def _rmtree_inner(path):
+            for name in force_run(path, os.listdir, path):
+                fullname = os.path.join(path, name)
+                try:
+                    mode = os.lstat(fullname).st_mode
+                except OSError:
+                    mode = 0
+                if stat.S_ISDIR(mode):
+                    _rmtree_inner(fullname)
+                    force_run(path, os.rmdir, fullname)
+                else:
+                    force_run(path, os.unlink, fullname)
+        _rmtree_inner(path)
+        os.rmdir(path)
 
 def unlink(filename):
     try:
diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py
--- a/Lib/test/test_support.py
+++ b/Lib/test/test_support.py
@@ -1,5 +1,6 @@
 import importlib
 import shutil
+import stat
 import sys
 import os
 import unittest
@@ -12,9 +13,6 @@
 
 
 class TestSupport(unittest.TestCase):
-    def setUp(self):
-        support.unlink(TESTFN)
-    tearDown = setUp
 
     def test_import_module(self):
         support.import_module("ftplib")
@@ -46,15 +44,28 @@
         support.unlink(TESTFN)
 
     def test_rmtree(self):
-        TESTDIRN = os.path.basename(tempfile.mkdtemp(dir='.'))
-        self.addCleanup(support.rmtree, TESTDIRN)
-        support.rmtree(TESTDIRN)
+        dirpath = support.TESTFN + 'd'
+        subdirpath = os.path.join(dirpath, 'subdir')
+        os.mkdir(dirpath)
+        os.mkdir(subdirpath)
+        support.rmtree(dirpath)
+        self.assertFalse(os.path.exists(dirpath))
+        with support.swap_attr(support, 'verbose', 0):
+            support.rmtree(dirpath)
 
-        os.mkdir(TESTDIRN)
-        os.mkdir(os.path.join(TESTDIRN, TESTDIRN))
-        support.rmtree(TESTDIRN)
-        self.assertFalse(os.path.exists(TESTDIRN))
-        support.rmtree(TESTDIRN)
+        os.mkdir(dirpath)
+        os.mkdir(subdirpath)
+        os.chmod(dirpath, stat.S_IRUSR|stat.S_IXUSR)
+        with support.swap_attr(support, 'verbose', 0):
+            support.rmtree(dirpath)
+        self.assertFalse(os.path.exists(dirpath))
+
+        os.mkdir(dirpath)
+        os.mkdir(subdirpath)
+        os.chmod(dirpath, 0)
+        with support.swap_attr(support, 'verbose', 0):
+            support.rmtree(dirpath)
+        self.assertFalse(os.path.exists(dirpath))
 
     def test_forget(self):
         mod_filename = TESTFN + '.py'
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -470,6 +470,9 @@
 Tests
 -----
 
+- Issue #28666: Now test.support.rmtree is able to remove unwritable or
+  unreadable directories.
+
 - Issue #23839: Various caches now are cleared before running every test file.
 
 - Issue #26944: Fix test_posix for Android where 'id -G' is entirely wrong or

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 09:28:55 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 14:28:55 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4NjY2?=
 =?utf-8?q?=3A_Now_test=2Esupport=2Ermtree_is_able_to_remove_unwritable_or?=
Message-ID: <20161120142053.7475.16451.2CA0C4F1@psf.io>

https://hg.python.org/cpython/rev/63820871014d
changeset:   105215:63820871014d
branch:      2.7
parent:      105141:ea91bb92c28b
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 16:15:35 2016 +0200
summary:
  Issue #28666: Now test.support.rmtree is able to remove unwritable or
unreadable directories.

files:
  Lib/test/test_support.py |  33 +++++++++++++++++++++++++++-
  Misc/NEWS                |   3 ++
  2 files changed, 35 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py
--- a/Lib/test/test_support.py
+++ b/Lib/test/test_support.py
@@ -236,7 +236,38 @@
 else:
     _unlink = os.unlink
     _rmdir = os.rmdir
-    _rmtree = shutil.rmtree
+
+    def _rmtree(path):
+        import stat
+        try:
+            shutil.rmtree(path)
+            return
+        except EnvironmentError:
+            pass
+
+        def force_run(path, func, *args):
+            try:
+                return func(*args)
+            except EnvironmentError as err:
+                if verbose >= 2:
+                    print('%s: %s' % (err.__class__.__name__, err))
+                    print('re-run %s%r' % (func.__name__, args))
+                os.chmod(path, stat.S_IRWXU)
+                return func(*args)
+        def _rmtree_inner(path):
+            for name in force_run(path, os.listdir, path):
+                fullname = os.path.join(path, name)
+                try:
+                    mode = os.lstat(fullname).st_mode
+                except EnvironmentError:
+                    mode = 0
+                if stat.S_ISDIR(mode):
+                    _rmtree_inner(fullname)
+                    force_run(path, os.rmdir, fullname)
+                else:
+                    force_run(path, os.unlink, fullname)
+        _rmtree_inner(path)
+        os.rmdir(path)
 
 def unlink(filename):
     try:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -262,6 +262,9 @@
 Tests
 -----
 
+- Issue #28666: Now test.test_support.rmtree is able to remove unwritable or
+  unreadable directories.
+
 - Issue #23839: Various caches now are cleared before running every test file.
 
 - Issue #27369: In test_pyexpat, avoid testing an error message detail that

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 09:28:55 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 14:28:55 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjY2?=
 =?utf-8?q?=3A_Now_test=2Esupport=2Ermtree_is_able_to_remove_unwritable_or?=
Message-ID: <20161120142053.41115.61282.0DBF42F0@psf.io>

https://hg.python.org/cpython/rev/c92f9be77b9b
changeset:   105216:c92f9be77b9b
branch:      3.5
parent:      105211:75fe67538905
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 16:16:06 2016 +0200
summary:
  Issue #28666: Now test.support.rmtree is able to remove unwritable or
unreadable directories.

files:
  Lib/test/support/__init__.py |  32 ++++++++++++++++++++++-
  Lib/test/test_support.py     |  33 ++++++++++++++++-------
  Misc/NEWS                    |   3 ++
  3 files changed, 57 insertions(+), 11 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -357,7 +357,37 @@
 else:
     _unlink = os.unlink
     _rmdir = os.rmdir
-    _rmtree = shutil.rmtree
+
+    def _rmtree(path):
+        try:
+            shutil.rmtree(path)
+            return
+        except OSError:
+            pass
+
+        def force_run(path, func, *args):
+            try:
+                return func(*args)
+            except OSError as err:
+                if verbose >= 2:
+                    print('%s: %s' % (err.__class__.__name__, err))
+                    print('re-run %s%r' % (func.__name__, args))
+                os.chmod(path, stat.S_IRWXU)
+                return func(*args)
+        def _rmtree_inner(path):
+            for name in force_run(path, os.listdir, path):
+                fullname = os.path.join(path, name)
+                try:
+                    mode = os.lstat(fullname).st_mode
+                except OSError:
+                    mode = 0
+                if stat.S_ISDIR(mode):
+                    _rmtree_inner(fullname)
+                    force_run(path, os.rmdir, fullname)
+                else:
+                    force_run(path, os.unlink, fullname)
+        _rmtree_inner(path)
+        os.rmdir(path)
 
 def unlink(filename):
     try:
diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py
--- a/Lib/test/test_support.py
+++ b/Lib/test/test_support.py
@@ -1,5 +1,6 @@
 import importlib
 import shutil
+import stat
 import sys
 import os
 import unittest
@@ -9,14 +10,9 @@
 from test import support
 
 TESTFN = support.TESTFN
-TESTDIRN = os.path.basename(tempfile.mkdtemp(dir='.'))
 
 
 class TestSupport(unittest.TestCase):
-    def setUp(self):
-        support.unlink(TESTFN)
-        support.rmtree(TESTDIRN)
-    tearDown = setUp
 
     def test_import_module(self):
         support.import_module("ftplib")
@@ -48,11 +44,28 @@
         support.unlink(TESTFN)
 
     def test_rmtree(self):
-        os.mkdir(TESTDIRN)
-        os.mkdir(os.path.join(TESTDIRN, TESTDIRN))
-        support.rmtree(TESTDIRN)
-        self.assertFalse(os.path.exists(TESTDIRN))
-        support.rmtree(TESTDIRN)
+        dirpath = support.TESTFN + 'd'
+        subdirpath = os.path.join(dirpath, 'subdir')
+        os.mkdir(dirpath)
+        os.mkdir(subdirpath)
+        support.rmtree(dirpath)
+        self.assertFalse(os.path.exists(dirpath))
+        with support.swap_attr(support, 'verbose', 0):
+            support.rmtree(dirpath)
+
+        os.mkdir(dirpath)
+        os.mkdir(subdirpath)
+        os.chmod(dirpath, stat.S_IRUSR|stat.S_IXUSR)
+        with support.swap_attr(support, 'verbose', 0):
+            support.rmtree(dirpath)
+        self.assertFalse(os.path.exists(dirpath))
+
+        os.mkdir(dirpath)
+        os.mkdir(subdirpath)
+        os.chmod(dirpath, 0)
+        with support.swap_attr(support, 'verbose', 0):
+            support.rmtree(dirpath)
+        self.assertFalse(os.path.exists(dirpath))
 
     def test_forget(self):
         mod_filename = TESTFN + '.py'
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -511,6 +511,9 @@
 Tests
 -----
 
+- Issue #28666: Now test.support.rmtree is able to remove unwritable or
+  unreadable directories.
+
 - Issue #23839: Various caches now are cleared before running every test file.
 
 - Issue #28409: regrtest: fix the parser of command line arguments.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 10:21:55 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 15:21:55 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328750=3A_Fixed_docs_of_of_unicode-escape_an_raw?=
 =?utf-8?q?-unicode-escape_C_API=2E?=
Message-ID: <20161120152153.10190.81224.885343EF@psf.io>

https://hg.python.org/cpython/rev/deff7bf26d00
changeset:   105221:deff7bf26d00
parent:      105218:3a1e75ecc17d
parent:      105220:0c6fccf04a79
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 17:21:38 2016 +0200
summary:
  Issue #28750: Fixed docs of of unicode-escape an raw-unicode-escape C API.
Patch by Xiang Zhang.

files:
  Doc/c-api/unicode.rst |  12 +++++-------
  1 files changed, 5 insertions(+), 7 deletions(-)


diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -1296,16 +1296,15 @@
 
 .. c:function:: PyObject* PyUnicode_AsUnicodeEscapeString(PyObject *unicode)
 
-   Encode a Unicode object using Unicode-Escape and return the result as Python
-   string object.  Error handling is "strict". Return *NULL* if an exception was
+   Encode a Unicode object using Unicode-Escape and return the result as a
+   bytes object.  Error handling is "strict".  Return *NULL* if an exception was
    raised by the codec.
 
 
 .. c:function:: PyObject* PyUnicode_EncodeUnicodeEscape(const Py_UNICODE *s, Py_ssize_t size)
 
    Encode the :c:type:`Py_UNICODE` buffer of the given *size* using Unicode-Escape and
-   return a Python string object.  Return *NULL* if an exception was raised by the
-   codec.
+   return a bytes object.  Return *NULL* if an exception was raised by the codec.
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
@@ -1328,7 +1327,7 @@
 .. c:function:: PyObject* PyUnicode_AsRawUnicodeEscapeString(PyObject *unicode)
 
    Encode a Unicode object using Raw-Unicode-Escape and return the result as
-   Python string object. Error handling is "strict". Return *NULL* if an exception
+   a bytes object.  Error handling is "strict".  Return *NULL* if an exception
    was raised by the codec.
 
 
@@ -1336,8 +1335,7 @@
                               Py_ssize_t size, const char *errors)
 
    Encode the :c:type:`Py_UNICODE` buffer of the given *size* using Raw-Unicode-Escape
-   and return a Python string object.  Return *NULL* if an exception was raised by
-   the codec.
+   and return a bytes object.  Return *NULL* if an exception was raised by the codec.
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 10:21:55 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 15:21:55 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328750=3A_Fixed_docs_of_of_unicode-escape_an_raw-unico?=
 =?utf-8?q?de-escape_C_API=2E?=
Message-ID: <20161120152153.12159.98334.CF39D68C@psf.io>

https://hg.python.org/cpython/rev/0c6fccf04a79
changeset:   105220:0c6fccf04a79
branch:      3.6
parent:      105217:efe2993b20e2
parent:      105219:059b8e15b738
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 17:21:04 2016 +0200
summary:
  Issue #28750: Fixed docs of of unicode-escape an raw-unicode-escape C API.
Patch by Xiang Zhang.

files:
  Doc/c-api/unicode.rst |  12 +++++-------
  1 files changed, 5 insertions(+), 7 deletions(-)


diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -1296,16 +1296,15 @@
 
 .. c:function:: PyObject* PyUnicode_AsUnicodeEscapeString(PyObject *unicode)
 
-   Encode a Unicode object using Unicode-Escape and return the result as Python
-   string object.  Error handling is "strict". Return *NULL* if an exception was
+   Encode a Unicode object using Unicode-Escape and return the result as a
+   bytes object.  Error handling is "strict".  Return *NULL* if an exception was
    raised by the codec.
 
 
 .. c:function:: PyObject* PyUnicode_EncodeUnicodeEscape(const Py_UNICODE *s, Py_ssize_t size)
 
    Encode the :c:type:`Py_UNICODE` buffer of the given *size* using Unicode-Escape and
-   return a Python string object.  Return *NULL* if an exception was raised by the
-   codec.
+   return a bytes object.  Return *NULL* if an exception was raised by the codec.
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
@@ -1328,7 +1327,7 @@
 .. c:function:: PyObject* PyUnicode_AsRawUnicodeEscapeString(PyObject *unicode)
 
    Encode a Unicode object using Raw-Unicode-Escape and return the result as
-   Python string object. Error handling is "strict". Return *NULL* if an exception
+   a bytes object.  Error handling is "strict".  Return *NULL* if an exception
    was raised by the codec.
 
 
@@ -1336,8 +1335,7 @@
                               Py_ssize_t size, const char *errors)
 
    Encode the :c:type:`Py_UNICODE` buffer of the given *size* using Raw-Unicode-Escape
-   and return a Python string object.  Return *NULL* if an exception was raised by
-   the codec.
+   and return a bytes object.  Return *NULL* if an exception was raised by the codec.
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 10:21:55 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 15:21:55 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NzUw?=
 =?utf-8?q?=3A_Fixed_docs_of_of_unicode-escape_an_raw-unicode-escape_C_API?=
 =?utf-8?q?=2E?=
Message-ID: <20161120152153.27941.71424.CAB6EFC5@psf.io>

https://hg.python.org/cpython/rev/059b8e15b738
changeset:   105219:059b8e15b738
branch:      3.5
parent:      105216:c92f9be77b9b
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 17:20:19 2016 +0200
summary:
  Issue #28750: Fixed docs of of unicode-escape an raw-unicode-escape C API.
Patch by Xiang Zhang.

files:
  Doc/c-api/unicode.rst |  12 +++++-------
  1 files changed, 5 insertions(+), 7 deletions(-)


diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -1291,16 +1291,15 @@
 
 .. c:function:: PyObject* PyUnicode_AsUnicodeEscapeString(PyObject *unicode)
 
-   Encode a Unicode object using Unicode-Escape and return the result as Python
-   string object.  Error handling is "strict". Return *NULL* if an exception was
+   Encode a Unicode object using Unicode-Escape and return the result as a
+   bytes object.  Error handling is "strict".  Return *NULL* if an exception was
    raised by the codec.
 
 
 .. c:function:: PyObject* PyUnicode_EncodeUnicodeEscape(const Py_UNICODE *s, Py_ssize_t size)
 
    Encode the :c:type:`Py_UNICODE` buffer of the given *size* using Unicode-Escape and
-   return a Python string object.  Return *NULL* if an exception was raised by the
-   codec.
+   return a bytes object.  Return *NULL* if an exception was raised by the codec.
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using
@@ -1323,7 +1322,7 @@
 .. c:function:: PyObject* PyUnicode_AsRawUnicodeEscapeString(PyObject *unicode)
 
    Encode a Unicode object using Raw-Unicode-Escape and return the result as
-   Python string object. Error handling is "strict". Return *NULL* if an exception
+   a bytes object.  Error handling is "strict".  Return *NULL* if an exception
    was raised by the codec.
 
 
@@ -1331,8 +1330,7 @@
                               Py_ssize_t size, const char *errors)
 
    Encode the :c:type:`Py_UNICODE` buffer of the given *size* using Raw-Unicode-Escape
-   and return a Python string object.  Return *NULL* if an exception was raised by
-   the codec.
+   and return a bytes object.  Return *NULL* if an exception was raised by the codec.
 
    .. deprecated-removed:: 3.3 4.0
       Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 10:43:23 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 15:43:23 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjY2?=
 =?utf-8?q?=3A_Now_test=2Esupport=2Ermtree_is_able_to_remove_unwritable_or?=
Message-ID: <20161120154323.40774.40251.4E1935AE@psf.io>

https://hg.python.org/cpython/rev/9e23b8996584
changeset:   105223:9e23b8996584
branch:      3.5
parent:      105219:059b8e15b738
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 17:42:32 2016 +0200
summary:
  Issue #28666: Now test.support.rmtree is able to remove unwritable or
unreadable directories on Windows too.

files:
  Lib/test/support/__init__.py |  31 ++++++++++++-----------
  1 files changed, 16 insertions(+), 15 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -297,6 +297,16 @@
     except KeyError:
         pass
 
+def _force_run(path, func, *args):
+    try:
+        return func(*args)
+    except OSError as err:
+        if verbose >= 2:
+            print('%s: %s' % (err.__class__.__name__, err))
+            print('re-run %s%r' % (func.__name__, args))
+        os.chmod(path, stat.S_IRWXU)
+        return func(*args)
+
 if sys.platform.startswith("win"):
     def _waitfor(func, pathname, waitall=False):
         # Perform the operation
@@ -339,7 +349,7 @@
 
     def _rmtree(path):
         def _rmtree_inner(path):
-            for name in os.listdir(path):
+            for name in _force_run(path, os.listdir, path):
                 fullname = os.path.join(path, name)
                 try:
                     mode = os.lstat(fullname).st_mode
@@ -349,9 +359,9 @@
                     mode = 0
                 if stat.S_ISDIR(mode):
                     _waitfor(_rmtree_inner, fullname, waitall=True)
-                    os.rmdir(fullname)
+                    _force_run(path, os.rmdir, fullname)
                 else:
-                    os.unlink(fullname)
+                    _force_run(path, os.unlink, fullname)
         _waitfor(_rmtree_inner, path, waitall=True)
         _waitfor(os.rmdir, path)
 else:
@@ -365,17 +375,8 @@
         except OSError:
             pass
 
-        def force_run(path, func, *args):
-            try:
-                return func(*args)
-            except OSError as err:
-                if verbose >= 2:
-                    print('%s: %s' % (err.__class__.__name__, err))
-                    print('re-run %s%r' % (func.__name__, args))
-                os.chmod(path, stat.S_IRWXU)
-                return func(*args)
         def _rmtree_inner(path):
-            for name in force_run(path, os.listdir, path):
+            for name in _force_run(path, os.listdir, path):
                 fullname = os.path.join(path, name)
                 try:
                     mode = os.lstat(fullname).st_mode
@@ -383,9 +384,9 @@
                     mode = 0
                 if stat.S_ISDIR(mode):
                     _rmtree_inner(fullname)
-                    force_run(path, os.rmdir, fullname)
+                    _force_run(path, os.rmdir, fullname)
                 else:
-                    force_run(path, os.unlink, fullname)
+                    _force_run(path, os.unlink, fullname)
         _rmtree_inner(path)
         os.rmdir(path)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 10:43:24 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 15:43:24 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328666=3A_Now_test=2Esupport=2Ermtree_is_able_to_remov?=
 =?utf-8?q?e_unwritable_or?=
Message-ID: <20161120154323.12257.84540.7BE2C557@psf.io>

https://hg.python.org/cpython/rev/82ca763882f5
changeset:   105224:82ca763882f5
branch:      3.6
parent:      105220:0c6fccf04a79
parent:      105223:9e23b8996584
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 17:42:58 2016 +0200
summary:
  Issue #28666: Now test.support.rmtree is able to remove unwritable or
unreadable directories on Windows too.

files:
  Lib/test/support/__init__.py |  31 ++++++++++++-----------
  1 files changed, 16 insertions(+), 15 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -298,6 +298,16 @@
     except KeyError:
         pass
 
+def _force_run(path, func, *args):
+    try:
+        return func(*args)
+    except OSError as err:
+        if verbose >= 2:
+            print('%s: %s' % (err.__class__.__name__, err))
+            print('re-run %s%r' % (func.__name__, args))
+        os.chmod(path, stat.S_IRWXU)
+        return func(*args)
+
 if sys.platform.startswith("win"):
     def _waitfor(func, pathname, waitall=False):
         # Perform the operation
@@ -340,7 +350,7 @@
 
     def _rmtree(path):
         def _rmtree_inner(path):
-            for name in os.listdir(path):
+            for name in _force_run(path, os.listdir, path):
                 fullname = os.path.join(path, name)
                 try:
                     mode = os.lstat(fullname).st_mode
@@ -350,9 +360,9 @@
                     mode = 0
                 if stat.S_ISDIR(mode):
                     _waitfor(_rmtree_inner, fullname, waitall=True)
-                    os.rmdir(fullname)
+                    _force_run(path, os.rmdir, fullname)
                 else:
-                    os.unlink(fullname)
+                    _force_run(path, os.unlink, fullname)
         _waitfor(_rmtree_inner, path, waitall=True)
         _waitfor(os.rmdir, path)
 else:
@@ -366,17 +376,8 @@
         except OSError:
             pass
 
-        def force_run(path, func, *args):
-            try:
-                return func(*args)
-            except OSError as err:
-                if verbose >= 2:
-                    print('%s: %s' % (err.__class__.__name__, err))
-                    print('re-run %s%r' % (func.__name__, args))
-                os.chmod(path, stat.S_IRWXU)
-                return func(*args)
         def _rmtree_inner(path):
-            for name in force_run(path, os.listdir, path):
+            for name in _force_run(path, os.listdir, path):
                 fullname = os.path.join(path, name)
                 try:
                     mode = os.lstat(fullname).st_mode
@@ -384,9 +385,9 @@
                     mode = 0
                 if stat.S_ISDIR(mode):
                     _rmtree_inner(fullname)
-                    force_run(path, os.rmdir, fullname)
+                    _force_run(path, os.rmdir, fullname)
                 else:
-                    force_run(path, os.unlink, fullname)
+                    _force_run(path, os.unlink, fullname)
         _rmtree_inner(path)
         os.rmdir(path)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 10:43:23 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 15:43:23 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4NjY2?=
 =?utf-8?q?=3A_Now_test=2Etest=5Fsupport=2Ermtree_is_able_to_remove_unwrit?=
 =?utf-8?q?able_or?=
Message-ID: <20161120154323.40969.83160.D660FFE5@psf.io>

https://hg.python.org/cpython/rev/b51607ea54c5
changeset:   105222:b51607ea54c5
branch:      2.7
parent:      105215:63820871014d
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 17:42:03 2016 +0200
summary:
  Issue #28666: Now test.test_support.rmtree is able to remove unwritable or
unreadable directories on Windows too.

files:
  Lib/test/test_support.py |  31 ++++++++++++++-------------
  1 files changed, 16 insertions(+), 15 deletions(-)


diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py
--- a/Lib/test/test_support.py
+++ b/Lib/test/test_support.py
@@ -182,6 +182,16 @@
     except KeyError:
         pass
 
+def _force_run(path, func, *args):
+    try:
+        return func(*args)
+    except EnvironmentError as err:
+        if verbose >= 2:
+            print('%s: %s' % (err.__class__.__name__, err))
+            print('re-run %s%r' % (func.__name__, args))
+        os.chmod(path, stat.S_IRWXU)
+        return func(*args)
+
 if sys.platform.startswith("win"):
     def _waitfor(func, pathname, waitall=False):
         # Perform the operation
@@ -224,13 +234,13 @@
 
     def _rmtree(path):
         def _rmtree_inner(path):
-            for name in os.listdir(path):
+            for name in _force_run(path, os.listdir, path):
                 fullname = os.path.join(path, name)
                 if os.path.isdir(fullname):
                     _waitfor(_rmtree_inner, fullname, waitall=True)
-                    os.rmdir(fullname)
+                    _force_run(path, os.rmdir, fullname)
                 else:
-                    os.unlink(fullname)
+                    _force_run(path, os.unlink, fullname)
         _waitfor(_rmtree_inner, path, waitall=True)
         _waitfor(os.rmdir, path)
 else:
@@ -245,17 +255,8 @@
         except EnvironmentError:
             pass
 
-        def force_run(path, func, *args):
-            try:
-                return func(*args)
-            except EnvironmentError as err:
-                if verbose >= 2:
-                    print('%s: %s' % (err.__class__.__name__, err))
-                    print('re-run %s%r' % (func.__name__, args))
-                os.chmod(path, stat.S_IRWXU)
-                return func(*args)
         def _rmtree_inner(path):
-            for name in force_run(path, os.listdir, path):
+            for name in _force_run(path, os.listdir, path):
                 fullname = os.path.join(path, name)
                 try:
                     mode = os.lstat(fullname).st_mode
@@ -263,9 +264,9 @@
                     mode = 0
                 if stat.S_ISDIR(mode):
                     _rmtree_inner(fullname)
-                    force_run(path, os.rmdir, fullname)
+                    _force_run(path, os.rmdir, fullname)
                 else:
-                    force_run(path, os.unlink, fullname)
+                    _force_run(path, os.unlink, fullname)
         _rmtree_inner(path)
         os.rmdir(path)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 10:43:24 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 15:43:24 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328666=3A_Now_test=2Esupport=2Ermtree_is_able_to?=
 =?utf-8?q?_remove_unwritable_or?=
Message-ID: <20161120154323.12277.49092.ADA963D3@psf.io>

https://hg.python.org/cpython/rev/593ec9658f4b
changeset:   105225:593ec9658f4b
parent:      105221:deff7bf26d00
parent:      105224:82ca763882f5
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 17:43:09 2016 +0200
summary:
  Issue #28666: Now test.support.rmtree is able to remove unwritable or
unreadable directories on Windows too.

files:
  Lib/test/support/__init__.py |  31 ++++++++++++-----------
  1 files changed, 16 insertions(+), 15 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -298,6 +298,16 @@
     except KeyError:
         pass
 
+def _force_run(path, func, *args):
+    try:
+        return func(*args)
+    except OSError as err:
+        if verbose >= 2:
+            print('%s: %s' % (err.__class__.__name__, err))
+            print('re-run %s%r' % (func.__name__, args))
+        os.chmod(path, stat.S_IRWXU)
+        return func(*args)
+
 if sys.platform.startswith("win"):
     def _waitfor(func, pathname, waitall=False):
         # Perform the operation
@@ -340,7 +350,7 @@
 
     def _rmtree(path):
         def _rmtree_inner(path):
-            for name in os.listdir(path):
+            for name in _force_run(path, os.listdir, path):
                 fullname = os.path.join(path, name)
                 try:
                     mode = os.lstat(fullname).st_mode
@@ -350,9 +360,9 @@
                     mode = 0
                 if stat.S_ISDIR(mode):
                     _waitfor(_rmtree_inner, fullname, waitall=True)
-                    os.rmdir(fullname)
+                    _force_run(path, os.rmdir, fullname)
                 else:
-                    os.unlink(fullname)
+                    _force_run(path, os.unlink, fullname)
         _waitfor(_rmtree_inner, path, waitall=True)
         _waitfor(os.rmdir, path)
 else:
@@ -366,17 +376,8 @@
         except OSError:
             pass
 
-        def force_run(path, func, *args):
-            try:
-                return func(*args)
-            except OSError as err:
-                if verbose >= 2:
-                    print('%s: %s' % (err.__class__.__name__, err))
-                    print('re-run %s%r' % (func.__name__, args))
-                os.chmod(path, stat.S_IRWXU)
-                return func(*args)
         def _rmtree_inner(path):
-            for name in force_run(path, os.listdir, path):
+            for name in _force_run(path, os.listdir, path):
                 fullname = os.path.join(path, name)
                 try:
                     mode = os.lstat(fullname).st_mode
@@ -384,9 +385,9 @@
                     mode = 0
                 if stat.S_ISDIR(mode):
                     _rmtree_inner(fullname)
-                    force_run(path, os.rmdir, fullname)
+                    _force_run(path, os.rmdir, fullname)
                 else:
-                    force_run(path, os.unlink, fullname)
+                    _force_run(path, os.unlink, fullname)
         _rmtree_inner(path)
         os.rmdir(path)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 13:05:18 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 18:05:18 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328666=3A_Try_to_f?=
 =?utf-8?q?ix_removing_readonly_directories_on_Windows=2E?=
Message-ID: <20161120180518.14307.7406.F24C3E70@psf.io>

https://hg.python.org/cpython/rev/da1880183693
changeset:   105226:da1880183693
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 20:04:54 2016 +0200
summary:
  Issue #28666: Try to fix removing readonly directories on Windows.

files:
  Lib/test/support/__init__.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -364,7 +364,7 @@
                 else:
                     _force_run(path, os.unlink, fullname)
         _waitfor(_rmtree_inner, path, waitall=True)
-        _waitfor(os.rmdir, path)
+        _waitfor(lambda p: _force_run(p, os.rmdir, p), path)
 else:
     _unlink = os.unlink
     _rmdir = os.rmdir

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 13:38:03 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 18:38:03 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328666=3A_Fix_removing_readonly_directories_on_Windows?=
 =?utf-8?q?=2E?=
Message-ID: <20161120183803.7431.26539.2EA09C90@psf.io>

https://hg.python.org/cpython/rev/b9e1a51a2d19
changeset:   105229:b9e1a51a2d19
branch:      3.6
parent:      105224:82ca763882f5
parent:      105227:573fd9607c75
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 20:36:51 2016 +0200
summary:
  Issue #28666: Fix removing readonly directories on Windows.

files:
  Lib/test/support/__init__.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -364,7 +364,7 @@
                 else:
                     _force_run(path, os.unlink, fullname)
         _waitfor(_rmtree_inner, path, waitall=True)
-        _waitfor(os.rmdir, path)
+        _waitfor(lambda p: _force_run(p, os.rmdir, p), path)
 else:
     _unlink = os.unlink
     _rmdir = os.rmdir

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 13:38:03 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 18:38:03 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Null_merge?=
Message-ID: <20161120183803.45298.37252.FB757DFD@psf.io>

https://hg.python.org/cpython/rev/021f3b32e11f
changeset:   105230:021f3b32e11f
parent:      105226:da1880183693
parent:      105229:b9e1a51a2d19
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 20:37:08 2016 +0200
summary:
  Null merge

files:



-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 13:38:03 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 18:38:03 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4NjY2?=
 =?utf-8?q?=3A_Fix_removing_readonly_directories_on_Windows=2E?=
Message-ID: <20161120183803.14044.71982.96F6FD02@psf.io>

https://hg.python.org/cpython/rev/01f867e9cd34
changeset:   105228:01f867e9cd34
branch:      2.7
parent:      105222:b51607ea54c5
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 20:04:54 2016 +0200
summary:
  Issue #28666: Fix removing readonly directories on Windows.

files:
  Lib/test/test_support.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py
--- a/Lib/test/test_support.py
+++ b/Lib/test/test_support.py
@@ -242,7 +242,7 @@
                 else:
                     _force_run(path, os.unlink, fullname)
         _waitfor(_rmtree_inner, path, waitall=True)
-        _waitfor(os.rmdir, path)
+        _waitfor(lambda p: _force_run(p, os.rmdir, p), path)
 else:
     _unlink = os.unlink
     _rmdir = os.rmdir

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 13:38:03 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 20 Nov 2016 18:38:03 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjY2?=
 =?utf-8?q?=3A_Fix_removing_readonly_directories_on_Windows=2E?=
Message-ID: <20161120183803.10046.53250.5AE7F4C6@psf.io>

https://hg.python.org/cpython/rev/573fd9607c75
changeset:   105227:573fd9607c75
branch:      3.5
parent:      105223:9e23b8996584
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Nov 20 20:04:54 2016 +0200
summary:
  Issue #28666: Fix removing readonly directories on Windows.

files:
  Lib/test/support/__init__.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -363,7 +363,7 @@
                 else:
                     _force_run(path, os.unlink, fullname)
         _waitfor(_rmtree_inner, path, waitall=True)
-        _waitfor(os.rmdir, path)
+        _waitfor(lambda p: _force_run(p, os.rmdir, p), path)
 else:
     _unlink = os.unlink
     _rmdir = os.rmdir

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 14:33:29 2016
From: python-checkins at python.org (larry.hastings)
Date: Sun, 20 Nov 2016 19:33:29 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2Uu?=
Message-ID: <20161120193329.14203.14751.8DD95D41@psf.io>

https://hg.python.org/cpython/rev/8a4eaab4eeb3
changeset:   105232:8a4eaab4eeb3
parent:      105231:0a18d2cfeb52
parent:      105230:021f3b32e11f
user:        Larry Hastings <larry at hastings.org>
date:        Sun Nov 20 11:33:15 2016 -0800
summary:
  Merge.

files:



-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 14:33:29 2016
From: python-checkins at python.org (larry.hastings)
Date: Sun, 20 Nov 2016 19:33:29 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_28753=3A_Argument_Cl?=
 =?utf-8?q?inic_howto_docfix=2C_courtesy_Julien_Palard=2E?=
Message-ID: <20161120193329.45859.60786.ABF2DE7D@psf.io>

https://hg.python.org/cpython/rev/0a18d2cfeb52
changeset:   105231:0a18d2cfeb52
parent:      105226:da1880183693
user:        Larry Hastings <larry at hastings.org>
date:        Sun Nov 20 11:30:36 2016 -0800
summary:
  Issue 28753: Argument Clinic howto docfix, courtesy Julien Palard.

files:
  Doc/howto/clinic.rst |  29 +++++++++++++++--------------
  1 files changed, 15 insertions(+), 14 deletions(-)


diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst
--- a/Doc/howto/clinic.rst
+++ b/Doc/howto/clinic.rst
@@ -375,15 +375,12 @@
         Write a pickled representation of obj to the open file.
         [clinic start generated code]*/
 
-12. Save and close the file, then run ``Tools/clinic/clinic.py`` on it.
-    With luck everything worked and your block now has output!  Reopen
-    the file in your text editor to see::
+12. Save and close the file, then run ``Tools/clinic/clinic.py`` on
+    it.  With luck everything worked--your block now has output, and
+    a ``.c.h`` file has been generated! Reopen the file in your
+    text editor to see:
 
-       /*[clinic input]
-       module _pickle
-       class _pickle.Pickler "PicklerObject *" "&Pickler_Type"
-       [clinic start generated code]*/
-       /*[clinic end generated code: checksum=da39a3ee5e6b4b0d3255bfef95601890afd80709]*/
+    .. code-block:: c
 
        /*[clinic input]
        _pickle.Pickler.dump
@@ -395,18 +392,22 @@
        Write a pickled representation of obj to the open file.
        [clinic start generated code]*/
 
-       PyDoc_STRVAR(_pickle_Pickler_dump__doc__,
-       "Write a pickled representation of obj to the open file.\n"
-       "\n"
-       ...
        static PyObject *
-       _pickle_Pickler_dump_impl(PicklerObject *self, PyObject *obj)
-       /*[clinic end generated code: checksum=3bd30745bf206a48f8b576a1da3d90f55a0a4187]*/
+       _pickle_Pickler_dump(PicklerObject *self, PyObject *obj)
+       /*[clinic end generated code: output=87ecad1261e02ac7 input=552eb1c0f52260d9]*/
 
     Obviously, if Argument Clinic didn't produce any output, it's because
     it found an error in your input.  Keep fixing your errors and retrying
     until Argument Clinic processes your file without complaint.
 
+    For readability, most of the glue code has been generated to a ``.c.h``
+    file.  You'll need to include that in your original ``.c`` file,
+    typically right after the clinic module block:
+
+     .. code-block:: c
+
+       #include "clinic/_pickle.c.h"
+
 13. Double-check that the argument-parsing code Argument Clinic generated
     looks basically the same as the existing code.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 16:19:52 2016
From: python-checkins at python.org (gregory.p.smith)
Date: Sun, 20 Nov 2016 21:19:52 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Rename_the_new?=
 =?utf-8?q?_--with-optimiations_flag_to_--enable-optimizations=2E?=
Message-ID: <20161120211952.45580.71359.A883B38D@psf.io>

https://hg.python.org/cpython/rev/c0ea81315fb6
changeset:   105233:c0ea81315fb6
branch:      3.5
parent:      105227:573fd9607c75
user:        Gregory P. Smith <greg at krypto.org> [Google Inc.]
date:        Sun Nov 20 21:07:42 2016 +0000
summary:
  Rename the new --with-optimiations flag to --enable-optimizations.

files:
  Misc/NEWS    |   2 ++
  configure    |  23 +++++++++++------------
  configure.ac |  10 +++++-----
  3 files changed, 18 insertions(+), 17 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -555,6 +555,8 @@
 Build
 -----
 
+- Issue #26359: Rename --with-optimiations to --enable-optimizations.
+
 - Issue #28444: Fix missing extensions modules when cross compiling.
 
 - Issue #28248: Update Windows build and OS X installers to use OpenSSL 1.0.2j.
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -815,7 +815,7 @@
 enable_shared
 enable_profiling
 with_pydebug
-with_optimizations
+enable_optimizations
 with_lto
 with_hash_algorithm
 with_address_sanitizer
@@ -1486,6 +1486,8 @@
                           Build (MacOSX|Darwin) framework
   --enable-shared         disable/enable building shared python library
   --enable-profiling      enable C-level code profiling
+  --enable-optimizations  Enable expensive optimizations (PGO, maybe LTO,
+                          etc). Disabled by default.
   --enable-loadable-sqlite-extensions
                           support loadable extensions in _sqlite module
   --enable-ipv6           Enable ipv6 (with ipv4) support
@@ -1509,8 +1511,6 @@
                           compiler
   --with-suffix=.exe      set executable suffix
   --with-pydebug          build with Py_DEBUG defined
-  --with-optimizations    Enable expensive optimizations (PGO, maybe LTO,
-                          etc). Disabled by default.
   --with-lto              Enable Link Time Optimization in PGO builds.
                           Disabled by default.
   --with-hash-algorithm=[fnv|siphash24]
@@ -6543,12 +6543,11 @@
 
 
 Py_OPT='false'
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-optimizations" >&5
-$as_echo_n "checking for --with-optimizations... " >&6; }
-
-# Check whether --with-optimizations was given.
-if test "${with_optimizations+set}" = set; then :
-  withval=$with_optimizations;
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --enable-optimizations" >&5
+$as_echo_n "checking for --enable-optimizations... " >&6; }
+# Check whether --enable-optimizations was given.
+if test "${enable_optimizations+set}" = set; then :
+  enableval=$enable_optimizations;
 if test "$withval" != no
 then
   Py_OPT='true'
@@ -6808,7 +6807,7 @@
       LLVM_PROF_ERR=yes
       if test "${REQUIRE_PGO}" = "yes"
       then
-        as_fn_error $? "llvm-profdata is required for a --with-optimizations build but could not be found." "$LINENO" 5
+        as_fn_error $? "llvm-profdata is required for a --enable-optimizations build but could not be found." "$LINENO" 5
       fi
     fi
     ;;
@@ -6824,7 +6823,7 @@
           LLVM_PROF_ERR=yes
           if test "${REQUIRE_PGO}" = "yes"
 	  then
-	    as_fn_error $? "llvm-profdata is required for a --with-optimizations build but could not be found." "$LINENO" 5
+	    as_fn_error $? "llvm-profdata is required for a --enable-optimizations build but could not be found." "$LINENO" 5
 	  fi
         fi
         ;;
@@ -17829,7 +17828,7 @@
     echo "" >&6
     echo "" >&6
     echo "If you want a release build with all optimizations active (LTO, PGO, etc),"
-    echo "please run ./configure --with-optimizations" >&6
+    echo "please run ./configure --enable-optimizations" >&6
     echo "" >&6
     echo "" >&6
 fi
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -1234,8 +1234,8 @@
 AC_SUBST(DEF_MAKE_ALL_RULE)
 AC_SUBST(DEF_MAKE_RULE)
 Py_OPT='false'
-AC_MSG_CHECKING(for --with-optimizations)
-AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable expensive optimizations (PGO, maybe LTO, etc).  Disabled by default.]),
+AC_MSG_CHECKING(for --enable-optimizations)
+AC_ARG_ENABLE(optimizations, AS_HELP_STRING([--enable-optimizations], [Enable expensive optimizations (PGO, maybe LTO, etc).  Disabled by default.]),
 [
 if test "$withval" != no
 then
@@ -1351,7 +1351,7 @@
       LLVM_PROF_ERR=yes
       if test "${REQUIRE_PGO}" = "yes"
       then
-        AC_MSG_ERROR([llvm-profdata is required for a --with-optimizations build but could not be found.])
+        AC_MSG_ERROR([llvm-profdata is required for a --enable-optimizations build but could not be found.])
       fi
     fi
     ;;
@@ -1367,7 +1367,7 @@
           LLVM_PROF_ERR=yes
           if test "${REQUIRE_PGO}" = "yes"
 	  then
-	    AC_MSG_ERROR([llvm-profdata is required for a --with-optimizations build but could not be found.])
+	    AC_MSG_ERROR([llvm-profdata is required for a --enable-optimizations build but could not be found.])
 	  fi
         fi
         ;;
@@ -5379,7 +5379,7 @@
     echo "" >&AS_MESSAGE_FD
     echo "" >&AS_MESSAGE_FD
     echo "If you want a release build with all optimizations active (LTO, PGO, etc),"
-    echo "please run ./configure --with-optimizations" >&AS_MESSAGE_FD
+    echo "please run ./configure --enable-optimizations" >&AS_MESSAGE_FD
     echo "" >&AS_MESSAGE_FD
     echo "" >&AS_MESSAGE_FD
 fi

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 16:19:52 2016
From: python-checkins at python.org (gregory.p.smith)
Date: Sun, 20 Nov 2016 21:19:52 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Rename_the_new_--with-optimiations_flag_to_--enable-opti?=
 =?utf-8?q?mizations=2E?=
Message-ID: <20161120211952.28084.68543.5FBAF935@psf.io>

https://hg.python.org/cpython/rev/11cacf9f9a33
changeset:   105235:11cacf9f9a33
parent:      105232:8a4eaab4eeb3
parent:      105234:58c1a49a10b4
user:        Gregory P. Smith <greg at krypto.org> [Google Inc.]
date:        Sun Nov 20 21:17:01 2016 +0000
summary:
  Rename the new --with-optimiations flag to --enable-optimizations.

files:
  Misc/NEWS    |   2 ++
  configure    |  37 ++++++++++++++++++++++++-------------
  configure.ac |  10 +++++-----
  3 files changed, 31 insertions(+), 18 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -425,6 +425,8 @@
 Build
 -----
 
+- Issue #26359: Rename --with-optimiations to --enable-optimizations.
+
 - Issue #28444: Fix missing extensions modules when cross compiling.
 
 - Issue #28208: Update Windows build and OS X installers to use SQLite 3.14.2.
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -784,6 +784,7 @@
 docdir
 oldincludedir
 includedir
+runstatedir
 localstatedir
 sharedstatedir
 sysconfdir
@@ -817,7 +818,7 @@
 enable_shared
 enable_profiling
 with_pydebug
-with_optimizations
+enable_optimizations
 with_lto
 with_hash_algorithm
 with_address_sanitizer
@@ -894,6 +895,7 @@
 sysconfdir='${prefix}/etc'
 sharedstatedir='${prefix}/com'
 localstatedir='${prefix}/var'
+runstatedir='${localstatedir}/run'
 includedir='${prefix}/include'
 oldincludedir='/usr/include'
 docdir='${datarootdir}/doc/${PACKAGE_TARNAME}'
@@ -1146,6 +1148,15 @@
   | -silent | --silent | --silen | --sile | --sil)
     silent=yes ;;
 
+  -runstatedir | --runstatedir | --runstatedi | --runstated \
+  | --runstate | --runstat | --runsta | --runst | --runs \
+  | --run | --ru | --r)
+    ac_prev=runstatedir ;;
+  -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \
+  | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \
+  | --run=* | --ru=* | --r=*)
+    runstatedir=$ac_optarg ;;
+
   -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb)
     ac_prev=sbindir ;;
   -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \
@@ -1283,7 +1294,7 @@
 for ac_var in	exec_prefix prefix bindir sbindir libexecdir datarootdir \
 		datadir sysconfdir sharedstatedir localstatedir includedir \
 		oldincludedir docdir infodir htmldir dvidir pdfdir psdir \
-		libdir localedir mandir
+		libdir localedir mandir runstatedir
 do
   eval ac_val=\$$ac_var
   # Remove trailing slashes.
@@ -1436,6 +1447,7 @@
   --sysconfdir=DIR        read-only single-machine data [PREFIX/etc]
   --sharedstatedir=DIR    modifiable architecture-independent data [PREFIX/com]
   --localstatedir=DIR     modifiable single-machine data [PREFIX/var]
+  --runstatedir=DIR       modifiable per-process data [LOCALSTATEDIR/run]
   --libdir=DIR            object code libraries [EPREFIX/lib]
   --includedir=DIR        C header files [PREFIX/include]
   --oldincludedir=DIR     C header files for non-gcc [/usr/include]
@@ -1476,6 +1488,8 @@
                           Build (MacOSX|Darwin) framework
   --enable-shared         disable/enable building shared python library
   --enable-profiling      enable C-level code profiling
+  --enable-optimizations  Enable expensive optimizations (PGO, etc). Disabled
+                          by default.
   --enable-loadable-sqlite-extensions
                           support loadable extensions in _sqlite module
   --enable-ipv6           Enable ipv6 (with ipv4) support
@@ -1499,8 +1513,6 @@
                           compiler
   --with-suffix=.exe      set executable suffix
   --with-pydebug          build with Py_DEBUG defined
-  --with-optimizations    Enable expensive optimizations (PGO, etc). Disabled
-                          by default.
   --with-lto              Enable Link Time Optimization in PGO builds.
                           Disabled by default.
   --with-hash-algorithm=[fnv|siphash24]
@@ -6524,12 +6536,11 @@
 
 
 Py_OPT='false'
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-optimizations" >&5
-$as_echo_n "checking for --with-optimizations... " >&6; }
-
-# Check whether --with-optimizations was given.
-if test "${with_optimizations+set}" = set; then :
-  withval=$with_optimizations;
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --enable-optimizations" >&5
+$as_echo_n "checking for --enable-optimizations... " >&6; }
+# Check whether --enable-optimizations was given.
+if test "${enable_optimizations+set}" = set; then :
+  enableval=$enable_optimizations;
 if test "$withval" != no
 then
   Py_OPT='true'
@@ -6796,7 +6807,7 @@
       LLVM_PROF_ERR=yes
       if test "${REQUIRE_PGO}" = "yes"
       then
-        as_fn_error $? "llvm-profdata is required for a --with-optimizations build but could not be found." "$LINENO" 5
+        as_fn_error $? "llvm-profdata is required for a --enable-optimizations build but could not be found." "$LINENO" 5
       fi
     fi
     ;;
@@ -6812,7 +6823,7 @@
           LLVM_PROF_ERR=yes
           if test "${REQUIRE_PGO}" = "yes"
 	  then
-	    as_fn_error $? "llvm-profdata is required for a --with-optimizations build but could not be found." "$LINENO" 5
+	    as_fn_error $? "llvm-profdata is required for a --enable-optimizations build but could not be found." "$LINENO" 5
 	  fi
         fi
         ;;
@@ -17877,7 +17888,7 @@
     echo "" >&6
     echo "" >&6
     echo "If you want a release build with all optimizations active (LTO, PGO, etc)," >&6
-    echo "please run ./configure --with-optimizations" >&6
+    echo "please run ./configure --enable-optimizations" >&6
     echo "" >&6
     echo "" >&6
 fi
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -1281,8 +1281,8 @@
 AC_SUBST(DEF_MAKE_ALL_RULE)
 AC_SUBST(DEF_MAKE_RULE)
 Py_OPT='false'
-AC_MSG_CHECKING(for --with-optimizations)
-AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable expensive optimizations (PGO, etc).  Disabled by default.]),
+AC_MSG_CHECKING(for --enable-optimizations)
+AC_ARG_ENABLE(optimizations, AS_HELP_STRING([--enable-optimizations], [Enable expensive optimizations (PGO, etc).  Disabled by default.]),
 [
 if test "$withval" != no
 then
@@ -1405,7 +1405,7 @@
       LLVM_PROF_ERR=yes
       if test "${REQUIRE_PGO}" = "yes"
       then
-        AC_MSG_ERROR([llvm-profdata is required for a --with-optimizations build but could not be found.])
+        AC_MSG_ERROR([llvm-profdata is required for a --enable-optimizations build but could not be found.])
       fi
     fi
     ;;
@@ -1421,7 +1421,7 @@
           LLVM_PROF_ERR=yes
           if test "${REQUIRE_PGO}" = "yes"
 	  then
-	    AC_MSG_ERROR([llvm-profdata is required for a --with-optimizations build but could not be found.])
+	    AC_MSG_ERROR([llvm-profdata is required for a --enable-optimizations build but could not be found.])
 	  fi
         fi
         ;;
@@ -5399,7 +5399,7 @@
     echo "" >&AS_MESSAGE_FD
     echo "" >&AS_MESSAGE_FD
     echo "If you want a release build with all optimizations active (LTO, PGO, etc)," >&AS_MESSAGE_FD
-    echo "please run ./configure --with-optimizations" >&AS_MESSAGE_FD
+    echo "please run ./configure --enable-optimizations" >&AS_MESSAGE_FD
     echo "" >&AS_MESSAGE_FD
     echo "" >&AS_MESSAGE_FD
 fi

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 16:19:52 2016
From: python-checkins at python.org (gregory.p.smith)
Date: Sun, 20 Nov 2016 21:19:52 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Rename_the_new_--with-optimiations_flag_to_--enable-optimizati?=
 =?utf-8?q?ons=2E?=
Message-ID: <20161120211952.46126.5277.CA46A269@psf.io>

https://hg.python.org/cpython/rev/58c1a49a10b4
changeset:   105234:58c1a49a10b4
branch:      3.6
parent:      105229:b9e1a51a2d19
parent:      105233:c0ea81315fb6
user:        Gregory P. Smith <greg at krypto.org> [Google Inc.]
date:        Sun Nov 20 21:13:16 2016 +0000
summary:
  Rename the new --with-optimiations flag to --enable-optimizations.

files:
  Misc/NEWS    |   2 ++
  configure    |  37 ++++++++++++++++++++++++-------------
  configure.ac |  10 +++++-----
  3 files changed, 31 insertions(+), 18 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -931,6 +931,8 @@
 Build
 -----
 
+- Issue #26359: Rename --with-optimiations to --enable-optimizations.
+
 - Issue #27566: Fix clean target in freeze makefile (patch by Lisa Roach)
 
 - Issue #27705: Update message in validate_ucrtbase.py
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -784,6 +784,7 @@
 docdir
 oldincludedir
 includedir
+runstatedir
 localstatedir
 sharedstatedir
 sysconfdir
@@ -817,7 +818,7 @@
 enable_shared
 enable_profiling
 with_pydebug
-with_optimizations
+enable_optimizations
 with_lto
 with_hash_algorithm
 with_address_sanitizer
@@ -894,6 +895,7 @@
 sysconfdir='${prefix}/etc'
 sharedstatedir='${prefix}/com'
 localstatedir='${prefix}/var'
+runstatedir='${localstatedir}/run'
 includedir='${prefix}/include'
 oldincludedir='/usr/include'
 docdir='${datarootdir}/doc/${PACKAGE_TARNAME}'
@@ -1146,6 +1148,15 @@
   | -silent | --silent | --silen | --sile | --sil)
     silent=yes ;;
 
+  -runstatedir | --runstatedir | --runstatedi | --runstated \
+  | --runstate | --runstat | --runsta | --runst | --runs \
+  | --run | --ru | --r)
+    ac_prev=runstatedir ;;
+  -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \
+  | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \
+  | --run=* | --ru=* | --r=*)
+    runstatedir=$ac_optarg ;;
+
   -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb)
     ac_prev=sbindir ;;
   -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \
@@ -1283,7 +1294,7 @@
 for ac_var in	exec_prefix prefix bindir sbindir libexecdir datarootdir \
 		datadir sysconfdir sharedstatedir localstatedir includedir \
 		oldincludedir docdir infodir htmldir dvidir pdfdir psdir \
-		libdir localedir mandir
+		libdir localedir mandir runstatedir
 do
   eval ac_val=\$$ac_var
   # Remove trailing slashes.
@@ -1436,6 +1447,7 @@
   --sysconfdir=DIR        read-only single-machine data [PREFIX/etc]
   --sharedstatedir=DIR    modifiable architecture-independent data [PREFIX/com]
   --localstatedir=DIR     modifiable single-machine data [PREFIX/var]
+  --runstatedir=DIR       modifiable per-process data [LOCALSTATEDIR/run]
   --libdir=DIR            object code libraries [EPREFIX/lib]
   --includedir=DIR        C header files [PREFIX/include]
   --oldincludedir=DIR     C header files for non-gcc [/usr/include]
@@ -1476,6 +1488,8 @@
                           Build (MacOSX|Darwin) framework
   --enable-shared         disable/enable building shared python library
   --enable-profiling      enable C-level code profiling
+  --enable-optimizations  Enable expensive optimizations (PGO, etc). Disabled
+                          by default.
   --enable-loadable-sqlite-extensions
                           support loadable extensions in _sqlite module
   --enable-ipv6           Enable ipv6 (with ipv4) support
@@ -1499,8 +1513,6 @@
                           compiler
   --with-suffix=.exe      set executable suffix
   --with-pydebug          build with Py_DEBUG defined
-  --with-optimizations    Enable expensive optimizations (PGO, etc). Disabled
-                          by default.
   --with-lto              Enable Link Time Optimization in PGO builds.
                           Disabled by default.
   --with-hash-algorithm=[fnv|siphash24]
@@ -6524,12 +6536,11 @@
 
 
 Py_OPT='false'
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-optimizations" >&5
-$as_echo_n "checking for --with-optimizations... " >&6; }
-
-# Check whether --with-optimizations was given.
-if test "${with_optimizations+set}" = set; then :
-  withval=$with_optimizations;
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --enable-optimizations" >&5
+$as_echo_n "checking for --enable-optimizations... " >&6; }
+# Check whether --enable-optimizations was given.
+if test "${enable_optimizations+set}" = set; then :
+  enableval=$enable_optimizations;
 if test "$withval" != no
 then
   Py_OPT='true'
@@ -6796,7 +6807,7 @@
       LLVM_PROF_ERR=yes
       if test "${REQUIRE_PGO}" = "yes"
       then
-        as_fn_error $? "llvm-profdata is required for a --with-optimizations build but could not be found." "$LINENO" 5
+        as_fn_error $? "llvm-profdata is required for a --enable-optimizations build but could not be found." "$LINENO" 5
       fi
     fi
     ;;
@@ -6812,7 +6823,7 @@
           LLVM_PROF_ERR=yes
           if test "${REQUIRE_PGO}" = "yes"
 	  then
-	    as_fn_error $? "llvm-profdata is required for a --with-optimizations build but could not be found." "$LINENO" 5
+	    as_fn_error $? "llvm-profdata is required for a --enable-optimizations build but could not be found." "$LINENO" 5
 	  fi
         fi
         ;;
@@ -17861,7 +17872,7 @@
     echo "" >&6
     echo "" >&6
     echo "If you want a release build with all optimizations active (LTO, PGO, etc)," >&6
-    echo "please run ./configure --with-optimizations" >&6
+    echo "please run ./configure --enable-optimizations" >&6
     echo "" >&6
     echo "" >&6
 fi
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -1281,8 +1281,8 @@
 AC_SUBST(DEF_MAKE_ALL_RULE)
 AC_SUBST(DEF_MAKE_RULE)
 Py_OPT='false'
-AC_MSG_CHECKING(for --with-optimizations)
-AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable expensive optimizations (PGO, etc).  Disabled by default.]),
+AC_MSG_CHECKING(for --enable-optimizations)
+AC_ARG_ENABLE(optimizations, AS_HELP_STRING([--enable-optimizations], [Enable expensive optimizations (PGO, etc).  Disabled by default.]),
 [
 if test "$withval" != no
 then
@@ -1405,7 +1405,7 @@
       LLVM_PROF_ERR=yes
       if test "${REQUIRE_PGO}" = "yes"
       then
-        AC_MSG_ERROR([llvm-profdata is required for a --with-optimizations build but could not be found.])
+        AC_MSG_ERROR([llvm-profdata is required for a --enable-optimizations build but could not be found.])
       fi
     fi
     ;;
@@ -1421,7 +1421,7 @@
           LLVM_PROF_ERR=yes
           if test "${REQUIRE_PGO}" = "yes"
 	  then
-	    AC_MSG_ERROR([llvm-profdata is required for a --with-optimizations build but could not be found.])
+	    AC_MSG_ERROR([llvm-profdata is required for a --enable-optimizations build but could not be found.])
 	  fi
         fi
         ;;
@@ -5395,7 +5395,7 @@
     echo "" >&AS_MESSAGE_FD
     echo "" >&AS_MESSAGE_FD
     echo "If you want a release build with all optimizations active (LTO, PGO, etc)," >&AS_MESSAGE_FD
-    echo "please run ./configure --with-optimizations" >&AS_MESSAGE_FD
+    echo "please run ./configure --enable-optimizations" >&AS_MESSAGE_FD
     echo "" >&AS_MESSAGE_FD
     echo "" >&AS_MESSAGE_FD
 fi

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 16:19:52 2016
From: python-checkins at python.org (gregory.p.smith)
Date: Sun, 20 Nov 2016 21:19:52 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_null_merge?=
Message-ID: <20161120211952.12102.87413.19765466@psf.io>

https://hg.python.org/cpython/rev/2d1d70b53376
changeset:   105237:2d1d70b53376
parent:      105235:11cacf9f9a33
parent:      105236:0d2b42344ae5
user:        Gregory P. Smith <greg at krypto.org> [Google Inc.]
date:        Sun Nov 20 21:17:23 2016 +0000
summary:
  null merge

files:



-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 16:19:52 2016
From: python-checkins at python.org (gregory.p.smith)
Date: Sun, 20 Nov 2016 21:19:52 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_move_the_enabl?=
 =?utf-8?q?e-optimizations_mention_to_the_correct_Build_section=2E?=
Message-ID: <20161120211952.41775.15495.7E496AB9@psf.io>

https://hg.python.org/cpython/rev/0d2b42344ae5
changeset:   105236:0d2b42344ae5
branch:      3.6
parent:      105234:58c1a49a10b4
user:        Gregory P. Smith <greg at krypto.org> [Google Inc.]
date:        Sun Nov 20 21:16:41 2016 +0000
summary:
  move the enable-optimizations mention to the correct Build section.

files:
  Misc/NEWS |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -90,6 +90,8 @@
 Build
 -----
 
+- Issue #26359: Rename --with-optimiations to --enable-optimizations.
+
 - Issue #28676: Prevent missing 'getentropy' declaration warning on macOS.
   Patch by Gareth Rees.
 
@@ -931,8 +933,6 @@
 Build
 -----
 
-- Issue #26359: Rename --with-optimiations to --enable-optimizations.
-
 - Issue #27566: Fix clean target in freeze makefile (patch by Lisa Roach)
 
 - Issue #27705: Update message in validate_ucrtbase.py

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 16:19:52 2016
From: python-checkins at python.org (gregory.p.smith)
Date: Sun, 20 Nov 2016 21:19:52 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Rename_the_new?=
 =?utf-8?q?_--with-optimiations_flag_to_--enable-optimizations=2E?=
Message-ID: <20161120211952.41510.14688.8EB65C5F@psf.io>

https://hg.python.org/cpython/rev/d5ff5a2f33fd
changeset:   105238:d5ff5a2f33fd
branch:      2.7
parent:      105228:01f867e9cd34
user:        Gregory P. Smith <greg at krypto.org> [Google Inc.]
date:        Sun Nov 20 21:19:36 2016 +0000
summary:
  Rename the new --with-optimiations flag to --enable-optimizations.

files:
  Misc/NEWS    |   2 ++
  configure    |  23 +++++++++++------------
  configure.ac |  10 +++++-----
  3 files changed, 18 insertions(+), 17 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -273,6 +273,8 @@
 Build
 -----
 
+- Issue #26359: Rename --with-optimiations to --enable-optimizations.
+
 - Issue #28248: Update Windows build and OS X installers to use OpenSSL 1.0.2j.
 
 - Issue #28258: Fixed build with Estonian locale (distclean target in
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -804,7 +804,7 @@
 enable_shared
 enable_profiling
 with_pydebug
-with_optimizations
+enable_optimizations
 with_lto
 enable_toolbox_glue
 with_libs
@@ -1475,6 +1475,8 @@
                           Build (MacOSX|Darwin) framework
   --enable-shared         disable/enable building shared python library
   --enable-profiling      enable C-level code profiling
+  --enable-optimizations  Enable expensive optimizations (PGO, maybe LTO,
+                          etc). Disabled by default.
   --enable-toolbox-glue   disable/enable MacOSX glue code for extensions
   --enable-ipv6           Enable ipv6 (with ipv4) support
   --disable-ipv6          Disable ipv6 support
@@ -1499,8 +1501,6 @@
                           compiler
   --with-suffix=.exe      set executable suffix
   --with-pydebug          build with Py_DEBUG defined
-  --with-optimizations    Enable expensive optimizations (PGO, maybe LTO,
-                          etc). Disabled by default.
   --with-lto              Enable Link Time Optimization in PGO builds.
                           Disabled by default.
   --with-libs='lib1 ...'  link against additional libs
@@ -6386,12 +6386,11 @@
 
 
 Py_OPT='false'
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-optimizations" >&5
-$as_echo_n "checking for --with-optimizations... " >&6; }
-
-# Check whether --with-optimizations was given.
-if test "${with_optimizations+set}" = set; then :
-  withval=$with_optimizations;
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --enable-optimizations" >&5
+$as_echo_n "checking for --enable-optimizations... " >&6; }
+# Check whether --enable-optimizations was given.
+if test "${enable_optimizations+set}" = set; then :
+  enableval=$enable_optimizations;
 if test "$withval" != no
 then
   Py_OPT='true'
@@ -6654,7 +6653,7 @@
       LLVM_PROF_ERR=yes
       if test "${REQUIRE_PGO}" = "yes"
       then
-        as_fn_error $? "llvm-profdata is required for a --with-optimizations build but could not be found." "$LINENO" 5
+        as_fn_error $? "llvm-profdata is required for a --enable-optimizations build but could not be found." "$LINENO" 5
       fi
     fi
     ;;
@@ -6670,7 +6669,7 @@
           LLVM_PROF_ERR=yes
           if test "${REQUIRE_PGO}" = "yes"
 	  then
-	    as_fn_error $? "llvm-profdata is required for a --with-optimizations build but could not be found." "$LINENO" 5
+	    as_fn_error $? "llvm-profdata is required for a --enable-optimizations build but could not be found." "$LINENO" 5
 	  fi
         fi
         ;;
@@ -16500,7 +16499,7 @@
     echo "" >&6
     echo "" >&6
     echo "If you want a release build with all optimizations active (LTO, PGO, etc),"
-    echo "please run ./configure --with-optimizations" >&6
+    echo "please run ./configure --enable-optimizations" >&6
     echo "" >&6
     echo "" >&6
 fi
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -1370,8 +1370,8 @@
 AC_SUBST(DEF_MAKE_ALL_RULE)
 AC_SUBST(DEF_MAKE_RULE)
 Py_OPT='false'
-AC_MSG_CHECKING(for --with-optimizations)
-AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable expensive optimizations (PGO, maybe LTO, etc).  Disabled by default.]),
+AC_MSG_CHECKING(for --enable-optimizations)
+AC_ARG_ENABLE(optimizations, AS_HELP_STRING([--enable-optimizations], [Enable expensive optimizations (PGO, maybe LTO, etc).  Disabled by default.]),
 [
 if test "$withval" != no
 then
@@ -1490,7 +1490,7 @@
       LLVM_PROF_ERR=yes
       if test "${REQUIRE_PGO}" = "yes"
       then
-        AC_MSG_ERROR([llvm-profdata is required for a --with-optimizations build but could not be found.])
+        AC_MSG_ERROR([llvm-profdata is required for a --enable-optimizations build but could not be found.])
       fi
     fi
     ;;
@@ -1506,7 +1506,7 @@
           LLVM_PROF_ERR=yes
           if test "${REQUIRE_PGO}" = "yes"
 	  then
-	    AC_MSG_ERROR([llvm-profdata is required for a --with-optimizations build but could not be found.])
+	    AC_MSG_ERROR([llvm-profdata is required for a --enable-optimizations build but could not be found.])
 	  fi
         fi
         ;;
@@ -4843,7 +4843,7 @@
     echo "" >&AS_MESSAGE_FD
     echo "" >&AS_MESSAGE_FD
     echo "If you want a release build with all optimizations active (LTO, PGO, etc),"
-    echo "please run ./configure --with-optimizations" >&AS_MESSAGE_FD
+    echo "please run ./configure --enable-optimizations" >&AS_MESSAGE_FD
     echo "" >&AS_MESSAGE_FD
     echo "" >&AS_MESSAGE_FD
 fi

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 18:24:10 2016
From: python-checkins at python.org (martin.panter)
Date: Sun, 20 Nov 2016 23:24:10 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2325659=3A_Merge_ctypes_fix_from_3=2E5?=
Message-ID: <20161120232409.12102.6328.749734F8@psf.io>

https://hg.python.org/cpython/rev/1253ef20c947
changeset:   105242:1253ef20c947
branch:      3.6
parent:      105241:76d1f8001e27
parent:      105240:5f061870d49c
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sun Nov 20 22:07:29 2016 +0000
summary:
  Issue #25659: Merge ctypes fix from 3.5

files:
  Lib/ctypes/test/test_frombuffer.py |   8 ++++++++
  Misc/NEWS                          |   3 +++
  Modules/_ctypes/_ctypes.c          |  14 ++++++++++----
  3 files changed, 21 insertions(+), 4 deletions(-)


diff --git a/Lib/ctypes/test/test_frombuffer.py b/Lib/ctypes/test/test_frombuffer.py
--- a/Lib/ctypes/test/test_frombuffer.py
+++ b/Lib/ctypes/test/test_frombuffer.py
@@ -120,5 +120,13 @@
         with self.assertRaises(ValueError):
             (c_int * 1).from_buffer_copy(a, 16 * sizeof(c_int))
 
+    def test_abstract(self):
+        self.assertRaises(TypeError, Array.from_buffer, bytearray(10))
+        self.assertRaises(TypeError, Structure.from_buffer, bytearray(10))
+        self.assertRaises(TypeError, Union.from_buffer, bytearray(10))
+        self.assertRaises(TypeError, Array.from_buffer_copy, b"123")
+        self.assertRaises(TypeError, Structure.from_buffer_copy, b"123")
+        self.assertRaises(TypeError, Union.from_buffer_copy, b"123")
+
 if __name__ == '__main__':
     unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -40,6 +40,9 @@
 Library
 -------
 
+- Issue #25659: In ctypes, prevent a crash calling the from_buffer() and
+  from_buffer_copy() methods on abstract classes like Array.
+
 - Issue #19717: Makes Path.resolve() succeed on paths that do not exist.
   Patch by Vajrasky Kok
 
diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c
--- a/Modules/_ctypes/_ctypes.c
+++ b/Modules/_ctypes/_ctypes.c
@@ -463,7 +463,10 @@
     Py_ssize_t offset = 0;
 
     StgDictObject *dict = PyType_stgdict(type);
-    assert (dict);
+    if (!dict) {
+        PyErr_SetString(PyExc_TypeError, "abstract class");
+        return NULL;
+    }
 
     if (!PyArg_ParseTuple(args, "O|n:from_buffer", &obj, &offset))
         return NULL;
@@ -531,9 +534,12 @@
     Py_ssize_t offset = 0;
     PyObject *result;
     StgDictObject *dict = PyType_stgdict(type);
-    assert (dict);
-
-    if (!PyArg_ParseTuple(args, "y*|n:from_buffer", &buffer, &offset))
+    if (!dict) {
+        PyErr_SetString(PyExc_TypeError, "abstract class");
+        return NULL;
+    }
+
+    if (!PyArg_ParseTuple(args, "y*|n:from_buffer_copy", &buffer, &offset))
         return NULL;
 
     if (offset < 0) {

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 18:24:09 2016
From: python-checkins at python.org (martin.panter)
Date: Sun, 20 Nov 2016 23:24:09 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2310656=3A_Merge_AIX_build_fix_from_3=2E5?=
Message-ID: <20161120232409.10449.99458.111D1504@psf.io>

https://hg.python.org/cpython/rev/76d1f8001e27
changeset:   105241:76d1f8001e27
branch:      3.6
parent:      105236:0d2b42344ae5
parent:      105239:48526666321a
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sun Nov 20 22:06:44 2016 +0000
summary:
  Issue #10656: Merge AIX build fix from 3.5

files:
  Makefile.pre.in      |  2 +-
  Misc/ACKS            |  2 ++
  Misc/NEWS            |  3 +++
  Modules/ld_so_aix.in |  1 +
  configure            |  2 +-
  configure.ac         |  2 +-
  6 files changed, 9 insertions(+), 3 deletions(-)


diff --git a/Makefile.pre.in b/Makefile.pre.in
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -1421,7 +1421,7 @@
 		$(INSTALL_SCRIPT) $(srcdir)/Modules/makexp_aix	\
 				$(DESTDIR)$(LIBPL)/makexp_aix;		\
 		echo "$(LIBPL)/makexp_aix";			\
-		$(INSTALL_SCRIPT) $(srcdir)/Modules/ld_so_aix	\
+		$(INSTALL_SCRIPT) Modules/ld_so_aix	\
 				$(DESTDIR)$(LIBPL)/ld_so_aix;		\
 		echo "$(LIBPL)/ld_so_aix";			\
 		echo; echo "See Misc/AIX-NOTES for details.";	\
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -227,6 +227,7 @@
 Daniel Calvelo
 Tony Campbell
 Brett Cannon
+Tristan Carel
 Mike Carlton
 Pierre Carrier
 Terry Carroll
@@ -581,6 +582,7 @@
 Larry Hastings
 Tim Hatch
 Shane Hathaway
+Michael Haubenwallner
 Janko Hauser
 Rycharde Hawkes
 Ben Hayden
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -90,6 +90,9 @@
 Build
 -----
 
+- Issue #10656: Fix out-of-tree building on AIX.  Patch by Tristan Carel and
+  Michael Haubenwallner.
+
 - Issue #26359: Rename --with-optimiations to --enable-optimizations.
 
 - Issue #28676: Prevent missing 'getentropy' declaration warning on macOS.
diff --git a/Modules/ld_so_aix.in b/Modules/ld_so_aix.in
--- a/Modules/ld_so_aix.in
+++ b/Modules/ld_so_aix.in
@@ -70,6 +70,7 @@
 fi
 
 makexp=`dirname $0`/makexp_aix
+test -x "${makexp}" || makexp="@abs_srcdir@/makexp_aix"
 
 # Check for existence of compiler.
 CC=$1; shift
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -9165,7 +9165,7 @@
 then
 	case $ac_sys_system/$ac_sys_release in
 	AIX*)
-		BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:\$(srcdir)/Modules/python.exp"
+		BLDSHARED="Modules/ld_so_aix \$(CC) -bI:Modules/python.exp"
 		LDSHARED="\$(LIBPL)/ld_so_aix \$(CC) -bI:\$(LIBPL)/python.exp"
 		;;
 	IRIX/5*) LDSHARED="ld -shared";;
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -2409,7 +2409,7 @@
 then
 	case $ac_sys_system/$ac_sys_release in
 	AIX*)
-		BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:\$(srcdir)/Modules/python.exp"
+		BLDSHARED="Modules/ld_so_aix \$(CC) -bI:Modules/python.exp"
 		LDSHARED="\$(LIBPL)/ld_so_aix \$(CC) -bI:\$(LIBPL)/python.exp"
 		;;
 	IRIX/5*) LDSHARED="ld -shared";;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 18:24:09 2016
From: python-checkins at python.org (martin.panter)
Date: Sun, 20 Nov 2016 23:24:09 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1NjU5?=
 =?utf-8?q?=3A_Change_assert_to_TypeError_in_from=5Fbuffer/=5Fcopy=28=29?=
Message-ID: <20161120232409.5263.39365.83B35630@psf.io>

https://hg.python.org/cpython/rev/5f061870d49c
changeset:   105240:5f061870d49c
branch:      3.5
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sun Nov 20 07:58:35 2016 +0000
summary:
  Issue #25659: Change assert to TypeError in from_buffer/_copy()

Based on suggestion by Eryk Sun.

files:
  Lib/ctypes/test/test_frombuffer.py |   8 ++++++++
  Misc/NEWS                          |   3 +++
  Modules/_ctypes/_ctypes.c          |  14 ++++++++++----
  3 files changed, 21 insertions(+), 4 deletions(-)


diff --git a/Lib/ctypes/test/test_frombuffer.py b/Lib/ctypes/test/test_frombuffer.py
--- a/Lib/ctypes/test/test_frombuffer.py
+++ b/Lib/ctypes/test/test_frombuffer.py
@@ -120,5 +120,13 @@
         with self.assertRaises(ValueError):
             (c_int * 1).from_buffer_copy(a, 16 * sizeof(c_int))
 
+    def test_abstract(self):
+        self.assertRaises(TypeError, Array.from_buffer, bytearray(10))
+        self.assertRaises(TypeError, Structure.from_buffer, bytearray(10))
+        self.assertRaises(TypeError, Union.from_buffer, bytearray(10))
+        self.assertRaises(TypeError, Array.from_buffer_copy, b"123")
+        self.assertRaises(TypeError, Structure.from_buffer_copy, b"123")
+        self.assertRaises(TypeError, Union.from_buffer_copy, b"123")
+
 if __name__ == '__main__':
     unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -121,6 +121,9 @@
 Library
 -------
 
+- Issue #25659: In ctypes, prevent a crash calling the from_buffer() and
+  from_buffer_copy() methods on abstract classes like Array.
+
 - Issue #28732: Fix crash in os.spawnv() with no elements in args
 
 - Issue #28485: Always raise ValueError for negative
diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c
--- a/Modules/_ctypes/_ctypes.c
+++ b/Modules/_ctypes/_ctypes.c
@@ -469,7 +469,10 @@
     Py_ssize_t offset = 0;
 
     StgDictObject *dict = PyType_stgdict(type);
-    assert (dict);
+    if (!dict) {
+        PyErr_SetString(PyExc_TypeError, "abstract class");
+        return NULL;
+    }
 
     if (!PyArg_ParseTuple(args, "O|n:from_buffer", &obj, &offset))
         return NULL;
@@ -537,9 +540,12 @@
     Py_ssize_t offset = 0;
     PyObject *result;
     StgDictObject *dict = PyType_stgdict(type);
-    assert (dict);
-
-    if (!PyArg_ParseTuple(args, "y*|n:from_buffer", &buffer, &offset))
+    if (!dict) {
+        PyErr_SetString(PyExc_TypeError, "abstract class");
+        return NULL;
+    }
+
+    if (!PyArg_ParseTuple(args, "y*|n:from_buffer_copy", &buffer, &offset))
         return NULL;
 
     if (offset < 0) {

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 18:24:10 2016
From: python-checkins at python.org (martin.panter)
Date: Sun, 20 Nov 2016 23:24:10 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Fix_typos_in_d?=
 =?utf-8?q?ocumentation?=
Message-ID: <20161120232410.12689.72173.9A5B2496@psf.io>

https://hg.python.org/cpython/rev/8372b9a325fb
changeset:   105243:8372b9a325fb
branch:      3.6
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sun Nov 20 08:37:21 2016 +0000
summary:
  Fix typos in documentation

files:
  Doc/library/email.message.rst |  2 +-
  Misc/NEWS                     |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/email.message.rst b/Doc/library/email.message.rst
--- a/Doc/library/email.message.rst
+++ b/Doc/library/email.message.rst
@@ -275,7 +275,7 @@
       only the key will be added.
 
       If the value contains non-ASCII characters, the charset and language may
-      be explicitly controlled by specifing the value as a three tuple in the
+      be explicitly controlled by specifying the value as a three tuple in the
       format ``(CHARSET, LANGUAGE, VALUE)``, where ``CHARSET`` is a string
       naming the charset to be used to encode the value, ``LANGUAGE`` can
       usually be set to ``None`` or the empty string (see :rfc:`2231` for other
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -366,7 +366,7 @@
   if pass invalid string-like object as a name.  Patch by Xiang Zhang.
 
 - Issue #18844: random.choices() now has k as a keyword-only argument
-  to improve the readability of common cases and the come into line
+  to improve the readability of common cases and come into line
   with the signature used in other languages.
 
 - Issue #18893: Fix invalid exception handling in Lib/ctypes/macholib/dyld.py.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 18:24:09 2016
From: python-checkins at python.org (martin.panter)
Date: Sun, 20 Nov 2016 23:24:09 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzEwNjU2?=
 =?utf-8?q?=3A_Fix_out-of-tree_building_on_AIX?=
Message-ID: <20161120232409.12421.26731.7E147911@psf.io>

https://hg.python.org/cpython/rev/48526666321a
changeset:   105239:48526666321a
branch:      3.5
parent:      105233:c0ea81315fb6
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sun Nov 20 07:56:37 2016 +0000
summary:
  Issue #10656: Fix out-of-tree building on AIX

The ld_so_aix script and python.exp file are created in the build directory.
Patch by Tristan Carel and Michael Haubenwallner.

files:
  Makefile.pre.in      |  2 +-
  Misc/ACKS            |  2 ++
  Misc/NEWS            |  3 +++
  Modules/ld_so_aix.in |  1 +
  configure            |  2 +-
  configure.ac         |  2 +-
  6 files changed, 9 insertions(+), 3 deletions(-)


diff --git a/Makefile.pre.in b/Makefile.pre.in
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -1418,7 +1418,7 @@
 		$(INSTALL_SCRIPT) $(srcdir)/Modules/makexp_aix	\
 				$(DESTDIR)$(LIBPL)/makexp_aix;		\
 		echo "$(LIBPL)/makexp_aix";			\
-		$(INSTALL_SCRIPT) $(srcdir)/Modules/ld_so_aix	\
+		$(INSTALL_SCRIPT) Modules/ld_so_aix	\
 				$(DESTDIR)$(LIBPL)/ld_so_aix;		\
 		echo "$(LIBPL)/ld_so_aix";			\
 		echo; echo "See Misc/AIX-NOTES for details.";	\
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -225,6 +225,7 @@
 Daniel Calvelo
 Tony Campbell
 Brett Cannon
+Tristan Carel
 Mike Carlton
 Pierre Carrier
 Terry Carroll
@@ -573,6 +574,7 @@
 Larry Hastings
 Tim Hatch
 Shane Hathaway
+Michael Haubenwallner
 Janko Hauser
 Rycharde Hawkes
 Ben Hayden
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -555,6 +555,9 @@
 Build
 -----
 
+- Issue #10656: Fix out-of-tree building on AIX.  Patch by Tristan Carel and
+  Michael Haubenwallner.
+
 - Issue #26359: Rename --with-optimiations to --enable-optimizations.
 
 - Issue #28444: Fix missing extensions modules when cross compiling.
diff --git a/Modules/ld_so_aix.in b/Modules/ld_so_aix.in
--- a/Modules/ld_so_aix.in
+++ b/Modules/ld_so_aix.in
@@ -70,6 +70,7 @@
 fi
 
 makexp=`dirname $0`/makexp_aix
+test -x "${makexp}" || makexp="@abs_srcdir@/makexp_aix"
 
 # Check for existence of compiler.
 CC=$1; shift
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -9226,7 +9226,7 @@
 then
 	case $ac_sys_system/$ac_sys_release in
 	AIX*)
-		BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:\$(srcdir)/Modules/python.exp"
+		BLDSHARED="Modules/ld_so_aix \$(CC) -bI:Modules/python.exp"
 		LDSHARED="\$(LIBPL)/ld_so_aix \$(CC) -bI:\$(LIBPL)/python.exp"
 		;;
 	IRIX/5*) LDSHARED="ld -shared";;
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -2358,7 +2358,7 @@
 then
 	case $ac_sys_system/$ac_sys_release in
 	AIX*)
-		BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:\$(srcdir)/Modules/python.exp"
+		BLDSHARED="Modules/ld_so_aix \$(CC) -bI:Modules/python.exp"
 		LDSHARED="\$(LIBPL)/ld_so_aix \$(CC) -bI:\$(LIBPL)/python.exp"
 		;;
 	IRIX/5*) LDSHARED="ld -shared";;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 18:24:10 2016
From: python-checkins at python.org (martin.panter)
Date: Sun, 20 Nov 2016 23:24:10 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2310656=3A_Merge_AIX_build_fix_from_3=2E6?=
Message-ID: <20161120232410.7167.14564.11DFC020@psf.io>

https://hg.python.org/cpython/rev/180f046b597e
changeset:   105244:180f046b597e
parent:      105237:2d1d70b53376
parent:      105241:76d1f8001e27
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sun Nov 20 22:16:46 2016 +0000
summary:
  Issue #10656: Merge AIX build fix from 3.6

files:
  Makefile.pre.in      |  2 +-
  Misc/ACKS            |  2 ++
  Misc/NEWS            |  3 +++
  Modules/ld_so_aix.in |  1 +
  configure            |  2 +-
  configure.ac         |  2 +-
  6 files changed, 9 insertions(+), 3 deletions(-)


diff --git a/Makefile.pre.in b/Makefile.pre.in
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -1421,7 +1421,7 @@
 		$(INSTALL_SCRIPT) $(srcdir)/Modules/makexp_aix	\
 				$(DESTDIR)$(LIBPL)/makexp_aix;		\
 		echo "$(LIBPL)/makexp_aix";			\
-		$(INSTALL_SCRIPT) $(srcdir)/Modules/ld_so_aix	\
+		$(INSTALL_SCRIPT) Modules/ld_so_aix	\
 				$(DESTDIR)$(LIBPL)/ld_so_aix;		\
 		echo "$(LIBPL)/ld_so_aix";			\
 		echo; echo "See Misc/AIX-NOTES for details.";	\
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -227,6 +227,7 @@
 Daniel Calvelo
 Tony Campbell
 Brett Cannon
+Tristan Carel
 Mike Carlton
 Pierre Carrier
 Terry Carroll
@@ -582,6 +583,7 @@
 Larry Hastings
 Tim Hatch
 Shane Hathaway
+Michael Haubenwallner
 Janko Hauser
 Rycharde Hawkes
 Ben Hayden
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -425,6 +425,9 @@
 Build
 -----
 
+- Issue #10656: Fix out-of-tree building on AIX.  Patch by Tristan Carel and
+  Michael Haubenwallner.
+
 - Issue #26359: Rename --with-optimiations to --enable-optimizations.
 
 - Issue #28444: Fix missing extensions modules when cross compiling.
diff --git a/Modules/ld_so_aix.in b/Modules/ld_so_aix.in
--- a/Modules/ld_so_aix.in
+++ b/Modules/ld_so_aix.in
@@ -70,6 +70,7 @@
 fi
 
 makexp=`dirname $0`/makexp_aix
+test -x "${makexp}" || makexp="@abs_srcdir@/makexp_aix"
 
 # Check for existence of compiler.
 CC=$1; shift
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -9165,7 +9165,7 @@
 then
 	case $ac_sys_system/$ac_sys_release in
 	AIX*)
-		BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:\$(srcdir)/Modules/python.exp"
+		BLDSHARED="Modules/ld_so_aix \$(CC) -bI:Modules/python.exp"
 		LDSHARED="\$(LIBPL)/ld_so_aix \$(CC) -bI:\$(LIBPL)/python.exp"
 		;;
 	IRIX/5*) LDSHARED="ld -shared";;
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -2409,7 +2409,7 @@
 then
 	case $ac_sys_system/$ac_sys_release in
 	AIX*)
-		BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:\$(srcdir)/Modules/python.exp"
+		BLDSHARED="Modules/ld_so_aix \$(CC) -bI:Modules/python.exp"
 		LDSHARED="\$(LIBPL)/ld_so_aix \$(CC) -bI:\$(LIBPL)/python.exp"
 		;;
 	IRIX/5*) LDSHARED="ld -shared";;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 18:24:10 2016
From: python-checkins at python.org (martin.panter)
Date: Sun, 20 Nov 2016 23:24:10 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2325659=3A_Merge_ctypes_fix_from_3=2E6?=
Message-ID: <20161120232410.45713.4240.47A95AD6@psf.io>

https://hg.python.org/cpython/rev/821da4891051
changeset:   105245:821da4891051
parent:      105244:180f046b597e
parent:      105242:1253ef20c947
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sun Nov 20 22:17:44 2016 +0000
summary:
  Issue #25659: Merge ctypes fix from 3.6

files:
  Lib/ctypes/test/test_frombuffer.py |   8 ++++++++
  Misc/NEWS                          |   3 +++
  Modules/_ctypes/_ctypes.c          |  14 ++++++++++----
  3 files changed, 21 insertions(+), 4 deletions(-)


diff --git a/Lib/ctypes/test/test_frombuffer.py b/Lib/ctypes/test/test_frombuffer.py
--- a/Lib/ctypes/test/test_frombuffer.py
+++ b/Lib/ctypes/test/test_frombuffer.py
@@ -120,5 +120,13 @@
         with self.assertRaises(ValueError):
             (c_int * 1).from_buffer_copy(a, 16 * sizeof(c_int))
 
+    def test_abstract(self):
+        self.assertRaises(TypeError, Array.from_buffer, bytearray(10))
+        self.assertRaises(TypeError, Structure.from_buffer, bytearray(10))
+        self.assertRaises(TypeError, Union.from_buffer, bytearray(10))
+        self.assertRaises(TypeError, Array.from_buffer_copy, b"123")
+        self.assertRaises(TypeError, Structure.from_buffer_copy, b"123")
+        self.assertRaises(TypeError, Union.from_buffer_copy, b"123")
+
 if __name__ == '__main__':
     unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -131,6 +131,9 @@
 Library
 -------
 
+- Issue #25659: In ctypes, prevent a crash calling the from_buffer() and
+  from_buffer_copy() methods on abstract classes like Array.
+
 - Issue #28548: In the "http.server" module, parse the protocol version if
   possible, to avoid using HTTP 0.9 in some error responses.
 
diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c
--- a/Modules/_ctypes/_ctypes.c
+++ b/Modules/_ctypes/_ctypes.c
@@ -463,7 +463,10 @@
     Py_ssize_t offset = 0;
 
     StgDictObject *dict = PyType_stgdict(type);
-    assert (dict);
+    if (!dict) {
+        PyErr_SetString(PyExc_TypeError, "abstract class");
+        return NULL;
+    }
 
     if (!PyArg_ParseTuple(args, "O|n:from_buffer", &obj, &offset))
         return NULL;
@@ -531,9 +534,12 @@
     Py_ssize_t offset = 0;
     PyObject *result;
     StgDictObject *dict = PyType_stgdict(type);
-    assert (dict);
-
-    if (!PyArg_ParseTuple(args, "y*|n:from_buffer", &buffer, &offset))
+    if (!dict) {
+        PyErr_SetString(PyExc_TypeError, "abstract class");
+        return NULL;
+    }
+
+    if (!PyArg_ParseTuple(args, "y*|n:from_buffer_copy", &buffer, &offset))
         return NULL;
 
     if (offset < 0) {

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 18:24:10 2016
From: python-checkins at python.org (martin.panter)
Date: Sun, 20 Nov 2016 23:24:10 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4NjY2?=
 =?utf-8?q?=3A_Fix_stat_import?=
Message-ID: <20161120232410.28394.91802.3AEA5FE9@psf.io>

https://hg.python.org/cpython/rev/dd378356c77c
changeset:   105247:dd378356c77c
branch:      2.7
parent:      105238:d5ff5a2f33fd
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sun Nov 20 23:06:58 2016 +0000
summary:
  Issue #28666: Fix stat import

files:
  Lib/test/test_support.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py
--- a/Lib/test/test_support.py
+++ b/Lib/test/test_support.py
@@ -8,6 +8,7 @@
 import functools
 import gc
 import socket
+import stat
 import sys
 import os
 import platform
@@ -248,7 +249,6 @@
     _rmdir = os.rmdir
 
     def _rmtree(path):
-        import stat
         try:
             shutil.rmtree(path)
             return

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 18:24:10 2016
From: python-checkins at python.org (martin.panter)
Date: Sun, 20 Nov 2016 23:24:10 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_typo_fixes_from_3=2E6?=
Message-ID: <20161120232410.10531.87336.9F99CA63@psf.io>

https://hg.python.org/cpython/rev/044e48c396b5
changeset:   105246:044e48c396b5
parent:      105245:821da4891051
parent:      105243:8372b9a325fb
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sun Nov 20 22:18:08 2016 +0000
summary:
  Merge typo fixes from 3.6

files:
  Doc/library/email.message.rst |  2 +-
  Misc/NEWS                     |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/email.message.rst b/Doc/library/email.message.rst
--- a/Doc/library/email.message.rst
+++ b/Doc/library/email.message.rst
@@ -275,7 +275,7 @@
       only the key will be added.
 
       If the value contains non-ASCII characters, the charset and language may
-      be explicitly controlled by specifing the value as a three tuple in the
+      be explicitly controlled by specifying the value as a three tuple in the
       format ``(CHARSET, LANGUAGE, VALUE)``, where ``CHARSET`` is a string
       naming the charset to be used to encode the value, ``LANGUAGE`` can
       usually be set to ``None`` or the empty string (see :rfc:`2231` for other
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -295,7 +295,7 @@
   if pass invalid string-like object as a name.  Patch by Xiang Zhang.
 
 - Issue #18844: random.choices() now has k as a keyword-only argument
-  to improve the readability of common cases and the come into line
+  to improve the readability of common cases and come into line
   with the signature used in other languages.
 
 - Issue #18893: Fix invalid exception handling in Lib/ctypes/macholib/dyld.py.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 18:24:10 2016
From: python-checkins at python.org (martin.panter)
Date: Sun, 20 Nov 2016 23:24:10 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEwNjU2?=
 =?utf-8?q?=3A_Fix_out-of-tree_building_on_AIX?=
Message-ID: <20161120232410.14486.21857.3238CCE6@psf.io>

https://hg.python.org/cpython/rev/ca46883fc5cf
changeset:   105248:ca46883fc5cf
branch:      2.7
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sun Nov 20 09:31:41 2016 +0000
summary:
  Issue #10656: Fix out-of-tree building on AIX

The ld_so_aix script and python.exp file are created in the build directory.
Patch by Tristan Carel and Michael Haubenwallner.

files:
  Makefile.pre.in      |  2 +-
  Misc/ACKS            |  2 ++
  Misc/NEWS            |  3 +++
  Modules/ld_so_aix.in |  1 +
  configure            |  2 +-
  configure.ac         |  2 +-
  6 files changed, 9 insertions(+), 3 deletions(-)


diff --git a/Makefile.pre.in b/Makefile.pre.in
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -1232,7 +1232,7 @@
 		$(INSTALL_SCRIPT) $(srcdir)/Modules/makexp_aix	\
 				$(DESTDIR)$(LIBPL)/makexp_aix;		\
 		echo "$(LIBPL)/makexp_aix";			\
-		$(INSTALL_SCRIPT) $(srcdir)/Modules/ld_so_aix	\
+		$(INSTALL_SCRIPT) Modules/ld_so_aix	\
 				$(DESTDIR)$(LIBPL)/ld_so_aix;		\
 		echo "$(LIBPL)/ld_so_aix";			\
 		echo; echo "See Misc/AIX-NOTES for details.";	\
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -209,6 +209,7 @@
 Daniel Calvelo
 Tony Campbell
 Brett Cannon
+Tristan Carel
 Mike Carlton
 Pierre Carrier
 Terry Carroll
@@ -541,6 +542,7 @@
 Larry Hastings
 Tim Hatch
 Shane Hathaway
+Michael Haubenwallner
 Janko Hauser
 Rycharde Hawkes
 Ben Hayden
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -273,6 +273,9 @@
 Build
 -----
 
+- Issue #10656: Fix out-of-tree building on AIX.  Patch by Tristan Carel and
+  Michael Haubenwallner.
+
 - Issue #26359: Rename --with-optimiations to --enable-optimizations.
 
 - Issue #28248: Update Windows build and OS X installers to use OpenSSL 1.0.2j.
diff --git a/Modules/ld_so_aix.in b/Modules/ld_so_aix.in
--- a/Modules/ld_so_aix.in
+++ b/Modules/ld_so_aix.in
@@ -70,6 +70,7 @@
 fi
 
 makexp=`dirname $0`/makexp_aix
+test -x "${makexp}" || makexp="@abs_srcdir@/makexp_aix"
 
 # Check for existence of compiler.
 CC=$1; shift
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -8551,7 +8551,7 @@
 then
 	case $ac_sys_system/$ac_sys_release in
 	AIX*)
-		BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:\$(srcdir)/Modules/python.exp"
+		BLDSHARED="Modules/ld_so_aix \$(CC) -bI:Modules/python.exp"
 		LDSHARED="\$(BINLIBDEST)/config/ld_so_aix \$(CC) -bI:\$(BINLIBDEST)/config/python.exp"
 		;;
 	BeOS*)
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -2124,7 +2124,7 @@
 then
 	case $ac_sys_system/$ac_sys_release in
 	AIX*)
-		BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:\$(srcdir)/Modules/python.exp"
+		BLDSHARED="Modules/ld_so_aix \$(CC) -bI:Modules/python.exp"
 		LDSHARED="\$(BINLIBDEST)/config/ld_so_aix \$(CC) -bI:\$(BINLIBDEST)/config/python.exp"
 		;;
 	BeOS*)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 18:24:10 2016
From: python-checkins at python.org (martin.panter)
Date: Sun, 20 Nov 2016 23:24:10 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1NjU5?=
 =?utf-8?q?=3A_Change_assert_to_TypeError_in_from=5Fbuffer/=5Fcopy=28=29?=
Message-ID: <20161120232410.12232.55914.B2F7D5AB@psf.io>

https://hg.python.org/cpython/rev/4de956751cf1
changeset:   105249:4de956751cf1
branch:      2.7
user:        Martin Panter <vadmium+py at gmail.com>
date:        Sun Nov 20 09:35:06 2016 +0000
summary:
  Issue #25659: Change assert to TypeError in from_buffer/_copy()

Based on suggestion by Eryk Sun.

files:
  Lib/ctypes/test/test_frombuffer.py |   8 ++++++++
  Misc/NEWS                          |   3 +++
  Modules/_ctypes/_ctypes.c          |  14 ++++++++++----
  3 files changed, 21 insertions(+), 4 deletions(-)


diff --git a/Lib/ctypes/test/test_frombuffer.py b/Lib/ctypes/test/test_frombuffer.py
--- a/Lib/ctypes/test/test_frombuffer.py
+++ b/Lib/ctypes/test/test_frombuffer.py
@@ -77,5 +77,13 @@
         self.assertRaises(ValueError,
                           (c_int * 1).from_buffer_copy, a, 16 * sizeof(c_int))
 
+    def test_abstract(self):
+        self.assertRaises(TypeError, Array.from_buffer, bytearray(10))
+        self.assertRaises(TypeError, Structure.from_buffer, bytearray(10))
+        self.assertRaises(TypeError, Union.from_buffer, bytearray(10))
+        self.assertRaises(TypeError, Array.from_buffer_copy, b"123")
+        self.assertRaises(TypeError, Structure.from_buffer_copy, b"123")
+        self.assertRaises(TypeError, Union.from_buffer_copy, b"123")
+
 if __name__ == '__main__':
     unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -63,6 +63,9 @@
 Library
 -------
 
+- Issue #25659: In ctypes, prevent a crash calling the from_buffer() and
+  from_buffer_copy() methods on abstract classes like Array.
+
 - Issue #28563: Fixed possible DoS and arbitrary code execution when handle
   plural form selections in the gettext module.  The expression parser now
   supports exact syntax supported by GNU gettext.
diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c
--- a/Modules/_ctypes/_ctypes.c
+++ b/Modules/_ctypes/_ctypes.c
@@ -501,7 +501,10 @@
     Py_ssize_t offset = 0;
     PyObject *obj, *result;
     StgDictObject *dict = PyType_stgdict(type);
-    assert (dict);
+    if (!dict) {
+        PyErr_SetString(PyExc_TypeError, "abstract class");
+        return NULL;
+    }
 
     if (!PyArg_ParseTuple(args,
 #if (PY_VERSION_HEX < 0x02050000)
@@ -557,13 +560,16 @@
     Py_ssize_t offset = 0;
     PyObject *obj, *result;
     StgDictObject *dict = PyType_stgdict(type);
-    assert (dict);
+    if (!dict) {
+        PyErr_SetString(PyExc_TypeError, "abstract class");
+        return NULL;
+    }
 
     if (!PyArg_ParseTuple(args,
 #if (PY_VERSION_HEX < 0x02050000)
-                          "O|i:from_buffer",
+                          "O|i:from_buffer_copy",
 #else
-                          "O|n:from_buffer",
+                          "O|n:from_buffer_copy",
 #endif
                           &obj, &offset))
         return NULL;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 19:31:16 2016
From: python-checkins at python.org (gregory.p.smith)
Date: Mon, 21 Nov 2016 00:31:16 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzIwNTcy?=
 =?utf-8?q?=3A_The_subprocess=2EPopen=2Ewait_method=27s_undocumented_endti?=
 =?utf-8?q?me?=
Message-ID: <20161121003116.7475.36589.804F950D@psf.io>

https://hg.python.org/cpython/rev/0e8aa537c565
changeset:   105250:0e8aa537c565
branch:      3.6
parent:      105243:8372b9a325fb
user:        Gregory P. Smith <greg at krypto.org>
date:        Sun Nov 20 16:25:14 2016 -0800
summary:
  Issue #20572: The subprocess.Popen.wait method's undocumented endtime
parameter now raises a DeprecationWarning.  It was deprecated in 3.4.
It was never documented prior to that.

files:
  Lib/subprocess.py           |  11 ++++++++-
  Lib/test/test_subprocess.py |  27 ++++++++++++------------
  Misc/NEWS                   |   3 ++
  3 files changed, 25 insertions(+), 16 deletions(-)


diff --git a/Lib/subprocess.py b/Lib/subprocess.py
--- a/Lib/subprocess.py
+++ b/Lib/subprocess.py
@@ -1031,6 +1031,10 @@
             """Wait for child process to terminate.  Returns returncode
             attribute."""
             if endtime is not None:
+                warnings.warn(
+                    "'endtime' argument is deprecated; use 'timeout'.",
+                    DeprecationWarning,
+                    stacklevel=2)
                 timeout = self._remaining_time(endtime)
             if timeout is None:
                 timeout_millis = _winapi.INFINITE
@@ -1392,8 +1396,11 @@
             if self.returncode is not None:
                 return self.returncode
 
-            # endtime is preferred to timeout.  timeout is only used for
-            # printing.
+            if endtime is not None:
+                warnings.warn(
+                    "'endtime' argument is deprecated; use 'timeout'.",
+                    DeprecationWarning,
+                    stacklevel=2)
             if endtime is not None or timeout is not None:
                 if endtime is None:
                     endtime = _time() + timeout
diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -1015,6 +1015,19 @@
         # time to start.
         self.assertEqual(p.wait(timeout=3), 0)
 
+    def test_wait_endtime(self):
+        """Confirm that the deprecated endtime parameter warns."""
+        p = subprocess.Popen([sys.executable, "-c", "pass"])
+        try:
+            with self.assertWarns(DeprecationWarning) as warn_cm:
+                p.wait(endtime=time.time()+0.01)
+        except subprocess.TimeoutExpired:
+            pass  # We're not testing endtime timeout behavior.
+        finally:
+            p.kill()
+        self.assertIn('test_subprocess.py', warn_cm.filename)
+        self.assertIn('endtime', str(warn_cm.warning))
+
     def test_invalid_bufsize(self):
         # an invalid type of the bufsize argument should raise
         # TypeError.
@@ -2777,19 +2790,5 @@
         self.assertTrue(proc.stdin.closed)
 
 
-def test_main():
-    unit_tests = (ProcessTestCase,
-                  POSIXProcessTestCase,
-                  Win32ProcessTestCase,
-                  MiscTests,
-                  ProcessTestCaseNoPoll,
-                  CommandsWithSpaces,
-                  ContextManagerTests,
-                  RunFuncTestCase,
-                  )
-
-    support.run_unittest(*unit_tests)
-    support.reap_children()
-
 if __name__ == "__main__":
     unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -40,6 +40,9 @@
 Library
 -------
 
+- Issue #20572: The subprocess.Popen.wait method's undocumented
+  endtime parameter now raises a DeprecationWarning.
+
 - Issue #25659: In ctypes, prevent a crash calling the from_buffer() and
   from_buffer_copy() methods on abstract classes like Array.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 19:31:18 2016
From: python-checkins at python.org (gregory.p.smith)
Date: Mon, 21 Nov 2016 00:31:18 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2320572=3A_Remove_the_subprocess=2EPopen=2Ewait_e?=
 =?utf-8?q?ndtime_parameter=2E?=
Message-ID: <20161121003116.45170.94379.F50C2391@psf.io>

https://hg.python.org/cpython/rev/f02422c6110a
changeset:   105251:f02422c6110a
parent:      105246:044e48c396b5
parent:      105250:0e8aa537c565
user:        Gregory P. Smith <greg at krypto.org>
date:        Sun Nov 20 16:31:07 2016 -0800
summary:
  Issue #20572: Remove the subprocess.Popen.wait endtime parameter.
It was deprecated in 3.4 and undocumented prior to that.

files:
  Lib/subprocess.py           |  17 ++++-------------
  Lib/test/test_subprocess.py |  14 --------------
  Misc/NEWS                   |   3 +++
  3 files changed, 7 insertions(+), 27 deletions(-)


diff --git a/Lib/subprocess.py b/Lib/subprocess.py
--- a/Lib/subprocess.py
+++ b/Lib/subprocess.py
@@ -1027,11 +1027,9 @@
             return self.returncode
 
 
-        def wait(self, timeout=None, endtime=None):
+        def wait(self, timeout=None):
             """Wait for child process to terminate.  Returns returncode
             attribute."""
-            if endtime is not None:
-                timeout = self._remaining_time(endtime)
             if timeout is None:
                 timeout_millis = _winapi.INFINITE
             else:
@@ -1386,21 +1384,14 @@
             return (pid, sts)
 
 
-        def wait(self, timeout=None, endtime=None):
+        def wait(self, timeout=None):
             """Wait for child process to terminate.  Returns returncode
             attribute."""
             if self.returncode is not None:
                 return self.returncode
 
-            # endtime is preferred to timeout.  timeout is only used for
-            # printing.
-            if endtime is not None or timeout is not None:
-                if endtime is None:
-                    endtime = _time() + timeout
-                elif timeout is None:
-                    timeout = self._remaining_time(endtime)
-
-            if endtime is not None:
+            if timeout is not None:
+                endtime = _time() + timeout
                 # Enter a busy loop if we have a timeout.  This busy loop was
                 # cribbed from Lib/threading.py in Thread.wait() at r71065.
                 delay = 0.0005 # 500 us -> initial delay of 1 ms
diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -2777,19 +2777,5 @@
         self.assertTrue(proc.stdin.closed)
 
 
-def test_main():
-    unit_tests = (ProcessTestCase,
-                  POSIXProcessTestCase,
-                  Win32ProcessTestCase,
-                  MiscTests,
-                  ProcessTestCaseNoPoll,
-                  CommandsWithSpaces,
-                  ContextManagerTests,
-                  RunFuncTestCase,
-                  )
-
-    support.run_unittest(*unit_tests)
-    support.reap_children()
-
 if __name__ == "__main__":
     unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -131,6 +131,9 @@
 Library
 -------
 
+- Issue #20572: Remove the subprocess.Popen.wait endtime parameter.  It was
+  deprecated in 3.4 and undocumented prior to that.
+
 - Issue #25659: In ctypes, prevent a crash calling the from_buffer() and
   from_buffer_copy() methods on abstract classes like Array.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 19:35:35 2016
From: python-checkins at python.org (gregory.p.smith)
Date: Mon, 21 Nov 2016 00:35:35 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_issue_20572=3A_remove_the_?=
 =?utf-8?q?deprecation_notice_for_the_deleted_endtime_parameter=2E?=
Message-ID: <20161121003534.40861.32752.3B951E51@psf.io>

https://hg.python.org/cpython/rev/54b2f377653d
changeset:   105252:54b2f377653d
user:        Gregory P. Smith <greg at krypto.org>
date:        Sun Nov 20 16:35:30 2016 -0800
summary:
  issue 20572: remove the deprecation notice for the deleted endtime parameter.

files:
  Doc/library/subprocess.rst |  6 ------
  1 files changed, 0 insertions(+), 6 deletions(-)


diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst
--- a/Doc/library/subprocess.rst
+++ b/Doc/library/subprocess.rst
@@ -608,12 +608,6 @@
    .. versionchanged:: 3.3
       *timeout* was added.
 
-   .. deprecated:: 3.4
-
-      Do not use the *endtime* parameter.  It is was unintentionally
-      exposed in 3.3 but was left undocumented as it was intended to be
-      private for internal use.  Use *timeout* instead.
-
 .. method:: Popen.communicate(input=None, timeout=None)
 
    Interact with process: Send data to stdin.  Read data from stdout and stderr,

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 21:02:21 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Mon, 21 Nov 2016 02:02:21 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328556=3A_upstream_improvements_to_docstrings_and_erro?=
 =?utf-8?q?r_messages_by_Ivan?=
Message-ID: <20161121020221.8131.49819.B8D8FFE4@psf.io>

https://hg.python.org/cpython/rev/294525aac5eb
changeset:   105254:294525aac5eb
branch:      3.6
parent:      105250:0e8aa537c565
parent:      105253:75c7bc2c1ad8
user:        Guido van Rossum <guido at python.org>
date:        Sun Nov 20 18:01:33 2016 -0800
summary:
  Issue #28556: upstream improvements to docstrings and error messages by Ivan Levkivskyi (#331) (3.5->3.6)

files:
  Lib/typing.py |  86 ++++++++++++++++++++------------------
  1 files changed, 45 insertions(+), 41 deletions(-)


diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -100,16 +100,16 @@
 
 
 class TypingMeta(type):
-    """Metaclass for every type defined below.
+    """Metaclass for most types defined in typing module
+    (not a part of public API).
 
     This overrides __new__() to require an extra keyword parameter
     '_root', which serves as a guard against naive subclassing of the
     typing classes.  Any legitimate class defined using a metaclass
-    derived from TypingMeta (including internal subclasses created by
-    e.g.  Union[X, Y]) must pass _root=True.
-
-    This also defines a dummy constructor (all the work is done in
-    __new__) and a nicer repr().
+    derived from TypingMeta must pass _root=True.
+
+    This also defines a dummy constructor (all the work for most typing
+    constructs is done in __new__) and a nicer repr().
     """
 
     _is_protocol = False
@@ -126,8 +126,8 @@
     def _eval_type(self, globalns, localns):
         """Override this in subclasses to interpret forward references.
 
-        For example, Union['C'] is internally stored as
-        Union[_ForwardRef('C')], which should evaluate to _Union[C],
+        For example, List['C'] is internally stored as
+        List[_ForwardRef('C')], which should evaluate to List[C],
         where C is an object found in globalns or localns (searching
         localns first, of course).
         """
@@ -142,7 +142,7 @@
 
 
 class _TypingBase(metaclass=TypingMeta, _root=True):
-    """Indicator of special typing constructs."""
+    """Internal indicator of special typing constructs."""
 
     __slots__ = ()
 
@@ -179,10 +179,10 @@
 
 
 class _FinalTypingBase(_TypingBase, _root=True):
-    """Mix-in class to prevent instantiation.
+    """Internal mix-in class to prevent instantiation.
 
     Prevents instantiation unless _root=True is given in class call.
-    It is used to create pseudo-singleton instances Any, Union, Tuple, etc.
+    It is used to create pseudo-singleton instances Any, Union, Optional, etc.
     """
 
     __slots__ = ()
@@ -198,7 +198,7 @@
 
 
 class _ForwardRef(_TypingBase, _root=True):
-    """Wrapper to hold a forward reference."""
+    """Internal wrapper to hold a forward reference."""
 
     __slots__ = ('__forward_arg__', '__forward_code__',
                  '__forward_evaluated__', '__forward_value__')
@@ -206,11 +206,11 @@
     def __init__(self, arg):
         super().__init__(arg)
         if not isinstance(arg, str):
-            raise TypeError('ForwardRef must be a string -- got %r' % (arg,))
+            raise TypeError('Forward reference must be a string -- got %r' % (arg,))
         try:
             code = compile(arg, '<string>', 'eval')
         except SyntaxError:
-            raise SyntaxError('ForwardRef must be an expression -- got %r' %
+            raise SyntaxError('Forward reference must be an expression -- got %r' %
                               (arg,))
         self.__forward_arg__ = arg
         self.__forward_code__ = code
@@ -336,7 +336,7 @@
 
 
 def _type_check(arg, msg):
-    """Check that the argument is a type, and return it.
+    """Check that the argument is a type, and return it (internal helper).
 
     As a special case, accept None and return type(None) instead.
     Also, _TypeAlias instances (e.g. Match, Pattern) are acceptable.
@@ -363,7 +363,7 @@
 
 
 def _type_repr(obj):
-    """Return the repr() of an object, special-casing types.
+    """Return the repr() of an object, special-casing types (internal helper).
 
     If obj is a type, we return a shorter version than the default
     type.__repr__, based on the module and qualified name, which is
@@ -418,7 +418,7 @@
     as for generic function definitions.  See class Generic for more
     information on generic types.  Generic functions work as follows:
 
-      def repeat(x: T, n: int) -> Sequence[T]:
+      def repeat(x: T, n: int) -> List[T]:
           '''Return a list containing n references to x.'''
           return [x]*n
 
@@ -431,10 +431,7 @@
     that if the arguments are instances of some subclass of str,
     the return type is still plain str.
 
-    At runtime, isinstance(x, T) will raise TypeError.  However,
-    issubclass(C, T) is true for any class C, and issubclass(str, A)
-    and issubclass(bytes, A) are true, and issubclass(int, A) is
-    false.  (TODO: Why is this needed?  This may change.  See #136.)
+    At runtime, isinstance(x, T) and issubclass(C, T) will raise TypeError.
 
     Type variables defined with covariant=True or contravariant=True
     can be used do declare covariant or contravariant generic types.
@@ -509,7 +506,7 @@
 
 
 def _replace_arg(arg, tvars, args):
-    """ A helper fuunction: replace arg if it is a type variable
+    """An internal helper function: replace arg if it is a type variable
     found in tvars with corresponding substitution from args or
     with corresponding substitution sub-tree if arg is a generic type.
     """
@@ -526,9 +523,15 @@
 
 
 def _subs_tree(cls, tvars=None, args=None):
-    """ Calculate substitution tree for generic cls after
-    replacing its type parameters with substitutions in tvars -> args (if any).
-    Repeat the same cyclicaly following __origin__'s.
+    """An internal helper function: calculate substitution tree
+    for generic cls after replacing its type parameters with
+    substitutions in tvars -> args (if any).
+    Repeat the same following __origin__'s.
+
+    Return a list of arguments with all possible substitutions
+    performed. Arguments that are generic classes themselves are represented
+    as tuples (so that no new classes are created by this function).
+    For example: _subs_tree(List[Tuple[int, T]][str]) == [(Tuple, int, str)]
     """
 
     if cls.__origin__ is None:
@@ -553,7 +556,7 @@
 
 
 def _remove_dups_flatten(parameters):
-    """ A helper for Union creation and substitution: flatten Union's
+    """An internal helper for Union creation and substitution: flatten Union's
     among parameters, then remove duplicates and strict subclasses.
     """
 
@@ -594,7 +597,7 @@
 
 
 def _check_generic(cls, parameters):
-    # Check correct count for parameters of a generic cls.
+    # Check correct count for parameters of a generic cls (internal helper).
     if not cls.__parameters__:
         raise TypeError("%s is not a generic class" % repr(cls))
     alen = len(parameters)
@@ -608,7 +611,7 @@
 
 
 def _tp_cache(func):
-    """ Caching for __getitem__ of generic types with a fallback to
+    """Internal wrapper caching __getitem__ of generic types with a fallback to
     original function for non-hashable arguments.
     """
 
@@ -788,7 +791,7 @@
 
 
 def _gorg(a):
-    """Return the farthest origin of a generic class."""
+    """Return the farthest origin of a generic class (internal helper)."""
     assert isinstance(a, GenericMeta)
     while a.__origin__ is not None:
         a = a.__origin__
@@ -796,10 +799,10 @@
 
 
 def _geqv(a, b):
-    """Return whether two generic classes are equivalent.
+    """Return whether two generic classes are equivalent (internal helper).
 
     The intention is to consider generic class X and any of its
-    parameterized forms (X[T], X[int], etc.)  as equivalent.
+    parameterized forms (X[T], X[int], etc.) as equivalent.
 
     However, X is not equivalent to a subclass of X.
 
@@ -825,6 +828,7 @@
 
 
 def _valid_for_check(cls):
+    """An internal helper to prohibit isinstance([1], List[str]) etc."""
     if cls is Generic:
         raise TypeError("Class %r cannot be used with class "
                         "or instance checks" % cls)
@@ -1083,8 +1087,8 @@
 class Generic(metaclass=GenericMeta):
     """Abstract base class for generic types.
 
-    A generic type is typically declared by inheriting from an
-    instantiation of this class with one or more type variables.
+    A generic type is typically declared by inheriting from
+    this class parameterized with one or more type variables.
     For example, a generic mapping type might be defined as::
 
       class Mapping(Generic[KT, VT]):
@@ -1111,18 +1115,18 @@
 
 
 class _TypingEmpty:
-    """Placeholder for () or []. Used by TupleMeta and CallableMeta
-    to allow empy list/tuple in specific places, without allowing them
+    """Internal placeholder for () or []. Used by TupleMeta and CallableMeta
+    to allow empty list/tuple in specific places, without allowing them
     to sneak in where prohibited.
     """
 
 
 class _TypingEllipsis:
-    """Ditto for ..."""
+    """Internal placeholder for ... (ellipsis)."""
 
 
 class TupleMeta(GenericMeta):
-    """Metaclass for Tuple"""
+    """Metaclass for Tuple (internal)."""
 
     @_tp_cache
     def __getitem__(self, parameters):
@@ -1175,7 +1179,7 @@
 
 
 class CallableMeta(GenericMeta):
-    """ Metaclass for Callable."""
+    """Metaclass for Callable (internal)."""
 
     def __repr__(self):
         if self.__origin__ is None:
@@ -1199,7 +1203,7 @@
                 '[[%s], %s]' % (', '.join(arg_list[:-1]), arg_list[-1]))
 
     def __getitem__(self, parameters):
-        """ A thin wrapper around __getitem_inner__ to provide the latter
+        """A thin wrapper around __getitem_inner__ to provide the latter
         with hashable arguments to improve speed.
         """
 
@@ -1236,7 +1240,7 @@
 
     The subscription syntax must always be used with exactly two
     values: the argument list and the return type.  The argument list
-    must be a list of types; the return type must be a single type.
+    must be a list of types or ellipsis; the return type must be a single type.
 
     There is no syntax to indicate optional or keyword arguments,
     such function types are rarely used as callback types.
@@ -1565,7 +1569,7 @@
 class _Protocol(metaclass=_ProtocolMeta):
     """Internal base class for protocol classes.
 
-    This implements a simple-minded structural isinstance check
+    This implements a simple-minded structural issubclass check
     (similar but more general than the one-offs in collections.abc
     such as Hashable).
     """

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 21:02:21 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Mon, 21 Nov 2016 02:02:21 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328556=3A_upstream_improvements_to_docstrings_an?=
 =?utf-8?q?d_error_messages_by_Ivan?=
Message-ID: <20161121020221.27875.9775.1E83BCA2@psf.io>

https://hg.python.org/cpython/rev/30f154d9abf0
changeset:   105255:30f154d9abf0
parent:      105252:54b2f377653d
parent:      105254:294525aac5eb
user:        Guido van Rossum <guido at python.org>
date:        Sun Nov 20 18:01:37 2016 -0800
summary:
  Issue #28556: upstream improvements to docstrings and error messages by Ivan Levkivskyi (#331) (3.6->3.7)

files:
  Lib/typing.py |  86 ++++++++++++++++++++------------------
  1 files changed, 45 insertions(+), 41 deletions(-)


diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -100,16 +100,16 @@
 
 
 class TypingMeta(type):
-    """Metaclass for every type defined below.
+    """Metaclass for most types defined in typing module
+    (not a part of public API).
 
     This overrides __new__() to require an extra keyword parameter
     '_root', which serves as a guard against naive subclassing of the
     typing classes.  Any legitimate class defined using a metaclass
-    derived from TypingMeta (including internal subclasses created by
-    e.g.  Union[X, Y]) must pass _root=True.
-
-    This also defines a dummy constructor (all the work is done in
-    __new__) and a nicer repr().
+    derived from TypingMeta must pass _root=True.
+
+    This also defines a dummy constructor (all the work for most typing
+    constructs is done in __new__) and a nicer repr().
     """
 
     _is_protocol = False
@@ -126,8 +126,8 @@
     def _eval_type(self, globalns, localns):
         """Override this in subclasses to interpret forward references.
 
-        For example, Union['C'] is internally stored as
-        Union[_ForwardRef('C')], which should evaluate to _Union[C],
+        For example, List['C'] is internally stored as
+        List[_ForwardRef('C')], which should evaluate to List[C],
         where C is an object found in globalns or localns (searching
         localns first, of course).
         """
@@ -142,7 +142,7 @@
 
 
 class _TypingBase(metaclass=TypingMeta, _root=True):
-    """Indicator of special typing constructs."""
+    """Internal indicator of special typing constructs."""
 
     __slots__ = ()
 
@@ -179,10 +179,10 @@
 
 
 class _FinalTypingBase(_TypingBase, _root=True):
-    """Mix-in class to prevent instantiation.
+    """Internal mix-in class to prevent instantiation.
 
     Prevents instantiation unless _root=True is given in class call.
-    It is used to create pseudo-singleton instances Any, Union, Tuple, etc.
+    It is used to create pseudo-singleton instances Any, Union, Optional, etc.
     """
 
     __slots__ = ()
@@ -198,7 +198,7 @@
 
 
 class _ForwardRef(_TypingBase, _root=True):
-    """Wrapper to hold a forward reference."""
+    """Internal wrapper to hold a forward reference."""
 
     __slots__ = ('__forward_arg__', '__forward_code__',
                  '__forward_evaluated__', '__forward_value__')
@@ -206,11 +206,11 @@
     def __init__(self, arg):
         super().__init__(arg)
         if not isinstance(arg, str):
-            raise TypeError('ForwardRef must be a string -- got %r' % (arg,))
+            raise TypeError('Forward reference must be a string -- got %r' % (arg,))
         try:
             code = compile(arg, '<string>', 'eval')
         except SyntaxError:
-            raise SyntaxError('ForwardRef must be an expression -- got %r' %
+            raise SyntaxError('Forward reference must be an expression -- got %r' %
                               (arg,))
         self.__forward_arg__ = arg
         self.__forward_code__ = code
@@ -336,7 +336,7 @@
 
 
 def _type_check(arg, msg):
-    """Check that the argument is a type, and return it.
+    """Check that the argument is a type, and return it (internal helper).
 
     As a special case, accept None and return type(None) instead.
     Also, _TypeAlias instances (e.g. Match, Pattern) are acceptable.
@@ -363,7 +363,7 @@
 
 
 def _type_repr(obj):
-    """Return the repr() of an object, special-casing types.
+    """Return the repr() of an object, special-casing types (internal helper).
 
     If obj is a type, we return a shorter version than the default
     type.__repr__, based on the module and qualified name, which is
@@ -418,7 +418,7 @@
     as for generic function definitions.  See class Generic for more
     information on generic types.  Generic functions work as follows:
 
-      def repeat(x: T, n: int) -> Sequence[T]:
+      def repeat(x: T, n: int) -> List[T]:
           '''Return a list containing n references to x.'''
           return [x]*n
 
@@ -431,10 +431,7 @@
     that if the arguments are instances of some subclass of str,
     the return type is still plain str.
 
-    At runtime, isinstance(x, T) will raise TypeError.  However,
-    issubclass(C, T) is true for any class C, and issubclass(str, A)
-    and issubclass(bytes, A) are true, and issubclass(int, A) is
-    false.  (TODO: Why is this needed?  This may change.  See #136.)
+    At runtime, isinstance(x, T) and issubclass(C, T) will raise TypeError.
 
     Type variables defined with covariant=True or contravariant=True
     can be used do declare covariant or contravariant generic types.
@@ -509,7 +506,7 @@
 
 
 def _replace_arg(arg, tvars, args):
-    """ A helper fuunction: replace arg if it is a type variable
+    """An internal helper function: replace arg if it is a type variable
     found in tvars with corresponding substitution from args or
     with corresponding substitution sub-tree if arg is a generic type.
     """
@@ -526,9 +523,15 @@
 
 
 def _subs_tree(cls, tvars=None, args=None):
-    """ Calculate substitution tree for generic cls after
-    replacing its type parameters with substitutions in tvars -> args (if any).
-    Repeat the same cyclicaly following __origin__'s.
+    """An internal helper function: calculate substitution tree
+    for generic cls after replacing its type parameters with
+    substitutions in tvars -> args (if any).
+    Repeat the same following __origin__'s.
+
+    Return a list of arguments with all possible substitutions
+    performed. Arguments that are generic classes themselves are represented
+    as tuples (so that no new classes are created by this function).
+    For example: _subs_tree(List[Tuple[int, T]][str]) == [(Tuple, int, str)]
     """
 
     if cls.__origin__ is None:
@@ -553,7 +556,7 @@
 
 
 def _remove_dups_flatten(parameters):
-    """ A helper for Union creation and substitution: flatten Union's
+    """An internal helper for Union creation and substitution: flatten Union's
     among parameters, then remove duplicates and strict subclasses.
     """
 
@@ -594,7 +597,7 @@
 
 
 def _check_generic(cls, parameters):
-    # Check correct count for parameters of a generic cls.
+    # Check correct count for parameters of a generic cls (internal helper).
     if not cls.__parameters__:
         raise TypeError("%s is not a generic class" % repr(cls))
     alen = len(parameters)
@@ -608,7 +611,7 @@
 
 
 def _tp_cache(func):
-    """ Caching for __getitem__ of generic types with a fallback to
+    """Internal wrapper caching __getitem__ of generic types with a fallback to
     original function for non-hashable arguments.
     """
 
@@ -788,7 +791,7 @@
 
 
 def _gorg(a):
-    """Return the farthest origin of a generic class."""
+    """Return the farthest origin of a generic class (internal helper)."""
     assert isinstance(a, GenericMeta)
     while a.__origin__ is not None:
         a = a.__origin__
@@ -796,10 +799,10 @@
 
 
 def _geqv(a, b):
-    """Return whether two generic classes are equivalent.
+    """Return whether two generic classes are equivalent (internal helper).
 
     The intention is to consider generic class X and any of its
-    parameterized forms (X[T], X[int], etc.)  as equivalent.
+    parameterized forms (X[T], X[int], etc.) as equivalent.
 
     However, X is not equivalent to a subclass of X.
 
@@ -825,6 +828,7 @@
 
 
 def _valid_for_check(cls):
+    """An internal helper to prohibit isinstance([1], List[str]) etc."""
     if cls is Generic:
         raise TypeError("Class %r cannot be used with class "
                         "or instance checks" % cls)
@@ -1083,8 +1087,8 @@
 class Generic(metaclass=GenericMeta):
     """Abstract base class for generic types.
 
-    A generic type is typically declared by inheriting from an
-    instantiation of this class with one or more type variables.
+    A generic type is typically declared by inheriting from
+    this class parameterized with one or more type variables.
     For example, a generic mapping type might be defined as::
 
       class Mapping(Generic[KT, VT]):
@@ -1111,18 +1115,18 @@
 
 
 class _TypingEmpty:
-    """Placeholder for () or []. Used by TupleMeta and CallableMeta
-    to allow empy list/tuple in specific places, without allowing them
+    """Internal placeholder for () or []. Used by TupleMeta and CallableMeta
+    to allow empty list/tuple in specific places, without allowing them
     to sneak in where prohibited.
     """
 
 
 class _TypingEllipsis:
-    """Ditto for ..."""
+    """Internal placeholder for ... (ellipsis)."""
 
 
 class TupleMeta(GenericMeta):
-    """Metaclass for Tuple"""
+    """Metaclass for Tuple (internal)."""
 
     @_tp_cache
     def __getitem__(self, parameters):
@@ -1175,7 +1179,7 @@
 
 
 class CallableMeta(GenericMeta):
-    """ Metaclass for Callable."""
+    """Metaclass for Callable (internal)."""
 
     def __repr__(self):
         if self.__origin__ is None:
@@ -1199,7 +1203,7 @@
                 '[[%s], %s]' % (', '.join(arg_list[:-1]), arg_list[-1]))
 
     def __getitem__(self, parameters):
-        """ A thin wrapper around __getitem_inner__ to provide the latter
+        """A thin wrapper around __getitem_inner__ to provide the latter
         with hashable arguments to improve speed.
         """
 
@@ -1236,7 +1240,7 @@
 
     The subscription syntax must always be used with exactly two
     values: the argument list and the return type.  The argument list
-    must be a list of types; the return type must be a single type.
+    must be a list of types or ellipsis; the return type must be a single type.
 
     There is no syntax to indicate optional or keyword arguments,
     such function types are rarely used as callback types.
@@ -1565,7 +1569,7 @@
 class _Protocol(metaclass=_ProtocolMeta):
     """Internal base class for protocol classes.
 
-    This implements a simple-minded structural isinstance check
+    This implements a simple-minded structural issubclass check
     (similar but more general than the one-offs in collections.abc
     such as Hashable).
     """

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 21:02:21 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Mon, 21 Nov 2016 02:02:21 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NTU2?=
 =?utf-8?q?=3A_upstream_improvements_to_docstrings_and_error_messages_by_I?=
 =?utf-8?q?van?=
Message-ID: <20161121020221.10307.6902.81663E89@psf.io>

https://hg.python.org/cpython/rev/75c7bc2c1ad8
changeset:   105253:75c7bc2c1ad8
branch:      3.5
parent:      105240:5f061870d49c
user:        Guido van Rossum <guido at python.org>
date:        Sun Nov 20 18:01:29 2016 -0800
summary:
  Issue #28556: upstream improvements to docstrings and error messages by Ivan Levkivskyi (#331)

files:
  Lib/typing.py |  86 ++++++++++++++++++++------------------
  1 files changed, 45 insertions(+), 41 deletions(-)


diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -100,16 +100,16 @@
 
 
 class TypingMeta(type):
-    """Metaclass for every type defined below.
+    """Metaclass for most types defined in typing module
+    (not a part of public API).
 
     This overrides __new__() to require an extra keyword parameter
     '_root', which serves as a guard against naive subclassing of the
     typing classes.  Any legitimate class defined using a metaclass
-    derived from TypingMeta (including internal subclasses created by
-    e.g.  Union[X, Y]) must pass _root=True.
-
-    This also defines a dummy constructor (all the work is done in
-    __new__) and a nicer repr().
+    derived from TypingMeta must pass _root=True.
+
+    This also defines a dummy constructor (all the work for most typing
+    constructs is done in __new__) and a nicer repr().
     """
 
     _is_protocol = False
@@ -126,8 +126,8 @@
     def _eval_type(self, globalns, localns):
         """Override this in subclasses to interpret forward references.
 
-        For example, Union['C'] is internally stored as
-        Union[_ForwardRef('C')], which should evaluate to _Union[C],
+        For example, List['C'] is internally stored as
+        List[_ForwardRef('C')], which should evaluate to List[C],
         where C is an object found in globalns or localns (searching
         localns first, of course).
         """
@@ -142,7 +142,7 @@
 
 
 class _TypingBase(metaclass=TypingMeta, _root=True):
-    """Indicator of special typing constructs."""
+    """Internal indicator of special typing constructs."""
 
     __slots__ = ()
 
@@ -179,10 +179,10 @@
 
 
 class _FinalTypingBase(_TypingBase, _root=True):
-    """Mix-in class to prevent instantiation.
+    """Internal mix-in class to prevent instantiation.
 
     Prevents instantiation unless _root=True is given in class call.
-    It is used to create pseudo-singleton instances Any, Union, Tuple, etc.
+    It is used to create pseudo-singleton instances Any, Union, Optional, etc.
     """
 
     __slots__ = ()
@@ -198,7 +198,7 @@
 
 
 class _ForwardRef(_TypingBase, _root=True):
-    """Wrapper to hold a forward reference."""
+    """Internal wrapper to hold a forward reference."""
 
     __slots__ = ('__forward_arg__', '__forward_code__',
                  '__forward_evaluated__', '__forward_value__')
@@ -206,11 +206,11 @@
     def __init__(self, arg):
         super().__init__(arg)
         if not isinstance(arg, str):
-            raise TypeError('ForwardRef must be a string -- got %r' % (arg,))
+            raise TypeError('Forward reference must be a string -- got %r' % (arg,))
         try:
             code = compile(arg, '<string>', 'eval')
         except SyntaxError:
-            raise SyntaxError('ForwardRef must be an expression -- got %r' %
+            raise SyntaxError('Forward reference must be an expression -- got %r' %
                               (arg,))
         self.__forward_arg__ = arg
         self.__forward_code__ = code
@@ -336,7 +336,7 @@
 
 
 def _type_check(arg, msg):
-    """Check that the argument is a type, and return it.
+    """Check that the argument is a type, and return it (internal helper).
 
     As a special case, accept None and return type(None) instead.
     Also, _TypeAlias instances (e.g. Match, Pattern) are acceptable.
@@ -363,7 +363,7 @@
 
 
 def _type_repr(obj):
-    """Return the repr() of an object, special-casing types.
+    """Return the repr() of an object, special-casing types (internal helper).
 
     If obj is a type, we return a shorter version than the default
     type.__repr__, based on the module and qualified name, which is
@@ -418,7 +418,7 @@
     as for generic function definitions.  See class Generic for more
     information on generic types.  Generic functions work as follows:
 
-      def repeat(x: T, n: int) -> Sequence[T]:
+      def repeat(x: T, n: int) -> List[T]:
           '''Return a list containing n references to x.'''
           return [x]*n
 
@@ -431,10 +431,7 @@
     that if the arguments are instances of some subclass of str,
     the return type is still plain str.
 
-    At runtime, isinstance(x, T) will raise TypeError.  However,
-    issubclass(C, T) is true for any class C, and issubclass(str, A)
-    and issubclass(bytes, A) are true, and issubclass(int, A) is
-    false.  (TODO: Why is this needed?  This may change.  See #136.)
+    At runtime, isinstance(x, T) and issubclass(C, T) will raise TypeError.
 
     Type variables defined with covariant=True or contravariant=True
     can be used do declare covariant or contravariant generic types.
@@ -509,7 +506,7 @@
 
 
 def _replace_arg(arg, tvars, args):
-    """ A helper fuunction: replace arg if it is a type variable
+    """An internal helper function: replace arg if it is a type variable
     found in tvars with corresponding substitution from args or
     with corresponding substitution sub-tree if arg is a generic type.
     """
@@ -526,9 +523,15 @@
 
 
 def _subs_tree(cls, tvars=None, args=None):
-    """ Calculate substitution tree for generic cls after
-    replacing its type parameters with substitutions in tvars -> args (if any).
-    Repeat the same cyclicaly following __origin__'s.
+    """An internal helper function: calculate substitution tree
+    for generic cls after replacing its type parameters with
+    substitutions in tvars -> args (if any).
+    Repeat the same following __origin__'s.
+
+    Return a list of arguments with all possible substitutions
+    performed. Arguments that are generic classes themselves are represented
+    as tuples (so that no new classes are created by this function).
+    For example: _subs_tree(List[Tuple[int, T]][str]) == [(Tuple, int, str)]
     """
 
     if cls.__origin__ is None:
@@ -553,7 +556,7 @@
 
 
 def _remove_dups_flatten(parameters):
-    """ A helper for Union creation and substitution: flatten Union's
+    """An internal helper for Union creation and substitution: flatten Union's
     among parameters, then remove duplicates and strict subclasses.
     """
 
@@ -594,7 +597,7 @@
 
 
 def _check_generic(cls, parameters):
-    # Check correct count for parameters of a generic cls.
+    # Check correct count for parameters of a generic cls (internal helper).
     if not cls.__parameters__:
         raise TypeError("%s is not a generic class" % repr(cls))
     alen = len(parameters)
@@ -608,7 +611,7 @@
 
 
 def _tp_cache(func):
-    """ Caching for __getitem__ of generic types with a fallback to
+    """Internal wrapper caching __getitem__ of generic types with a fallback to
     original function for non-hashable arguments.
     """
 
@@ -788,7 +791,7 @@
 
 
 def _gorg(a):
-    """Return the farthest origin of a generic class."""
+    """Return the farthest origin of a generic class (internal helper)."""
     assert isinstance(a, GenericMeta)
     while a.__origin__ is not None:
         a = a.__origin__
@@ -796,10 +799,10 @@
 
 
 def _geqv(a, b):
-    """Return whether two generic classes are equivalent.
+    """Return whether two generic classes are equivalent (internal helper).
 
     The intention is to consider generic class X and any of its
-    parameterized forms (X[T], X[int], etc.)  as equivalent.
+    parameterized forms (X[T], X[int], etc.) as equivalent.
 
     However, X is not equivalent to a subclass of X.
 
@@ -825,6 +828,7 @@
 
 
 def _valid_for_check(cls):
+    """An internal helper to prohibit isinstance([1], List[str]) etc."""
     if cls is Generic:
         raise TypeError("Class %r cannot be used with class "
                         "or instance checks" % cls)
@@ -1083,8 +1087,8 @@
 class Generic(metaclass=GenericMeta):
     """Abstract base class for generic types.
 
-    A generic type is typically declared by inheriting from an
-    instantiation of this class with one or more type variables.
+    A generic type is typically declared by inheriting from
+    this class parameterized with one or more type variables.
     For example, a generic mapping type might be defined as::
 
       class Mapping(Generic[KT, VT]):
@@ -1111,18 +1115,18 @@
 
 
 class _TypingEmpty:
-    """Placeholder for () or []. Used by TupleMeta and CallableMeta
-    to allow empy list/tuple in specific places, without allowing them
+    """Internal placeholder for () or []. Used by TupleMeta and CallableMeta
+    to allow empty list/tuple in specific places, without allowing them
     to sneak in where prohibited.
     """
 
 
 class _TypingEllipsis:
-    """Ditto for ..."""
+    """Internal placeholder for ... (ellipsis)."""
 
 
 class TupleMeta(GenericMeta):
-    """Metaclass for Tuple"""
+    """Metaclass for Tuple (internal)."""
 
     @_tp_cache
     def __getitem__(self, parameters):
@@ -1175,7 +1179,7 @@
 
 
 class CallableMeta(GenericMeta):
-    """ Metaclass for Callable."""
+    """Metaclass for Callable (internal)."""
 
     def __repr__(self):
         if self.__origin__ is None:
@@ -1199,7 +1203,7 @@
                 '[[%s], %s]' % (', '.join(arg_list[:-1]), arg_list[-1]))
 
     def __getitem__(self, parameters):
-        """ A thin wrapper around __getitem_inner__ to provide the latter
+        """A thin wrapper around __getitem_inner__ to provide the latter
         with hashable arguments to improve speed.
         """
 
@@ -1236,7 +1240,7 @@
 
     The subscription syntax must always be used with exactly two
     values: the argument list and the return type.  The argument list
-    must be a list of types; the return type must be a single type.
+    must be a list of types or ellipsis; the return type must be a single type.
 
     There is no syntax to indicate optional or keyword arguments,
     such function types are rarely used as callback types.
@@ -1565,7 +1569,7 @@
 class _Protocol(metaclass=_ProtocolMeta):
     """Internal base class for protocol classes.
 
-    This implements a simple-minded structural isinstance check
+    This implements a simple-minded structural issubclass check
     (similar but more general than the one-offs in collections.abc
     such as Hashable).
     """

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 23:23:49 2016
From: python-checkins at python.org (martin.panter)
Date: Mon, 21 Nov 2016 04:23:49 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Change_double_?=
 =?utf-8?q?hyphens_=28en_dashes=29_to_em_=28longer=29_dashes?=
Message-ID: <20161121042349.45206.47887.71ED958A@psf.io>

https://hg.python.org/cpython/rev/7da35bced88e
changeset:   105256:7da35bced88e
branch:      2.7
parent:      105249:4de956751cf1
user:        Martin Panter <vadmium+py at gmail.com>
date:        Mon Nov 21 00:21:39 2016 +0000
summary:
  Change double hyphens (en dashes) to em (longer) dashes

files:
  Doc/howto/cporting.rst  |  2 +-
  Doc/howto/curses.rst    |  2 +-
  Doc/library/hmac.rst    |  2 +-
  Doc/library/io.rst      |  2 +-
  Doc/library/pdb.rst     |  2 +-
  Doc/library/whichdb.rst |  4 ++--
  6 files changed, 7 insertions(+), 7 deletions(-)


diff --git a/Doc/howto/cporting.rst b/Doc/howto/cporting.rst
--- a/Doc/howto/cporting.rst
+++ b/Doc/howto/cporting.rst
@@ -95,7 +95,7 @@
 --------------------
 
 Python 3 has only one integer type, :func:`int`.  But it actually
-corresponds to Python 2's :func:`long` type--the :func:`int` type
+corresponds to Python 2's :func:`long` type?the :func:`int` type
 used in Python 2 was removed.  In the C-API, ``PyInt_*`` functions
 are replaced by their ``PyLong_*`` equivalents.
 
diff --git a/Doc/howto/curses.rst b/Doc/howto/curses.rst
--- a/Doc/howto/curses.rst
+++ b/Doc/howto/curses.rst
@@ -35,7 +35,7 @@
 The curses library hides all the details of different terminals, and provides
 the programmer with an abstraction of a display, containing multiple
 non-overlapping windows.  The contents of a window can be changed in various
-ways-- adding text, erasing it, changing its appearance--and the curses library
+ways---adding text, erasing it, changing its appearance---and the curses library
 will automagically figure out what control codes need to be sent to the terminal
 to produce the right output.
 
diff --git a/Doc/library/hmac.rst b/Doc/library/hmac.rst
--- a/Doc/library/hmac.rst
+++ b/Doc/library/hmac.rst
@@ -79,7 +79,7 @@
 
       If *a* and *b* are of different lengths, or if an error occurs,
       a timing attack could theoretically reveal information about the
-      types and lengths of *a* and *b*--but not their values.
+      types and lengths of *a* and *b*?but not their values.
 
 
    .. versionadded:: 2.7.7
diff --git a/Doc/library/io.rst b/Doc/library/io.rst
--- a/Doc/library/io.rst
+++ b/Doc/library/io.rst
@@ -130,7 +130,7 @@
    the list of supported encodings.
 
    *errors* is an optional string that specifies how encoding and decoding
-   errors are to be handled--this cannot be used in binary mode.  Pass
+   errors are to be handled?this cannot be used in binary mode.  Pass
    ``'strict'`` to raise a :exc:`ValueError` exception if there is an encoding
    error (the default of ``None`` has the same effect), or pass ``'ignore'`` to
    ignore errors.  (Note that ignoring encoding errors can lead to data loss.)
diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst
--- a/Doc/library/pdb.rst
+++ b/Doc/library/pdb.rst
@@ -290,7 +290,7 @@
    return, jump, quit and their abbreviations) terminates the command list (as if
    that command was immediately followed by end). This is because any time you
    resume execution (even with a simple next or step), you may encounter another
-   breakpoint--which could have its own command list, leading to ambiguities about
+   breakpoint?which could have its own command list, leading to ambiguities about
    which list to execute.
 
    If you use the 'silent' command in the command list, the usual message about
diff --git a/Doc/library/whichdb.rst b/Doc/library/whichdb.rst
--- a/Doc/library/whichdb.rst
+++ b/Doc/library/whichdb.rst
@@ -11,8 +11,8 @@
 
 
 The single function in this module attempts to guess which of the several simple
-database modules available--\ :mod:`dbm`, :mod:`gdbm`, or :mod:`dbhash`\
---should be used to open a given file.
+database modules available?:mod:`dbm`, :mod:`gdbm`, or :mod:`dbhash`\
+?should be used to open a given file.
 
 
 .. function:: whichdb(filename)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 23:23:49 2016
From: python-checkins at python.org (martin.panter)
Date: Mon, 21 Nov 2016 04:23:49 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Another_en_?=
 =?utf-8?q?=E2=86=92_em_dash_fix_for_3=2E6?=
Message-ID: <20161121042349.10307.92963.F5653A35@psf.io>

https://hg.python.org/cpython/rev/b72391f54083
changeset:   105259:b72391f54083
branch:      3.6
user:        Martin Panter <vadmium+py at gmail.com>
date:        Mon Nov 21 04:10:45 2016 +0000
summary:
  Another en ? em dash fix for 3.6

files:
  Doc/reference/datamodel.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst
--- a/Doc/reference/datamodel.rst
+++ b/Doc/reference/datamodel.rst
@@ -2550,7 +2550,7 @@
 .. [#] "Does not support" here means that the class has no such method, or
    the method returns ``NotImplemented``.  Do not set the method to
    ``None`` if you want to force fallback to the right operand's reflected
-   method--that will instead have the opposite effect of explicitly
+   method?that will instead have the opposite effect of explicitly
    *blocking* such fallback.
 
 .. [#] For operands of the same type, it is assumed that if the non-reflected method

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 23:23:49 2016
From: python-checkins at python.org (martin.panter)
Date: Mon, 21 Nov 2016 04:23:49 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Change_double_?=
 =?utf-8?q?hyphens_=28en_dashes=29_to_em_=28longer=29_dashes?=
Message-ID: <20161121042349.28106.94528.94EF45A0@psf.io>

https://hg.python.org/cpython/rev/f17afa9327ad
changeset:   105257:f17afa9327ad
branch:      3.5
parent:      105253:75c7bc2c1ad8
user:        Martin Panter <vadmium+py at gmail.com>
date:        Mon Nov 21 00:15:20 2016 +0000
summary:
  Change double hyphens (en dashes) to em (longer) dashes

files:
  Doc/howto/clinic.rst      |  30 +++++++++++++-------------
  Doc/howto/cporting.rst    |   2 +-
  Doc/library/functions.rst |   2 +-
  Doc/library/hmac.rst      |   2 +-
  Doc/library/pdb.rst       |   2 +-
  Doc/library/shutil.rst    |   2 +-
  6 files changed, 20 insertions(+), 20 deletions(-)


diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst
--- a/Doc/howto/clinic.rst
+++ b/Doc/howto/clinic.rst
@@ -30,7 +30,7 @@
 is to take over responsibility for all argument parsing code
 inside CPython.  This means that, when you convert a function
 to work with Argument Clinic, that function should no longer
-do any of its own argument parsing--the code generated by
+do any of its own argument parsing?the code generated by
 Argument Clinic should be a "black box" to you, where CPython
 calls in at the top, and your code gets called at the bottom,
 with ``PyObject *args`` (and maybe ``PyObject *kwargs``)
@@ -43,12 +43,12 @@
 When you use Argument Clinic, you don't have to repeat yourself.
 
 Obviously, no one would want to use Argument Clinic unless
-it's solving their problem--and without creating new problems of
+it's solving their problem?and without creating new problems of
 its own.
 So it's paramount that Argument Clinic generate correct code.
 It'd be nice if the code was faster, too, but at the very least
 it should not introduce a major speed regression.  (Eventually Argument
-Clinic *should* make a major speedup possible--we could
+Clinic *should* make a major speedup possible?we could
 rewrite its code generator to produce tailor-made argument
 parsing code, rather than calling the general-purpose CPython
 argument parsing library.  That would make for the fastest
@@ -113,7 +113,7 @@
 
 You should never modify the output portion of an Argument Clinic block.  Instead,
 change the input until it produces the output you want.  (That's the purpose of the
-checksum--to detect if someone changed the output, as these edits would be lost
+checksum?to detect if someone changed the output, as these edits would be lost
 the next time Argument Clinic writes out fresh output.)
 
 For the sake of clarity, here's the terminology we'll use with Argument Clinic:
@@ -166,7 +166,7 @@
    or if it has multiple calls to :c:func:`PyArg_ParseTuple`,
    you should choose a different function.  Argument Clinic *does*
    support all of these scenarios.  But these are advanced
-   topics--let's do something simpler for your first function.
+   topics?let's do something simpler for your first function.
 
    Also, if the function has multiple calls to :c:func:`PyArg_ParseTuple`
    or :c:func:`PyArg_ParseTupleAndKeywords` where it supports different
@@ -188,7 +188,7 @@
 
    If the old docstring had a first line that looked like a function
    signature, throw that line away.  (The docstring doesn't need it
-   anymore--when you use ``help()`` on your builtin in the future,
+   anymore?when you use ``help()`` on your builtin in the future,
    the first line will be built automatically based on the function's
    signature.)
 
@@ -209,7 +209,7 @@
 6. Above the docstring, enter the name of the function, followed
    by a blank line.  This should be the Python name of the function,
    and should be the full dotted path
-   to the function--it should start with the name of the module,
+   to the function?it should start with the name of the module,
    include any sub-modules, and if the function is a method on
    a class it should include the class name too.
 
@@ -275,7 +275,7 @@
    What's a "converter"?  It establishes both the type
    of the variable used in C, and the method to convert the Python
    value into a C value at runtime.
-   For now you're going to use what's called a "legacy converter"--a
+   For now you're going to use what's called a "legacy converter"?a
    convenience syntax intended to make porting old code into Argument
    Clinic easier.
 
@@ -424,7 +424,7 @@
     (Argument Clinic always generates its format strings
     with a ``:`` followed by the name of the function.  If the
     existing code's format string ends with ``;``, to provide
-    usage help, this change is harmless--don't worry about it.)
+    usage help, this change is harmless?don't worry about it.)
 
     Third, for parameters whose format units require two arguments
     (like a length variable, or an encoding string, or a pointer
@@ -637,7 +637,7 @@
 Another example is ``curses.window.addch()``, which has a group of two
 arguments that must always be specified together.  (The arguments are
 called ``x`` and ``y``; if you call the function passing in ``x``,
-you must also pass in ``y``--and if you don't pass in ``x`` you may not
+you must also pass in ``y``?and if you don't pass in ``x`` you may not
 pass in ``y`` either.)
 
 In any case, the goal of Argument Clinic is to support argument parsing
@@ -888,7 +888,7 @@
 Remember those format units you skipped for your first
 time because they were advanced?  Here's how to handle those too.
 
-The trick is, all those format units take arguments--either
+The trick is, all those format units take arguments?either
 conversion functions, or types, or strings specifying an encoding.
 (But "legacy converters" don't support arguments.  That's why we
 skipped them for your first function.)  The argument you specified
@@ -1002,7 +1002,7 @@
 By default the impl function Argument Clinic generates for you returns ``PyObject *``.
 But your C function often computes some C type, then converts it into the ``PyObject *``
 at the last moment.  Argument Clinic handles converting your inputs from Python types
-into native C types--why not have it convert your return value from a native C type
+into native C types?why not have it convert your return value from a native C type
 into a Python type too?
 
 That's what a "return converter" does.  It changes your impl function to return
@@ -1184,7 +1184,7 @@
 As we hinted at in the previous section... you can write your own converters!
 A converter is simply a Python class that inherits from ``CConverter``.
 The main purpose of a custom converter is if you have a parameter using
-the ``O&`` format unit--parsing this parameter means calling
+the ``O&`` format unit?parsing this parameter means calling
 a :c:func:`PyArg_ParseTuple` "converter function".
 
 Your converter class should be named ``*something*_converter``.
@@ -1226,7 +1226,7 @@
     The default value used to initialize the C variable when
     there is no default, but not specifying a default may
     result in an "uninitialized variable" warning.  This can
-    easily happen when using option groups--although
+    easily happen when using option groups?although
     properly-written code will never actually use this value,
     the variable does get passed in to the impl, and the
     C compiler will complain about the "use" of the
@@ -1402,7 +1402,7 @@
     all of processing, even from Clinic blocks *after* the
 
   ``suppress``
-    The text is suppressed--thrown away.
+    The text is suppressed?thrown away.
 
 
 Clinic defines five new directives that let you reconfigure its output.
diff --git a/Doc/howto/cporting.rst b/Doc/howto/cporting.rst
--- a/Doc/howto/cporting.rst
+++ b/Doc/howto/cporting.rst
@@ -95,7 +95,7 @@
 --------------------
 
 Python 3 has only one integer type, :func:`int`.  But it actually
-corresponds to Python 2's :func:`long` type--the :func:`int` type
+corresponds to Python 2's :func:`long` type?the :func:`int` type
 used in Python 2 was removed.  In the C-API, ``PyInt_*`` functions
 are replaced by their ``PyLong_*`` equivalents.
 
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -949,7 +949,7 @@
    the list of supported encodings.
 
    *errors* is an optional string that specifies how encoding and decoding
-   errors are to be handled--this cannot be used in binary mode.
+   errors are to be handled?this cannot be used in binary mode.
    A variety of standard error handlers are available
    (listed under :ref:`error-handlers`), though any
    error handling name that has been registered with
diff --git a/Doc/library/hmac.rst b/Doc/library/hmac.rst
--- a/Doc/library/hmac.rst
+++ b/Doc/library/hmac.rst
@@ -111,7 +111,7 @@
 
       If *a* and *b* are of different lengths, or if an error occurs,
       a timing attack could theoretically reveal information about the
-      types and lengths of *a* and *b*--but not their values.
+      types and lengths of *a* and *b*?but not their values.
 
 
    .. versionadded:: 3.3
diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst
--- a/Doc/library/pdb.rst
+++ b/Doc/library/pdb.rst
@@ -328,7 +328,7 @@
    return, jump, quit and their abbreviations) terminates the command list (as if
    that command was immediately followed by end). This is because any time you
    resume execution (even with a simple next or step), you may encounter another
-   breakpoint--which could have its own command list, leading to ambiguities about
+   breakpoint?which could have its own command list, leading to ambiguities about
    which list to execute.
 
    If you use the 'silent' command in the command list, the usual message about
diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst
--- a/Doc/library/shutil.rst
+++ b/Doc/library/shutil.rst
@@ -107,7 +107,7 @@
    If *follow_symlinks* is false, and *src* and *dst* both
    refer to symbolic links, :func:`copystat` will operate on
    the symbolic links themselves rather than the files the
-   symbolic links refer to--reading the information from the
+   symbolic links refer to?reading the information from the
    *src* symbolic link, and writing the information to the
    *dst* symbolic link.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 23:23:49 2016
From: python-checkins at python.org (martin.panter)
Date: Mon, 21 Nov 2016 04:23:49 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_doc_fixups_from_3=2E5?=
Message-ID: <20161121042349.28360.1665.65BB04A6@psf.io>

https://hg.python.org/cpython/rev/997de80779ac
changeset:   105258:997de80779ac
branch:      3.6
parent:      105254:294525aac5eb
parent:      105257:f17afa9327ad
user:        Martin Panter <vadmium+py at gmail.com>
date:        Mon Nov 21 04:22:22 2016 +0000
summary:
  Merge doc fixups from 3.5

files:
  Doc/howto/clinic.rst      |  30 +++++++++++++-------------
  Doc/howto/cporting.rst    |   2 +-
  Doc/library/functions.rst |   2 +-
  Doc/library/hmac.rst      |   2 +-
  Doc/library/pdb.rst       |   2 +-
  Doc/library/shutil.rst    |   2 +-
  6 files changed, 20 insertions(+), 20 deletions(-)


diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst
--- a/Doc/howto/clinic.rst
+++ b/Doc/howto/clinic.rst
@@ -30,7 +30,7 @@
 is to take over responsibility for all argument parsing code
 inside CPython.  This means that, when you convert a function
 to work with Argument Clinic, that function should no longer
-do any of its own argument parsing--the code generated by
+do any of its own argument parsing?the code generated by
 Argument Clinic should be a "black box" to you, where CPython
 calls in at the top, and your code gets called at the bottom,
 with ``PyObject *args`` (and maybe ``PyObject *kwargs``)
@@ -43,12 +43,12 @@
 When you use Argument Clinic, you don't have to repeat yourself.
 
 Obviously, no one would want to use Argument Clinic unless
-it's solving their problem--and without creating new problems of
+it's solving their problem?and without creating new problems of
 its own.
 So it's paramount that Argument Clinic generate correct code.
 It'd be nice if the code was faster, too, but at the very least
 it should not introduce a major speed regression.  (Eventually Argument
-Clinic *should* make a major speedup possible--we could
+Clinic *should* make a major speedup possible?we could
 rewrite its code generator to produce tailor-made argument
 parsing code, rather than calling the general-purpose CPython
 argument parsing library.  That would make for the fastest
@@ -113,7 +113,7 @@
 
 You should never modify the output portion of an Argument Clinic block.  Instead,
 change the input until it produces the output you want.  (That's the purpose of the
-checksum--to detect if someone changed the output, as these edits would be lost
+checksum?to detect if someone changed the output, as these edits would be lost
 the next time Argument Clinic writes out fresh output.)
 
 For the sake of clarity, here's the terminology we'll use with Argument Clinic:
@@ -166,7 +166,7 @@
    or if it has multiple calls to :c:func:`PyArg_ParseTuple`,
    you should choose a different function.  Argument Clinic *does*
    support all of these scenarios.  But these are advanced
-   topics--let's do something simpler for your first function.
+   topics?let's do something simpler for your first function.
 
    Also, if the function has multiple calls to :c:func:`PyArg_ParseTuple`
    or :c:func:`PyArg_ParseTupleAndKeywords` where it supports different
@@ -188,7 +188,7 @@
 
    If the old docstring had a first line that looked like a function
    signature, throw that line away.  (The docstring doesn't need it
-   anymore--when you use ``help()`` on your builtin in the future,
+   anymore?when you use ``help()`` on your builtin in the future,
    the first line will be built automatically based on the function's
    signature.)
 
@@ -209,7 +209,7 @@
 6. Above the docstring, enter the name of the function, followed
    by a blank line.  This should be the Python name of the function,
    and should be the full dotted path
-   to the function--it should start with the name of the module,
+   to the function?it should start with the name of the module,
    include any sub-modules, and if the function is a method on
    a class it should include the class name too.
 
@@ -275,7 +275,7 @@
    What's a "converter"?  It establishes both the type
    of the variable used in C, and the method to convert the Python
    value into a C value at runtime.
-   For now you're going to use what's called a "legacy converter"--a
+   For now you're going to use what's called a "legacy converter"?a
    convenience syntax intended to make porting old code into Argument
    Clinic easier.
 
@@ -424,7 +424,7 @@
     (Argument Clinic always generates its format strings
     with a ``:`` followed by the name of the function.  If the
     existing code's format string ends with ``;``, to provide
-    usage help, this change is harmless--don't worry about it.)
+    usage help, this change is harmless?don't worry about it.)
 
     Third, for parameters whose format units require two arguments
     (like a length variable, or an encoding string, or a pointer
@@ -637,7 +637,7 @@
 Another example is ``curses.window.addch()``, which has a group of two
 arguments that must always be specified together.  (The arguments are
 called ``x`` and ``y``; if you call the function passing in ``x``,
-you must also pass in ``y``--and if you don't pass in ``x`` you may not
+you must also pass in ``y``?and if you don't pass in ``x`` you may not
 pass in ``y`` either.)
 
 In any case, the goal of Argument Clinic is to support argument parsing
@@ -888,7 +888,7 @@
 Remember those format units you skipped for your first
 time because they were advanced?  Here's how to handle those too.
 
-The trick is, all those format units take arguments--either
+The trick is, all those format units take arguments?either
 conversion functions, or types, or strings specifying an encoding.
 (But "legacy converters" don't support arguments.  That's why we
 skipped them for your first function.)  The argument you specified
@@ -1002,7 +1002,7 @@
 By default the impl function Argument Clinic generates for you returns ``PyObject *``.
 But your C function often computes some C type, then converts it into the ``PyObject *``
 at the last moment.  Argument Clinic handles converting your inputs from Python types
-into native C types--why not have it convert your return value from a native C type
+into native C types?why not have it convert your return value from a native C type
 into a Python type too?
 
 That's what a "return converter" does.  It changes your impl function to return
@@ -1184,7 +1184,7 @@
 As we hinted at in the previous section... you can write your own converters!
 A converter is simply a Python class that inherits from ``CConverter``.
 The main purpose of a custom converter is if you have a parameter using
-the ``O&`` format unit--parsing this parameter means calling
+the ``O&`` format unit?parsing this parameter means calling
 a :c:func:`PyArg_ParseTuple` "converter function".
 
 Your converter class should be named ``*something*_converter``.
@@ -1226,7 +1226,7 @@
     The default value used to initialize the C variable when
     there is no default, but not specifying a default may
     result in an "uninitialized variable" warning.  This can
-    easily happen when using option groups--although
+    easily happen when using option groups?although
     properly-written code will never actually use this value,
     the variable does get passed in to the impl, and the
     C compiler will complain about the "use" of the
@@ -1402,7 +1402,7 @@
     all of processing, even from Clinic blocks *after* the
 
   ``suppress``
-    The text is suppressed--thrown away.
+    The text is suppressed?thrown away.
 
 
 Clinic defines five new directives that let you reconfigure its output.
diff --git a/Doc/howto/cporting.rst b/Doc/howto/cporting.rst
--- a/Doc/howto/cporting.rst
+++ b/Doc/howto/cporting.rst
@@ -95,7 +95,7 @@
 --------------------
 
 Python 3 has only one integer type, :func:`int`.  But it actually
-corresponds to Python 2's :func:`long` type--the :func:`int` type
+corresponds to Python 2's :func:`long` type?the :func:`int` type
 used in Python 2 was removed.  In the C-API, ``PyInt_*`` functions
 are replaced by their ``PyLong_*`` equivalents.
 
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -959,7 +959,7 @@
    the list of supported encodings.
 
    *errors* is an optional string that specifies how encoding and decoding
-   errors are to be handled--this cannot be used in binary mode.
+   errors are to be handled?this cannot be used in binary mode.
    A variety of standard error handlers are available
    (listed under :ref:`error-handlers`), though any
    error handling name that has been registered with
diff --git a/Doc/library/hmac.rst b/Doc/library/hmac.rst
--- a/Doc/library/hmac.rst
+++ b/Doc/library/hmac.rst
@@ -111,7 +111,7 @@
 
       If *a* and *b* are of different lengths, or if an error occurs,
       a timing attack could theoretically reveal information about the
-      types and lengths of *a* and *b*--but not their values.
+      types and lengths of *a* and *b*?but not their values.
 
 
    .. versionadded:: 3.3
diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst
--- a/Doc/library/pdb.rst
+++ b/Doc/library/pdb.rst
@@ -334,7 +334,7 @@
    return, jump, quit and their abbreviations) terminates the command list (as if
    that command was immediately followed by end). This is because any time you
    resume execution (even with a simple next or step), you may encounter another
-   breakpoint--which could have its own command list, leading to ambiguities about
+   breakpoint?which could have its own command list, leading to ambiguities about
    which list to execute.
 
    If you use the 'silent' command in the command list, the usual message about
diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst
--- a/Doc/library/shutil.rst
+++ b/Doc/library/shutil.rst
@@ -107,7 +107,7 @@
    If *follow_symlinks* is false, and *src* and *dst* both
    refer to symbolic links, :func:`copystat` will operate on
    the symbolic links themselves rather than the files the
-   symbolic links refer to--reading the information from the
+   symbolic links refer to?reading the information from the
    *src* symbolic link, and writing the information to the
    *dst* symbolic link.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sun Nov 20 23:23:50 2016
From: python-checkins at python.org (martin.panter)
Date: Mon, 21 Nov 2016 04:23:50 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_doc_fixups_from_3=2E6?=
Message-ID: <20161121042349.45413.69490.8B35B810@psf.io>

https://hg.python.org/cpython/rev/45eab45be175
changeset:   105260:45eab45be175
parent:      105255:30f154d9abf0
parent:      105259:b72391f54083
user:        Martin Panter <vadmium+py at gmail.com>
date:        Mon Nov 21 04:22:43 2016 +0000
summary:
  Merge doc fixups from 3.6

files:
  Doc/howto/clinic.rst        |  30 ++++++++++++------------
  Doc/howto/cporting.rst      |   2 +-
  Doc/library/functions.rst   |   2 +-
  Doc/library/hmac.rst        |   2 +-
  Doc/library/pdb.rst         |   2 +-
  Doc/library/shutil.rst      |   2 +-
  Doc/reference/datamodel.rst |   2 +-
  7 files changed, 21 insertions(+), 21 deletions(-)


diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst
--- a/Doc/howto/clinic.rst
+++ b/Doc/howto/clinic.rst
@@ -30,7 +30,7 @@
 is to take over responsibility for all argument parsing code
 inside CPython.  This means that, when you convert a function
 to work with Argument Clinic, that function should no longer
-do any of its own argument parsing--the code generated by
+do any of its own argument parsing?the code generated by
 Argument Clinic should be a "black box" to you, where CPython
 calls in at the top, and your code gets called at the bottom,
 with ``PyObject *args`` (and maybe ``PyObject *kwargs``)
@@ -43,12 +43,12 @@
 When you use Argument Clinic, you don't have to repeat yourself.
 
 Obviously, no one would want to use Argument Clinic unless
-it's solving their problem--and without creating new problems of
+it's solving their problem?and without creating new problems of
 its own.
 So it's paramount that Argument Clinic generate correct code.
 It'd be nice if the code was faster, too, but at the very least
 it should not introduce a major speed regression.  (Eventually Argument
-Clinic *should* make a major speedup possible--we could
+Clinic *should* make a major speedup possible?we could
 rewrite its code generator to produce tailor-made argument
 parsing code, rather than calling the general-purpose CPython
 argument parsing library.  That would make for the fastest
@@ -113,7 +113,7 @@
 
 You should never modify the output portion of an Argument Clinic block.  Instead,
 change the input until it produces the output you want.  (That's the purpose of the
-checksum--to detect if someone changed the output, as these edits would be lost
+checksum?to detect if someone changed the output, as these edits would be lost
 the next time Argument Clinic writes out fresh output.)
 
 For the sake of clarity, here's the terminology we'll use with Argument Clinic:
@@ -166,7 +166,7 @@
    or if it has multiple calls to :c:func:`PyArg_ParseTuple`,
    you should choose a different function.  Argument Clinic *does*
    support all of these scenarios.  But these are advanced
-   topics--let's do something simpler for your first function.
+   topics?let's do something simpler for your first function.
 
    Also, if the function has multiple calls to :c:func:`PyArg_ParseTuple`
    or :c:func:`PyArg_ParseTupleAndKeywords` where it supports different
@@ -188,7 +188,7 @@
 
    If the old docstring had a first line that looked like a function
    signature, throw that line away.  (The docstring doesn't need it
-   anymore--when you use ``help()`` on your builtin in the future,
+   anymore?when you use ``help()`` on your builtin in the future,
    the first line will be built automatically based on the function's
    signature.)
 
@@ -209,7 +209,7 @@
 6. Above the docstring, enter the name of the function, followed
    by a blank line.  This should be the Python name of the function,
    and should be the full dotted path
-   to the function--it should start with the name of the module,
+   to the function?it should start with the name of the module,
    include any sub-modules, and if the function is a method on
    a class it should include the class name too.
 
@@ -275,7 +275,7 @@
    What's a "converter"?  It establishes both the type
    of the variable used in C, and the method to convert the Python
    value into a C value at runtime.
-   For now you're going to use what's called a "legacy converter"--a
+   For now you're going to use what's called a "legacy converter"?a
    convenience syntax intended to make porting old code into Argument
    Clinic easier.
 
@@ -425,7 +425,7 @@
     (Argument Clinic always generates its format strings
     with a ``:`` followed by the name of the function.  If the
     existing code's format string ends with ``;``, to provide
-    usage help, this change is harmless--don't worry about it.)
+    usage help, this change is harmless?don't worry about it.)
 
     Third, for parameters whose format units require two arguments
     (like a length variable, or an encoding string, or a pointer
@@ -638,7 +638,7 @@
 Another example is ``curses.window.addch()``, which has a group of two
 arguments that must always be specified together.  (The arguments are
 called ``x`` and ``y``; if you call the function passing in ``x``,
-you must also pass in ``y``--and if you don't pass in ``x`` you may not
+you must also pass in ``y``?and if you don't pass in ``x`` you may not
 pass in ``y`` either.)
 
 In any case, the goal of Argument Clinic is to support argument parsing
@@ -889,7 +889,7 @@
 Remember those format units you skipped for your first
 time because they were advanced?  Here's how to handle those too.
 
-The trick is, all those format units take arguments--either
+The trick is, all those format units take arguments?either
 conversion functions, or types, or strings specifying an encoding.
 (But "legacy converters" don't support arguments.  That's why we
 skipped them for your first function.)  The argument you specified
@@ -1003,7 +1003,7 @@
 By default the impl function Argument Clinic generates for you returns ``PyObject *``.
 But your C function often computes some C type, then converts it into the ``PyObject *``
 at the last moment.  Argument Clinic handles converting your inputs from Python types
-into native C types--why not have it convert your return value from a native C type
+into native C types?why not have it convert your return value from a native C type
 into a Python type too?
 
 That's what a "return converter" does.  It changes your impl function to return
@@ -1185,7 +1185,7 @@
 As we hinted at in the previous section... you can write your own converters!
 A converter is simply a Python class that inherits from ``CConverter``.
 The main purpose of a custom converter is if you have a parameter using
-the ``O&`` format unit--parsing this parameter means calling
+the ``O&`` format unit?parsing this parameter means calling
 a :c:func:`PyArg_ParseTuple` "converter function".
 
 Your converter class should be named ``*something*_converter``.
@@ -1227,7 +1227,7 @@
     The default value used to initialize the C variable when
     there is no default, but not specifying a default may
     result in an "uninitialized variable" warning.  This can
-    easily happen when using option groups--although
+    easily happen when using option groups?although
     properly-written code will never actually use this value,
     the variable does get passed in to the impl, and the
     C compiler will complain about the "use" of the
@@ -1403,7 +1403,7 @@
     all of processing, even from Clinic blocks *after* the
 
   ``suppress``
-    The text is suppressed--thrown away.
+    The text is suppressed?thrown away.
 
 
 Clinic defines five new directives that let you reconfigure its output.
diff --git a/Doc/howto/cporting.rst b/Doc/howto/cporting.rst
--- a/Doc/howto/cporting.rst
+++ b/Doc/howto/cporting.rst
@@ -95,7 +95,7 @@
 --------------------
 
 Python 3 has only one integer type, :func:`int`.  But it actually
-corresponds to Python 2's :func:`long` type--the :func:`int` type
+corresponds to Python 2's :func:`long` type?the :func:`int` type
 used in Python 2 was removed.  In the C-API, ``PyInt_*`` functions
 are replaced by their ``PyLong_*`` equivalents.
 
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -959,7 +959,7 @@
    the list of supported encodings.
 
    *errors* is an optional string that specifies how encoding and decoding
-   errors are to be handled--this cannot be used in binary mode.
+   errors are to be handled?this cannot be used in binary mode.
    A variety of standard error handlers are available
    (listed under :ref:`error-handlers`), though any
    error handling name that has been registered with
diff --git a/Doc/library/hmac.rst b/Doc/library/hmac.rst
--- a/Doc/library/hmac.rst
+++ b/Doc/library/hmac.rst
@@ -111,7 +111,7 @@
 
       If *a* and *b* are of different lengths, or if an error occurs,
       a timing attack could theoretically reveal information about the
-      types and lengths of *a* and *b*--but not their values.
+      types and lengths of *a* and *b*?but not their values.
 
 
    .. versionadded:: 3.3
diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst
--- a/Doc/library/pdb.rst
+++ b/Doc/library/pdb.rst
@@ -334,7 +334,7 @@
    return, jump, quit and their abbreviations) terminates the command list (as if
    that command was immediately followed by end). This is because any time you
    resume execution (even with a simple next or step), you may encounter another
-   breakpoint--which could have its own command list, leading to ambiguities about
+   breakpoint?which could have its own command list, leading to ambiguities about
    which list to execute.
 
    If you use the 'silent' command in the command list, the usual message about
diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst
--- a/Doc/library/shutil.rst
+++ b/Doc/library/shutil.rst
@@ -107,7 +107,7 @@
    If *follow_symlinks* is false, and *src* and *dst* both
    refer to symbolic links, :func:`copystat` will operate on
    the symbolic links themselves rather than the files the
-   symbolic links refer to--reading the information from the
+   symbolic links refer to?reading the information from the
    *src* symbolic link, and writing the information to the
    *dst* symbolic link.
 
diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst
--- a/Doc/reference/datamodel.rst
+++ b/Doc/reference/datamodel.rst
@@ -2550,7 +2550,7 @@
 .. [#] "Does not support" here means that the class has no such method, or
    the method returns ``NotImplemented``.  Do not set the method to
    ``None`` if you want to force fallback to the right operand's reflected
-   method--that will instead have the opposite effect of explicitly
+   method?that will instead have the opposite effect of explicitly
    *blocking* such fallback.
 
 .. [#] For operands of the same type, it is assumed that if the non-reflected method

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 03:14:09 2016
From: python-checkins at python.org (gregory.p.smith)
Date: Mon, 21 Nov 2016 08:14:09 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_replace_--with-optimizations_references_with_--enable-op?=
 =?utf-8?q?timizations_in_docs=2E?=
Message-ID: <20161121081409.41201.9099.B2BC4D13@psf.io>

https://hg.python.org/cpython/rev/38e9761ccc13
changeset:   105264:38e9761ccc13
parent:      105260:45eab45be175
parent:      105263:ea9cc29a274b
user:        Gregory P. Smith <greg at krypto.org>
date:        Mon Nov 21 00:13:15 2016 -0800
summary:
  replace --with-optimizations references with --enable-optimizations in docs.

files:
  Doc/whatsnew/3.6.rst |  2 +-
  README               |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -1795,7 +1795,7 @@
   yet a supported platform, the Python test suite runs on the Android emulator
   with only about 16 tests failures. See the Android meta-issue :issue:`26865`.
 
-* The ``--with-optimizations`` configure flag has been added. Turning it on
+* The ``--enable-optimizations`` configure flag has been added. Turning it on
   will activate expensive optimizations like PGO.
   (Original patch by Alecsandru Patrascu of Intel in :issue:`26539`.)
 
diff --git a/README b/README
--- a/README
+++ b/README
@@ -46,7 +46,7 @@
 (This will fail if you *also* built at the top-level directory.
 You should do a "make clean" at the toplevel first.)
 
-To get an optimized build of Python, "configure --with-optimizations" before
+To get an optimized build of Python, "configure --enable-optimizations" before
 you run make.  This sets the default make targets up to enable Profile Guided
 Optimization (PGO) and may be used to auto-enable Link Time Optimization (LTO)
 on some platforms.  For more details, see the sections bellow.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 03:14:09 2016
From: python-checkins at python.org (gregory.p.smith)
Date: Mon, 21 Nov 2016 08:14:09 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_replace_--with?=
 =?utf-8?q?-optimizations_references_with_--enable-optimizations_in_docs?=
 =?utf-8?q?=2E?=
Message-ID: <20161121081409.45206.39597.FC7B388D@psf.io>

https://hg.python.org/cpython/rev/6ae0e6d435de
changeset:   105262:6ae0e6d435de
branch:      3.5
parent:      105257:f17afa9327ad
user:        Gregory P. Smith <greg at krypto.org>
date:        Mon Nov 21 00:11:47 2016 -0800
summary:
  replace --with-optimizations references with --enable-optimizations in docs.

files:
  README |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/README b/README
--- a/README
+++ b/README
@@ -46,7 +46,7 @@
 (This will fail if you *also* built at the top-level directory.
 You should do a "make clean" at the toplevel first.)
 
-To get an optimized build of Python, "configure --with-optimizations" before
+To get an optimized build of Python, "configure --enable-optimizations" before
 you run make.  This sets the default make targets up to enable Profile Guided
 Optimization (PGO) and may be used to auto-enable Link Time Optimization (LTO)
 on some platforms.  For more details, see the sections bellow.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 03:14:10 2016
From: python-checkins at python.org (gregory.p.smith)
Date: Mon, 21 Nov 2016 08:14:10 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_replace_--with?=
 =?utf-8?q?-optimizations_references_with_--enable-optimizations_in_docs?=
 =?utf-8?q?=2E?=
Message-ID: <20161121081409.5574.2782.FAE4EFCA@psf.io>

https://hg.python.org/cpython/rev/a5e2add2c37b
changeset:   105261:a5e2add2c37b
branch:      2.7
parent:      105256:7da35bced88e
user:        Gregory P. Smith <greg at krypto.org>
date:        Mon Nov 21 00:13:44 2016 -0800
summary:
  replace --with-optimizations references with --enable-optimizations in docs.

files:
  README |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/README b/README
--- a/README
+++ b/README
@@ -173,7 +173,7 @@
 build your desired target.  The interpreter executable is built in the
 top level directory.
 
-To get an optimized build of Python, "configure --with-optimizations" before
+To get an optimized build of Python, "configure --enable-optimizations" before
 you run make.  This sets the default make targets up to enable Profile Guided
 Optimization (PGO) and may be used to auto-enable Link Time Optimization (LTO)
 on some platforms.  For more details, see the sections bellow.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 03:14:10 2016
From: python-checkins at python.org (gregory.p.smith)
Date: Mon, 21 Nov 2016 08:14:10 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_replace_--with-optimizations_references_with_--enable-optimiza?=
 =?utf-8?q?tions_in_docs=2E?=
Message-ID: <20161121081409.5536.60498.EA9BB3B7@psf.io>

https://hg.python.org/cpython/rev/ea9cc29a274b
changeset:   105263:ea9cc29a274b
branch:      3.6
parent:      105259:b72391f54083
parent:      105262:6ae0e6d435de
user:        Gregory P. Smith <greg at krypto.org>
date:        Mon Nov 21 00:12:40 2016 -0800
summary:
  replace --with-optimizations references with --enable-optimizations in docs.

files:
  Doc/whatsnew/3.6.rst |  2 +-
  README               |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -1795,7 +1795,7 @@
   yet a supported platform, the Python test suite runs on the Android emulator
   with only about 16 tests failures. See the Android meta-issue :issue:`26865`.
 
-* The ``--with-optimizations`` configure flag has been added. Turning it on
+* The ``--enable-optimizations`` configure flag has been added. Turning it on
   will activate expensive optimizations like PGO.
   (Original patch by Alecsandru Patrascu of Intel in :issue:`26539`.)
 
diff --git a/README b/README
--- a/README
+++ b/README
@@ -46,7 +46,7 @@
 (This will fail if you *also* built at the top-level directory.
 You should do a "make clean" at the toplevel first.)
 
-To get an optimized build of Python, "configure --with-optimizations" before
+To get an optimized build of Python, "configure --enable-optimizations" before
 you run make.  This sets the default make targets up to enable Profile Guided
 Optimization (PGO) and may be used to auto-enable Link Time Optimization (LTO)
 on some platforms.  For more details, see the sections bellow.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 03:26:09 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 21 Nov 2016 08:26:09 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328748=3A_Private_?=
 =?utf-8?q?variable_=5FPy=5FPackageContext_is_now_of_type_=22const_char_*?=
 =?utf-8?q?=22?=
Message-ID: <20161121082609.41115.60489.8407EFAE@psf.io>

https://hg.python.org/cpython/rev/d2dd6578aa16
changeset:   105265:d2dd6578aa16
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 21 10:25:54 2016 +0200
summary:
  Issue #28748: Private variable _Py_PackageContext is now of type "const char *"
rather of "char *".

files:
  Include/modsupport.h   |  2 +-
  Misc/NEWS              |  3 +++
  Objects/moduleobject.c |  2 +-
  Python/importdl.c      |  2 +-
  Python/modsupport.c    |  2 +-
  5 files changed, 7 insertions(+), 4 deletions(-)


diff --git a/Include/modsupport.h b/Include/modsupport.h
--- a/Include/modsupport.h
+++ b/Include/modsupport.h
@@ -176,7 +176,7 @@
 #endif /* New in 3.5 */
 
 #ifndef Py_LIMITED_API
-PyAPI_DATA(char *) _Py_PackageContext;
+PyAPI_DATA(const char *) _Py_PackageContext;
 #endif
 
 #ifdef __cplusplus
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -416,6 +416,9 @@
 C API
 -----
 
+- Issue #28748: Private variable _Py_PackageContext is now of type "const char *"
+  rather of "char *".
+
 - Issue #19569: Compiler warnings are now emitted if use most of deprecated
   functions.
 
diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c
--- a/Objects/moduleobject.c
+++ b/Objects/moduleobject.c
@@ -188,7 +188,7 @@
        (if the name actually matches).
     */
     if (_Py_PackageContext != NULL) {
-        char *p = strrchr(_Py_PackageContext, '.');
+        const char *p = strrchr(_Py_PackageContext, '.');
         if (p != NULL && strcmp(module->m_name, p+1) == 0) {
             name = _Py_PackageContext;
             _Py_PackageContext = NULL;
diff --git a/Python/importdl.c b/Python/importdl.c
--- a/Python/importdl.c
+++ b/Python/importdl.c
@@ -94,7 +94,7 @@
 #endif
     PyObject *name_unicode = NULL, *name = NULL, *path = NULL, *m = NULL;
     const char *name_buf, *hook_prefix;
-    char *oldcontext;
+    const char *oldcontext;
     dl_funcptr exportfunc;
     PyModuleDef *def;
     PyObject *(*p0)(void);
diff --git a/Python/modsupport.c b/Python/modsupport.c
--- a/Python/modsupport.c
+++ b/Python/modsupport.c
@@ -9,7 +9,7 @@
 static PyObject *va_build_value(const char *, va_list, int);
 
 /* Package context -- the full module name for package imports */
-char *_Py_PackageContext = NULL;
+const char *_Py_PackageContext = NULL;
 
 /* Helper for mkvalue() to scan the length of a format */
 

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Mon Nov 21 04:05:04 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Mon, 21 Nov 2016 09:05:04 +0000
Subject: [Python-checkins] Daily reference leaks (45eab45be175): sum=2
Message-ID: <20161121090503.45413.20433.952FE2D2@psf.io>

results for 45eab45be175 on branch "default"
--------------------------------------------

test_collections leaked [7, 0, -7] memory blocks, sum=0
test_functools leaked [0, 3, 1] memory blocks, sum=4
test_multiprocessing_spawn leaked [-1, 1, -2] memory blocks, sum=-2


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogYaSdvs', '--timeout', '7200']

From python-checkins at python.org  Mon Nov 21 04:38:52 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 21 Nov 2016 09:38:52 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Fixed_the_documentation_of_the_structure_PyMethodDef=2E?=
Message-ID: <20161121093851.10168.81892.8CE279A7@psf.io>

https://hg.python.org/cpython/rev/00870a1d0c25
changeset:   105268:00870a1d0c25
branch:      3.6
parent:      105263:ea9cc29a274b
parent:      105266:1cf7ecc37467
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 21 11:38:14 2016 +0200
summary:
  Fixed the documentation of the structure PyMethodDef.
The fields ml_name and ml_doc are of type "const char *" rather of "char *".

files:
  Doc/extending/newtypes.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst
--- a/Doc/extending/newtypes.rst
+++ b/Doc/extending/newtypes.rst
@@ -1122,10 +1122,10 @@
 structure::
 
    typedef struct PyMethodDef {
-       char        *ml_name;       /* method name */
+       const char  *ml_name;       /* method name */
        PyCFunction  ml_meth;       /* implementation function */
        int          ml_flags;      /* flags */
-       char        *ml_doc;        /* docstring */
+       const char  *ml_doc;        /* docstring */
    } PyMethodDef;
 
 One entry should be defined for each method provided by the type; no entries are

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 04:38:52 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 21 Nov 2016 09:38:52 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fixed_the_docu?=
 =?utf-8?q?mentation_of_the_structure_PyMethodDef=2E?=
Message-ID: <20161121093851.28125.57956.69F6652C@psf.io>

https://hg.python.org/cpython/rev/6eefdf0fade6
changeset:   105267:6eefdf0fade6
branch:      2.7
parent:      105261:a5e2add2c37b
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 21 11:37:18 2016 +0200
summary:
  Fixed the documentation of the structure PyMethodDef.
The fields ml_name and ml_doc are of type "const char *" rather of "char *".

files:
  Doc/extending/newtypes.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst
--- a/Doc/extending/newtypes.rst
+++ b/Doc/extending/newtypes.rst
@@ -1170,10 +1170,10 @@
 structure::
 
    typedef struct PyMethodDef {
-       char        *ml_name;       /* method name */
+       const char  *ml_name;       /* method name */
        PyCFunction  ml_meth;       /* implementation function */
        int          ml_flags;      /* flags */
-       char        *ml_doc;        /* docstring */
+       const char  *ml_doc;        /* docstring */
    } PyMethodDef;
 
 One entry should be defined for each method provided by the type; no entries are

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 04:38:51 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 21 Nov 2016 09:38:51 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Fixed_the_documentation_of_the_structure_PyMethodDef=2E?=
Message-ID: <20161121093851.7898.57241.A804074C@psf.io>

https://hg.python.org/cpython/rev/94d9e97f3ee1
changeset:   105269:94d9e97f3ee1
parent:      105265:d2dd6578aa16
parent:      105268:00870a1d0c25
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 21 11:38:37 2016 +0200
summary:
  Fixed the documentation of the structure PyMethodDef.
The fields ml_name and ml_doc are of type "const char *" rather of "char *".

files:
  Doc/extending/newtypes.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst
--- a/Doc/extending/newtypes.rst
+++ b/Doc/extending/newtypes.rst
@@ -1122,10 +1122,10 @@
 structure::
 
    typedef struct PyMethodDef {
-       char        *ml_name;       /* method name */
+       const char  *ml_name;       /* method name */
        PyCFunction  ml_meth;       /* implementation function */
        int          ml_flags;      /* flags */
-       char        *ml_doc;        /* docstring */
+       const char  *ml_doc;        /* docstring */
    } PyMethodDef;
 
 One entry should be defined for each method provided by the type; no entries are

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 04:38:52 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 21 Nov 2016 09:38:52 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fixed_the_docu?=
 =?utf-8?q?mentation_of_the_structure_PyMethodDef=2E?=
Message-ID: <20161121093851.5220.76803.E969C7A8@psf.io>

https://hg.python.org/cpython/rev/1cf7ecc37467
changeset:   105266:1cf7ecc37467
branch:      3.5
parent:      105262:6ae0e6d435de
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 21 11:37:18 2016 +0200
summary:
  Fixed the documentation of the structure PyMethodDef.
The fields ml_name and ml_doc are of type "const char *" rather of "char *".

files:
  Doc/extending/newtypes.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst
--- a/Doc/extending/newtypes.rst
+++ b/Doc/extending/newtypes.rst
@@ -1122,10 +1122,10 @@
 structure::
 
    typedef struct PyMethodDef {
-       char        *ml_name;       /* method name */
+       const char  *ml_name;       /* method name */
        PyCFunction  ml_meth;       /* implementation function */
        int          ml_flags;      /* flags */
-       char        *ml_doc;        /* docstring */
+       const char  *ml_doc;        /* docstring */
    } PyMethodDef;
 
 One entry should be defined for each method provided by the type; no entries are

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 04:47:29 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 21 Nov 2016 09:47:29 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NzYw?=
 =?utf-8?q?=3A_Clean_up_and_fix_comments_in_PyUnicode=5FAsUnicodeEscapeStr?=
 =?utf-8?b?aW5nKCku?=
Message-ID: <20161121094729.28084.95975.A76D4657@psf.io>

https://hg.python.org/cpython/rev/2d0ce3f4dfbd
changeset:   105270:2d0ce3f4dfbd
branch:      3.6
parent:      105268:00870a1d0c25
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 21 11:46:51 2016 +0200
summary:
  Issue #28760: Clean up and fix comments in PyUnicode_AsUnicodeEscapeString().
Patch by Xiang Zhang.

files:
  Objects/unicodeobject.c |  14 ++++----------
  1 files changed, 4 insertions(+), 10 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -6134,12 +6134,7 @@
     return result;
 }
 
-/* Return a Unicode-Escape string version of the Unicode object.
-
-   If quotes is true, the string is enclosed in u"" or u'' quotes as
-   appropriate.
-
-*/
+/* Return a Unicode-Escape string version of the Unicode object. */
 
 PyObject *
 PyUnicode_AsUnicodeEscapeString(PyObject *unicode)
@@ -6177,10 +6172,10 @@
     /* 4 byte characters can take up 10 bytes, 2 byte characters can take up 6
        bytes, and 1 byte characters 4. */
     expandsize = kind * 2 + 2;
-    if (len > (PY_SSIZE_T_MAX - 2 - 1) / expandsize) {
+    if (len > PY_SSIZE_T_MAX / expandsize) {
         return PyErr_NoMemory();
     }
-    repr = PyBytes_FromStringAndSize(NULL, 2 + expandsize * len + 1);
+    repr = PyBytes_FromStringAndSize(NULL, expandsize * len);
     if (repr == NULL) {
         return NULL;
     }
@@ -6225,9 +6220,8 @@
                 *p++ = Py_hexdigits[ch & 0x000F];
             }
         }
-        /* U+0000-U+00ff range: Map 16-bit characters to '\uHHHH' */
+        /* U+0100-U+ffff range: Map 16-bit characters to '\uHHHH' */
         else if (ch < 0x10000) {
-            /* U+0100-U+ffff */
             *p++ = '\\';
             *p++ = 'u';
             *p++ = Py_hexdigits[(ch >> 12) & 0x000F];

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 04:47:29 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 21 Nov 2016 09:47:29 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328760=3A_Clean_up_and_fix_comments_in_PyUnicode?=
 =?utf-8?q?=5FAsUnicodeEscapeString=28=29=2E?=
Message-ID: <20161121094729.7198.93920.94BBE313@psf.io>

https://hg.python.org/cpython/rev/d656b93c5603
changeset:   105271:d656b93c5603
parent:      105269:94d9e97f3ee1
parent:      105270:2d0ce3f4dfbd
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 21 11:47:16 2016 +0200
summary:
  Issue #28760: Clean up and fix comments in PyUnicode_AsUnicodeEscapeString().
Patch by Xiang Zhang.

files:
  Objects/unicodeobject.c |  14 ++++----------
  1 files changed, 4 insertions(+), 10 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -6118,12 +6118,7 @@
     return result;
 }
 
-/* Return a Unicode-Escape string version of the Unicode object.
-
-   If quotes is true, the string is enclosed in u"" or u'' quotes as
-   appropriate.
-
-*/
+/* Return a Unicode-Escape string version of the Unicode object. */
 
 PyObject *
 PyUnicode_AsUnicodeEscapeString(PyObject *unicode)
@@ -6161,10 +6156,10 @@
     /* 4 byte characters can take up 10 bytes, 2 byte characters can take up 6
        bytes, and 1 byte characters 4. */
     expandsize = kind * 2 + 2;
-    if (len > (PY_SSIZE_T_MAX - 2 - 1) / expandsize) {
+    if (len > PY_SSIZE_T_MAX / expandsize) {
         return PyErr_NoMemory();
     }
-    repr = PyBytes_FromStringAndSize(NULL, 2 + expandsize * len + 1);
+    repr = PyBytes_FromStringAndSize(NULL, expandsize * len);
     if (repr == NULL) {
         return NULL;
     }
@@ -6209,9 +6204,8 @@
                 *p++ = Py_hexdigits[ch & 0x000F];
             }
         }
-        /* U+0000-U+00ff range: Map 16-bit characters to '\uHHHH' */
+        /* U+0100-U+ffff range: Map 16-bit characters to '\uHHHH' */
         else if (ch < 0x10000) {
-            /* U+0100-U+ffff */
             *p++ = '\\';
             *p++ = 'u';
             *p++ = Py_hexdigits[(ch >> 12) & 0x000F];

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 05:00:12 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 10:00:12 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Extend_and_imp?=
 =?utf-8?q?rove_the_examples_for_the_random_module?=
Message-ID: <20161121100011.40975.49486.3F79997F@psf.io>

https://hg.python.org/cpython/rev/223967b49e49
changeset:   105272:223967b49e49
branch:      3.6
parent:      105270:2d0ce3f4dfbd
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 01:59:39 2016 -0800
summary:
  Extend and improve the examples for the random module

files:
  Doc/library/random.rst |  34 +++++++++++++++++++++++++----
  1 files changed, 29 insertions(+), 5 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -345,8 +345,8 @@
    >>> randrange(0, 101, 2)                 # Even integer from 0 to 100 inclusive
    26
 
-   >>> choice('abcdefghij')                 # Single random element from a sequence
-   'c'
+   >>> choice(['win', 'lose', 'draw'])      # Single random element from a sequence
+   'draw'
 
    >>> deck = 'ace two three four'.split()
    >>> shuffle(deck)                        # Shuffle a list
@@ -370,8 +370,9 @@
    >>> print(seen.count('tens') / 20)
    0.15
 
-   # Estimate the probability of getting 5 or more heads from 7 spins
-   # of a biased coin that settles on heads 60% of the time.
+   # Estimate the probability of getting 5 or more heads
+   # from 7 spins of a biased coin that settles on heads
+   # 60% of the time.
    >>> n = 10000
    >>> cw = [0.60, 1.00]
    >>> sum(choices('HT', cum_weights=cw, k=7).count('H') >= 5 for i in range(n)) / n
@@ -416,4 +417,27 @@
     print(f'{n} label reshufflings produced only {count} instances with a difference')
     print(f'at least as extreme as the observed difference of {observed_diff:.1f}.')
     print(f'The one-sided p-value of {count / n:.4f} leads us to reject the null')
-    print(f'hypothesis that the observed difference occurred due to chance.')
+    print(f'hypothesis that there is no difference between the drug and the placebo.')
+
+Simulation of arrival times and service deliveries in a single server queue::
+
+    from random import gauss, expovariate
+
+    average_arrival_interval = 5.6
+    average_service_time = 5.0
+    stdev_service_time = 0.5
+
+    num_waiting = 0
+    arrival = service_end = 0.0
+    for i in range(10000):
+        num_waiting += 1
+        arrival += expovariate(1.0 / average_arrival_interval)
+        print(f'{arrival:6.1f} arrived')
+
+        while arrival > service_end:
+            num_waiting -= 1
+            service_start = service_end if num_waiting else arrival
+            service_time = gauss(average_service_time, stdev_service_time)
+            service_end = service_start + service_time
+            print(f'\t\t{service_start:.1f} to {service_end:.1f} serviced')
+

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 05:00:13 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 10:00:13 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161121100012.14252.75237.D9B8354B@psf.io>

https://hg.python.org/cpython/rev/2b91121bc3fc
changeset:   105273:2b91121bc3fc
parent:      105271:d656b93c5603
parent:      105272:223967b49e49
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 02:00:04 2016 -0800
summary:
  merge

files:
  Doc/library/random.rst |  34 +++++++++++++++++++++++++----
  1 files changed, 29 insertions(+), 5 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -345,8 +345,8 @@
    >>> randrange(0, 101, 2)                 # Even integer from 0 to 100 inclusive
    26
 
-   >>> choice('abcdefghij')                 # Single random element from a sequence
-   'c'
+   >>> choice(['win', 'lose', 'draw'])      # Single random element from a sequence
+   'draw'
 
    >>> deck = 'ace two three four'.split()
    >>> shuffle(deck)                        # Shuffle a list
@@ -370,8 +370,9 @@
    >>> print(seen.count('tens') / 20)
    0.15
 
-   # Estimate the probability of getting 5 or more heads from 7 spins
-   # of a biased coin that settles on heads 60% of the time.
+   # Estimate the probability of getting 5 or more heads
+   # from 7 spins of a biased coin that settles on heads
+   # 60% of the time.
    >>> n = 10000
    >>> cw = [0.60, 1.00]
    >>> sum(choices('HT', cum_weights=cw, k=7).count('H') >= 5 for i in range(n)) / n
@@ -416,4 +417,27 @@
     print(f'{n} label reshufflings produced only {count} instances with a difference')
     print(f'at least as extreme as the observed difference of {observed_diff:.1f}.')
     print(f'The one-sided p-value of {count / n:.4f} leads us to reject the null')
-    print(f'hypothesis that the observed difference occurred due to chance.')
+    print(f'hypothesis that there is no difference between the drug and the placebo.')
+
+Simulation of arrival times and service deliveries in a single server queue::
+
+    from random import gauss, expovariate
+
+    average_arrival_interval = 5.6
+    average_service_time = 5.0
+    stdev_service_time = 0.5
+
+    num_waiting = 0
+    arrival = service_end = 0.0
+    for i in range(10000):
+        num_waiting += 1
+        arrival += expovariate(1.0 / average_arrival_interval)
+        print(f'{arrival:6.1f} arrived')
+
+        while arrival > service_end:
+            num_waiting -= 1
+            service_start = service_end if num_waiting else arrival
+            service_time = gauss(average_service_time, stdev_service_time)
+            service_end = service_start + service_time
+            print(f'\t\t{service_start:.1f} to {service_end:.1f} serviced')
+

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 06:41:01 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 21 Nov 2016 11:41:01 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Change_an_en-dash_to_an_em?=
 =?utf-8?q?-dash=2E?=
Message-ID: <20161121114055.14307.4719.5587FF11@psf.io>

https://hg.python.org/cpython/rev/81bdcda9e8ee
changeset:   105274:81bdcda9e8ee
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 21 13:40:45 2016 +0200
summary:
  Change an en-dash to an em-dash.

files:
  Doc/howto/clinic.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst
--- a/Doc/howto/clinic.rst
+++ b/Doc/howto/clinic.rst
@@ -376,7 +376,7 @@
         [clinic start generated code]*/
 
 12. Save and close the file, then run ``Tools/clinic/clinic.py`` on
-    it.  With luck everything worked--your block now has output, and
+    it.  With luck everything worked---your block now has output, and
     a ``.c.h`` file has been generated! Reopen the file in your
     text editor to see:
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 06:58:18 2016
From: python-checkins at python.org (inada.naoki)
Date: Mon, 21 Nov 2016 11:58:18 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NTMy?=
 =?utf-8?q?=3A_Show_sys=2Eversion_when_-V_option_is_supplied_twice?=
Message-ID: <20161121115818.41510.351.6090A196@psf.io>

https://hg.python.org/cpython/rev/fc6bb69cec05
changeset:   105275:fc6bb69cec05
branch:      3.6
parent:      105272:223967b49e49
user:        INADA Naoki <songofacandy at gmail.com>
date:        Mon Nov 21 20:57:14 2016 +0900
summary:
  Issue #28532: Show sys.version when -V option is supplied twice

files:
  Doc/using/cmdline.rst     |  7 ++++++-
  Lib/test/test_cmd_line.py |  2 +-
  Misc/NEWS                 |  2 ++
  Misc/python.man           |  3 ++-
  Modules/main.c            |  3 ++-
  5 files changed, 13 insertions(+), 4 deletions(-)


diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst
--- a/Doc/using/cmdline.rst
+++ b/Doc/using/cmdline.rst
@@ -180,7 +180,12 @@
 
    Print the Python version number and exit.  Example output could be::
 
-       Python 3.0
+       Python 3.6.0b2+
+
+   When given twice, print more information about the build, like::
+
+       Python 3.6.0b2+ (3.6:84a3c5003510+, Oct 26 2016, 02:33:55)
+       [GCC 6.2.0 20161005]
 
 
 .. _using-on-misc-options:
diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py
--- a/Lib/test/test_cmd_line.py
+++ b/Lib/test/test_cmd_line.py
@@ -43,7 +43,7 @@
 
     def test_version(self):
         version = ('Python %d.%d' % sys.version_info[:2]).encode("ascii")
-        for switch in '-V', '--version':
+        for switch in '-V', '--version', '-VV':
             rc, out, err = assert_python_ok(switch)
             self.assertFalse(err.startswith(version))
             self.assertTrue(out.startswith(version))
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,8 @@
 Core and Builtins
 -----------------
 
+- Issue #28532: Show sys.version when -V option is supplied twice.
+
 - Issue #28746: Fix the set_inheritable() file descriptor method on platforms
   that do not have the ioctl FIOCLEX and FIONCLEX commands.
 
diff --git a/Misc/python.man b/Misc/python.man
--- a/Misc/python.man
+++ b/Misc/python.man
@@ -194,7 +194,8 @@
 at exit.
 .TP
 .B \-V ", " \-\-version
-Prints the Python version number of the executable and exits.
+Prints the Python version number of the executable and exits.  When given
+twice, print more information about the build.
 .TP
 .BI "\-W " argument
 Warning control.  Python sometimes prints warning message to
diff --git a/Modules/main.c b/Modules/main.c
--- a/Modules/main.c
+++ b/Modules/main.c
@@ -74,6 +74,7 @@
 -v     : verbose (trace import statements); also PYTHONVERBOSE=x\n\
          can be supplied multiple times to increase verbosity\n\
 -V     : print the Python version number and exit (also --version)\n\
+         when given twice, print more information about the build\n\
 -W arg : warning control; arg is action:message:category:module:lineno\n\
          also PYTHONWARNINGS=arg\n\
 -x     : skip first line of source, allowing use of non-Unix forms of #!cmd\n\
@@ -512,7 +513,7 @@
         return usage(0, argv[0]);
 
     if (version) {
-        printf("Python %s\n", PY_VERSION);
+        printf("Python %s\n", version >= 2 ? Py_GetVersion() : PY_VERSION);
         return 0;
     }
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 06:58:18 2016
From: python-checkins at python.org (inada.naoki)
Date: Mon, 21 Nov 2016 11:58:18 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328532=3A_Show_sys=2Eversion_when_-V_option_is_s?=
 =?utf-8?q?upplied_twice?=
Message-ID: <20161121115818.8221.71013.7BA88369@psf.io>

https://hg.python.org/cpython/rev/02aa667bd02d
changeset:   105276:02aa667bd02d
parent:      105274:81bdcda9e8ee
parent:      105275:fc6bb69cec05
user:        INADA Naoki <songofacandy at gmail.com>
date:        Mon Nov 21 20:58:10 2016 +0900
summary:
  Issue #28532: Show sys.version when -V option is supplied twice

files:
  Doc/using/cmdline.rst     |  7 ++++++-
  Lib/test/test_cmd_line.py |  2 +-
  Misc/NEWS                 |  2 ++
  Misc/python.man           |  3 ++-
  Modules/main.c            |  3 ++-
  5 files changed, 13 insertions(+), 4 deletions(-)


diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst
--- a/Doc/using/cmdline.rst
+++ b/Doc/using/cmdline.rst
@@ -180,7 +180,12 @@
 
    Print the Python version number and exit.  Example output could be::
 
-       Python 3.0
+       Python 3.6.0b2+
+
+   When given twice, print more information about the build, like::
+
+       Python 3.6.0b2+ (3.6:84a3c5003510+, Oct 26 2016, 02:33:55)
+       [GCC 6.2.0 20161005]
 
 
 .. _using-on-misc-options:
diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py
--- a/Lib/test/test_cmd_line.py
+++ b/Lib/test/test_cmd_line.py
@@ -43,7 +43,7 @@
 
     def test_version(self):
         version = ('Python %d.%d' % sys.version_info[:2]).encode("ascii")
-        for switch in '-V', '--version':
+        for switch in '-V', '--version', '-VV':
             rc, out, err = assert_python_ok(switch)
             self.assertFalse(err.startswith(version))
             self.assertTrue(out.startswith(version))
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,8 @@
 Core and Builtins
 -----------------
 
+- Issue #28532: Show sys.version when -V option is supplied twice.
+
 - Issue #28746: Fix the set_inheritable() file descriptor method on platforms
   that do not have the ioctl FIOCLEX and FIONCLEX commands.
 
diff --git a/Misc/python.man b/Misc/python.man
--- a/Misc/python.man
+++ b/Misc/python.man
@@ -194,7 +194,8 @@
 at exit.
 .TP
 .B \-V ", " \-\-version
-Prints the Python version number of the executable and exits.
+Prints the Python version number of the executable and exits.  When given
+twice, print more information about the build.
 .TP
 .BI "\-W " argument
 Warning control.  Python sometimes prints warning message to
diff --git a/Modules/main.c b/Modules/main.c
--- a/Modules/main.c
+++ b/Modules/main.c
@@ -74,6 +74,7 @@
 -v     : verbose (trace import statements); also PYTHONVERBOSE=x\n\
          can be supplied multiple times to increase verbosity\n\
 -V     : print the Python version number and exit (also --version)\n\
+         when given twice, print more information about the build\n\
 -W arg : warning control; arg is action:message:category:module:lineno\n\
          also PYTHONWARNINGS=arg\n\
 -x     : skip first line of source, allowing use of non-Unix forms of #!cmd\n\
@@ -512,7 +513,7 @@
         return usage(0, argv[0]);
 
     if (version) {
-        printf("Python %s\n", PY_VERSION);
+        printf("Python %s\n", version >= 2 ? Py_GetVersion() : PY_VERSION);
         return 0;
     }
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 08:42:07 2016
From: python-checkins at python.org (martin.panter)
Date: Mon, 21 Nov 2016 13:42:07 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_indentatio?=
 =?utf-8?q?n?=
Message-ID: <20161121134206.14134.66851.579A3F97@psf.io>

https://hg.python.org/cpython/rev/4d2888039631
changeset:   105277:4d2888039631
branch:      3.5
parent:      105266:1cf7ecc37467
user:        Martin Panter <vadmium+py at gmail.com>
date:        Mon Nov 21 13:36:36 2016 +0000
summary:
  Fix indentation

files:
  Doc/library/pickletools.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/pickletools.rst b/Doc/library/pickletools.rst
--- a/Doc/library/pickletools.rst
+++ b/Doc/library/pickletools.rst
@@ -92,8 +92,8 @@
    a short description.  The value of *annotate* is used as a hint for
    the column where annotation should start.
 
-  .. versionadded:: 3.2
-     The *annotate* argument.
+   .. versionadded:: 3.2
+      The *annotate* argument.
 
 .. function:: genops(pickle)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 08:42:07 2016
From: python-checkins at python.org (martin.panter)
Date: Mon, 21 Nov 2016 13:42:07 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_doc_fixup_from_3=2E5?=
Message-ID: <20161121134207.45508.99783.604E1F44@psf.io>

https://hg.python.org/cpython/rev/228e5a50051a
changeset:   105278:228e5a50051a
branch:      3.6
parent:      105275:fc6bb69cec05
parent:      105277:4d2888039631
user:        Martin Panter <vadmium+py at gmail.com>
date:        Mon Nov 21 13:40:40 2016 +0000
summary:
  Merge doc fixup from 3.5

files:
  Doc/library/pickletools.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/pickletools.rst b/Doc/library/pickletools.rst
--- a/Doc/library/pickletools.rst
+++ b/Doc/library/pickletools.rst
@@ -92,8 +92,8 @@
    a short description.  The value of *annotate* is used as a hint for
    the column where annotation should start.
 
-  .. versionadded:: 3.2
-     The *annotate* argument.
+   .. versionadded:: 3.2
+      The *annotate* argument.
 
 .. function:: genops(pickle)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 08:42:08 2016
From: python-checkins at python.org (martin.panter)
Date: Mon, 21 Nov 2016 13:42:08 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_doc_fixes_from_3=2E6?=
Message-ID: <20161121134207.40895.94239.C5768DB7@psf.io>

https://hg.python.org/cpython/rev/acde821520fc
changeset:   105280:acde821520fc
parent:      105276:02aa667bd02d
parent:      105279:a429d29eafbf
user:        Martin Panter <vadmium+py at gmail.com>
date:        Mon Nov 21 13:41:07 2016 +0000
summary:
  Merge doc fixes from 3.6

files:
  Doc/library/pickletools.rst |    4 +-
  Doc/whatsnew/3.6.rst        |  102 ++++++++++++------------
  2 files changed, 53 insertions(+), 53 deletions(-)


diff --git a/Doc/library/pickletools.rst b/Doc/library/pickletools.rst
--- a/Doc/library/pickletools.rst
+++ b/Doc/library/pickletools.rst
@@ -92,8 +92,8 @@
    a short description.  The value of *annotate* is used as a hint for
    the column where annotation should start.
 
-  .. versionadded:: 3.2
-     The *annotate* argument.
+   .. versionadded:: 3.2
+      The *annotate* argument.
 
 .. function:: genops(pickle)
 
diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -425,16 +425,16 @@
 object.
 
 The built-in :func:`open` function has been updated to accept
-:class:`os.PathLike` objects as have all relevant functions in the
-:mod:`os` and :mod:`os.path` modules, as well as most functions and
+:class:`os.PathLike` objects, as have all relevant functions in the
+:mod:`os` and :mod:`os.path` modules, and most other functions and
 classes in the standard library.  The :class:`os.DirEntry` class
 and relevant classes in :mod:`pathlib` have also been updated to
 implement :class:`os.PathLike`.
 
-The hope in is that updating the fundamental functions for operating
+The hope is that updating the fundamental functions for operating
 on file system paths will lead to third-party code to implicitly
 support all :term:`path-like objects <path-like object>` without any
-code changes or at least very minimal ones (e.g. calling
+code changes, or at least very minimal ones (e.g. calling
 :func:`os.fspath` at the beginning of code before operating on a
 path-like object).
 
@@ -635,18 +635,18 @@
 ---------------------------------
 
 The new :envvar:`PYTHONMALLOC` environment variable allows setting the Python
-memory allocators and/or install debug hooks.
+memory allocators and installing debug hooks.
 
 It is now possible to install debug hooks on Python memory allocators on Python
 compiled in release mode using ``PYTHONMALLOC=debug``. Effects of debug hooks:
 
 * Newly allocated memory is filled with the byte ``0xCB``
 * Freed memory is filled with the byte ``0xDB``
-* Detect violations of Python memory allocator API. For example,
+* Detect violations of the Python memory allocator API. For example,
   :c:func:`PyObject_Free` called on a memory block allocated by
   :c:func:`PyMem_Malloc`.
-* Detect write before the start of the buffer (buffer underflow)
-* Detect write after the end of the buffer (buffer overflow)
+* Detect writes before the start of a buffer (buffer underflows)
+* Detect writes after the end of a buffer (buffer overflows)
 * Check that the :term:`GIL <global interpreter lock>` is held when allocator
   functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and
   :c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called.
@@ -658,8 +658,8 @@
 
 It is now also possible to force the usage of the :c:func:`malloc` allocator of
 the C library for all Python memory allocations using ``PYTHONMALLOC=malloc``.
-It helps to use external memory debuggers like Valgrind on a Python compiled in
-release mode.
+This is helpful when using external memory debuggers like Valgrind on
+a Python compiled in release mode.
 
 On error, the debug hooks on Python memory allocators now use the
 :mod:`tracemalloc` module to get the traceback where a memory block was
@@ -754,7 +754,7 @@
 
 * Import now raises the new exception :exc:`ModuleNotFoundError`
   (subclass of :exc:`ImportError`) when it cannot find a module.  Code
-  that current checks for ImportError (in try-except) will still work.
+  that currently checks for ImportError (in try-except) will still work.
   (Contributed by Eric Snow in :issue:`15767`.)
 
 * Class methods relying on zero-argument ``super()`` will now work correctly
@@ -810,7 +810,7 @@
 asyncio
 -------
 
-Starting with Python 3.6 the ``asyncio`` is no longer provisional and its
+Starting with Python 3.6 the ``asyncio`` module is no longer provisional and its
 API is considered stable.
 
 Notable changes in the :mod:`asyncio` module since Python 3.5.0
@@ -871,7 +871,7 @@
 
 * :meth:`Future.set_exception <asyncio.futures.Future.set_exception>`
   will now raise :exc:`TypeError` when passed an instance of
-  :exc:`StopIteration` exception.
+  the :exc:`StopIteration` exception.
   (Contributed by Chris Angelico in :issue:`26221`.)
 
 * New :meth:`Loop.connect_accepted_socket() <asyncio.BaseEventLoop.connect_accepted_socket>`
@@ -909,7 +909,7 @@
 (Contributed by Ivan Levkivskyi, docs by Neil Girdhar in :issue:`27598`.)
 
 The new :class:`~collections.abc.Reversible` abstract base class represents
-iterable classes that also provide the :meth:`__reversed__`.
+iterable classes that also provide the :meth:`__reversed__` method.
 (Contributed by Ivan Levkivskyi in :issue:`25987`.)
 
 The new :class:`~collections.abc.AsyncGenerator` abstract base class represents
@@ -932,7 +932,7 @@
 concurrent.futures
 ------------------
 
-The :class:`ThreadPoolExecutor <concurrent.futures.thread.ThreadPoolExecutor`
+The :class:`ThreadPoolExecutor <concurrent.futures.ThreadPoolExecutor>`
 class constructor now accepts an optional *thread_name_prefix* argument
 to make it possible to customize the names of the threads created by the
 pool.
@@ -998,7 +998,7 @@
 The ``default_format`` attribute has been removed from
 :class:`distutils.command.sdist.sdist` and the ``formats``
 attribute defaults to ``['gztar']``. Although not anticipated,
-Any code relying on the presence of ``default_format`` may
+any code relying on the presence of ``default_format`` may
 need to be adapted. See :issue:`27819` for more details.
 
 
@@ -1027,7 +1027,7 @@
 encodings
 ---------
 
-On Windows, added the ``'oem'`` encoding to use ``CP_OEMCP`` and the ``'ansi'``
+On Windows, added the ``'oem'`` encoding to use ``CP_OEMCP``, and the ``'ansi'``
 alias for the existing ``'mbcs'`` encoding, which uses the ``CP_ACP`` code page.
 (Contributed by Steve Dower in :issue:`27959`.)
 
@@ -1192,7 +1192,7 @@
 os
 --
 
-See the summary for :ref:`PEP 519 <whatsnew36-pep519>` for details on how the
+See the summary of :ref:`PEP 519 <whatsnew36-pep519>` for details on how the
 :mod:`os` and :mod:`os.path` modules now support
 :term:`path-like objects <path-like object>`.
 
@@ -1219,7 +1219,7 @@
 :mod:`pathlib` now supports :term:`path-like objects <path-like object>`.
 (Contributed by Brett Cannon in :issue:`27186`.)
 
-See the summary for :ref:`PEP 519 <whatsnew36-pep519>` for details.
+See the summary of :ref:`PEP 519 <whatsnew36-pep519>` for details.
 
 
 pdb
@@ -1232,7 +1232,7 @@
 pickle
 ------
 
-Objects that need calling ``__new__`` with keyword arguments can now be pickled
+Objects that need ``__new__`` called with keyword arguments can now be pickled
 using :ref:`pickle protocols <pickle-protocols>` older than protocol version 4.
 Protocol version 4 already supports this case.  (Contributed by Serhiy
 Storchaka in :issue:`24164`.)
@@ -1241,7 +1241,7 @@
 pickletools
 -----------
 
-:func:`pickletools.dis()` now outputs implicit memo index for the
+:func:`pickletools.dis()` now outputs the implicit memo index for the
 ``MEMOIZE`` opcode.
 (Contributed by Serhiy Storchaka in :issue:`25382`.)
 
@@ -1278,7 +1278,7 @@
 equivalent to ``group()``.  So ``mo['name']`` is now equivalent to
 ``mo.group('name')``.  (Contributed by Eric Smith in :issue:`24454`.)
 
-:class:`~re.Match` objects in the now support
+:class:`~re.Match` objects now support
 :meth:`index-like objects <object.__index__>` as group
 indices.
 (Contributed by Jeroen Demeyer and Xiang Zhang in :issue:`27177`.)
@@ -1338,7 +1338,7 @@
 
 The :meth:`~socket.socket.setsockopt` now supports the
 ``setsockopt(level, optname, None, optlen: int)`` form.
-(Contributed by Christian Heimes in issue:`27744`.)
+(Contributed by Christian Heimes in :issue:`27744`.)
 
 The socket module now supports the address family
 :data:`~socket.AF_ALG` to interface with Linux Kernel crypto API. ``ALG_*``,
@@ -1415,9 +1415,9 @@
 
 :class:`subprocess.Popen` destructor now emits a :exc:`ResourceWarning` warning
 if the child process is still running. Use the context manager protocol (``with
-proc: ...``) or call explicitly the :meth:`~subprocess.Popen.wait` method to
-read the exit status of the child process (Contributed by Victor Stinner in
-:issue:`26741`).
+proc: ...``) or explicitly call the :meth:`~subprocess.Popen.wait` method to
+read the exit status of the child process. (Contributed by Victor Stinner in
+:issue:`26741`.)
 
 The :class:`subprocess.Popen` constructor and all functions that pass arguments
 through to it now accept *encoding* and *errors* arguments. Specifying either
@@ -1625,8 +1625,8 @@
 :class:`warnings.WarningMessage` (contributed by Victor Stinner in
 :issue:`26568` and :issue:`26567`).
 
-When a :exc:`ResourceWarning` warning is logged, the :mod:`tracemalloc` is now
-used to try to retrieve the traceback where the detroyed object was allocated.
+When a :exc:`ResourceWarning` warning is logged, the :mod:`tracemalloc` module is now
+used to try to retrieve the traceback where the destroyed object was allocated.
 
 Example with the script ``example.py``::
 
@@ -1648,9 +1648,9 @@
       File "example.py", lineno 6
         f = func()
 
-The "Object allocated at" traceback is new and only displayed if
+The "Object allocated at" traceback is new and is only displayed if
 :mod:`tracemalloc` is tracing Python memory allocations and if the
-:mod:`warnings` was already imported.
+:mod:`warnings` module was already imported.
 
 
 winreg
@@ -1672,7 +1672,7 @@
 -------------
 
 The :mod:`xmlrpc.client` module now supports unmarshalling
-additional data types used by Apache XML-RPC implementation
+additional data types used by the Apache XML-RPC implementation
 for numerics and ``None``.
 (Contributed by Serhiy Storchaka in :issue:`26885`.)
 
@@ -1703,16 +1703,16 @@
 Optimizations
 =============
 
-* Python interpreter now uses 16-bit wordcode instead of bytecode which
+* The Python interpreter now uses a 16-bit wordcode instead of bytecode which
   made a number of opcode optimizations possible.
   (Contributed by Demur Rumed with input and reviews from
   Serhiy Storchaka and Victor Stinner in :issue:`26647` and :issue:`28050`.)
 
-* The :class:`Future <asyncio.futures.Future>` now has an optimized
+* The :class:`Future <asyncio.futures.Future>` class now has an optimized
   C implementation.
   (Contributed by Yury Selivanov and INADA Naoki in :issue:`26801`.)
 
-* The :class:`Task <asyncio.tasks.Task>` now has an optimized
+* The :class:`Task <asyncio.tasks.Task>` class now has an optimized
   C implementation. (Contributed by Yury Selivanov in :issue:`28544`.)
 
 * Various implementation improvements in the :mod:`typing` module
@@ -1758,7 +1758,7 @@
   deserializing many small objects (Contributed by Victor Stinner in
   :issue:`27056`).
 
-- Passing :term:`keyword arguments <keyword argument>` to a function has an
+* Passing :term:`keyword arguments <keyword argument>` to a function has an
   overhead in comparison with passing :term:`positional arguments
   <positional argument>`.  Now in extension functions implemented with using
   Argument Clinic this overhead is significantly decreased.
@@ -1791,7 +1791,7 @@
   For more information, see :pep:`7` and :issue:`17884`.
 
 * Cross-compiling CPython with the Android NDK and the Android API level set to
-  21 (Android 5.0 Lollilop) or greater, runs successfully. While Android is not
+  21 (Android 5.0 Lollilop) or greater runs successfully. While Android is not
   yet a supported platform, the Python test suite runs on the Android emulator
   with only about 16 tests failures. See the Android meta-issue :issue:`26865`.
 
@@ -1821,7 +1821,7 @@
   (Contributed by Eric Snow in :issue:`15767`.)
 
 * The new :c:func:`PyErr_ResourceWarning` function can be used to generate
-  the :exc:`ResourceWarning` providing the source of the resource allocation.
+  a :exc:`ResourceWarning` providing the source of the resource allocation.
   (Contributed by Victor Stinner in :issue:`26567`.)
 
 * The new :c:func:`PyOS_FSPath` function returns the file system
@@ -1977,7 +1977,7 @@
 Undocumented functions :c:func:`PyUnicode_AsEncodedObject`,
 :c:func:`PyUnicode_AsDecodedObject`, :c:func:`PyUnicode_AsEncodedUnicode`
 and :c:func:`PyUnicode_AsDecodedUnicode` are deprecated now.
-Use :ref:`generic codec based API <codec-registry>` instead.
+Use the :ref:`generic codec based API <codec-registry>` instead.
 
 
 Deprecated Build Options
@@ -2054,7 +2054,7 @@
   with ``'+'``.
   (Contributed by Jeff Balogh and John O'Connor in :issue:`2091`.)
 
-* :mod:`sqlite3` no longer implicitly commit an open transaction before DDL
+* :mod:`sqlite3` no longer implicitly commits an open transaction before DDL
   statements.
 
 * On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool
@@ -2067,12 +2067,12 @@
   argument is not set. Previously only ``NULL`` was returned.
 
 * The format of the ``co_lnotab`` attribute of code objects changed to support
-  negative line number delta. By default, Python does not emit bytecode with
-  negative line number delta. Functions using ``frame.f_lineno``,
+  a negative line number delta. By default, Python does not emit bytecode with
+  a negative line number delta. Functions using ``frame.f_lineno``,
   ``PyFrame_GetLineNumber()`` or ``PyCode_Addr2Line()`` are not affected.
-  Functions decoding directly ``co_lnotab`` should be updated to use a signed
-  8-bit integer type for the line number delta, but it's only required to
-  support applications using negative line number delta. See
+  Functions directly decoding ``co_lnotab`` should be updated to use a signed
+  8-bit integer type for the line number delta, but this is only required to
+  support applications using a negative line number delta. See
   ``Objects/lnotab_notes.txt`` for the ``co_lnotab`` format and how to decode
   it, and see the :pep:`511` for the rationale.
 
@@ -2124,7 +2124,7 @@
   an error (e.g. ``EBADF``) was reported by the underlying system call.
   (Contributed by Martin Panter in :issue:`26685`.)
 
-* The *decode_data* argument for :class:`smtpd.SMTPChannel` and
+* The *decode_data* argument for the :class:`smtpd.SMTPChannel` and
   :class:`smtpd.SMTPServer` constructors is now ``False`` by default.
   This means that the argument passed to
   :meth:`~smtpd.SMTPServer.process_message` is now a bytes object by
@@ -2204,15 +2204,15 @@
   (Contributed by Ramchandra Apte in :issue:`17211`.)
 
 * :func:`re.sub` now raises an error for invalid numerical group
-  reference in replacement template even if the pattern is not
-  found in the string.  Error message for invalid group reference
+  references in replacement templates even if the pattern is not
+  found in the string.  The error message for invalid group references
   now includes the group index and the position of the reference.
   (Contributed by SilentGhost, Serhiy Storchaka in :issue:`25953`.)
 
 * :class:`zipfile.ZipFile` will now raise :exc:`NotImplementedError` for
   unrecognized compression values.  Previously a plain :exc:`RuntimeError`
-  was raised.  Additionally, calling :class:`~zipfile.ZipFile` methods or
-  on a closed ZipFile or calling :meth:`~zipfile.ZipFile.write` methods
+  was raised.  Additionally, calling :class:`~zipfile.ZipFile` methods
+  on a closed ZipFile or calling the :meth:`~zipfile.ZipFile.write` method
   on a ZipFile created with mode ``'r'`` will raise a :exc:`ValueError`.
   Previously, a :exc:`RuntimeError` was raised in those scenarios.
 
@@ -2220,8 +2220,8 @@
 Changes in the C API
 --------------------
 
-* :c:func:`PyMem_Malloc` allocator family now uses the :ref:`pymalloc allocator
-  <pymalloc>` rather than system :c:func:`malloc`. Applications calling
+* The :c:func:`PyMem_Malloc` allocator family now uses the :ref:`pymalloc allocator
+  <pymalloc>` rather than the system :c:func:`malloc`. Applications calling
   :c:func:`PyMem_Malloc` without holding the GIL can now crash. Set the
   :envvar:`PYTHONMALLOC` environment variable to ``debug`` to validate the
   usage of memory allocators in your application. See :issue:`26249`.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 08:42:08 2016
From: python-checkins at python.org (martin.panter)
Date: Mon, 21 Nov 2016 13:42:08 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Fix_up_grammar?=
 =?utf-8?b?LCBtYXJrdXAsIGV0YyBpbiAzLjYgV2hhdOKAmXMgTmV3?=
Message-ID: <20161121134207.14068.51969.A1E55381@psf.io>

https://hg.python.org/cpython/rev/a429d29eafbf
changeset:   105279:a429d29eafbf
branch:      3.6
user:        Martin Panter <vadmium+py at gmail.com>
date:        Mon Nov 21 13:38:59 2016 +0000
summary:
  Fix up grammar, markup, etc in 3.6 What?s New

files:
  Doc/whatsnew/3.6.rst |  102 +++++++++++++++---------------
  1 files changed, 51 insertions(+), 51 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -425,16 +425,16 @@
 object.
 
 The built-in :func:`open` function has been updated to accept
-:class:`os.PathLike` objects as have all relevant functions in the
-:mod:`os` and :mod:`os.path` modules, as well as most functions and
+:class:`os.PathLike` objects, as have all relevant functions in the
+:mod:`os` and :mod:`os.path` modules, and most other functions and
 classes in the standard library.  The :class:`os.DirEntry` class
 and relevant classes in :mod:`pathlib` have also been updated to
 implement :class:`os.PathLike`.
 
-The hope in is that updating the fundamental functions for operating
+The hope is that updating the fundamental functions for operating
 on file system paths will lead to third-party code to implicitly
 support all :term:`path-like objects <path-like object>` without any
-code changes or at least very minimal ones (e.g. calling
+code changes, or at least very minimal ones (e.g. calling
 :func:`os.fspath` at the beginning of code before operating on a
 path-like object).
 
@@ -635,18 +635,18 @@
 ---------------------------------
 
 The new :envvar:`PYTHONMALLOC` environment variable allows setting the Python
-memory allocators and/or install debug hooks.
+memory allocators and installing debug hooks.
 
 It is now possible to install debug hooks on Python memory allocators on Python
 compiled in release mode using ``PYTHONMALLOC=debug``. Effects of debug hooks:
 
 * Newly allocated memory is filled with the byte ``0xCB``
 * Freed memory is filled with the byte ``0xDB``
-* Detect violations of Python memory allocator API. For example,
+* Detect violations of the Python memory allocator API. For example,
   :c:func:`PyObject_Free` called on a memory block allocated by
   :c:func:`PyMem_Malloc`.
-* Detect write before the start of the buffer (buffer underflow)
-* Detect write after the end of the buffer (buffer overflow)
+* Detect writes before the start of a buffer (buffer underflows)
+* Detect writes after the end of a buffer (buffer overflows)
 * Check that the :term:`GIL <global interpreter lock>` is held when allocator
   functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and
   :c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called.
@@ -658,8 +658,8 @@
 
 It is now also possible to force the usage of the :c:func:`malloc` allocator of
 the C library for all Python memory allocations using ``PYTHONMALLOC=malloc``.
-It helps to use external memory debuggers like Valgrind on a Python compiled in
-release mode.
+This is helpful when using external memory debuggers like Valgrind on
+a Python compiled in release mode.
 
 On error, the debug hooks on Python memory allocators now use the
 :mod:`tracemalloc` module to get the traceback where a memory block was
@@ -754,7 +754,7 @@
 
 * Import now raises the new exception :exc:`ModuleNotFoundError`
   (subclass of :exc:`ImportError`) when it cannot find a module.  Code
-  that current checks for ImportError (in try-except) will still work.
+  that currently checks for ImportError (in try-except) will still work.
   (Contributed by Eric Snow in :issue:`15767`.)
 
 * Class methods relying on zero-argument ``super()`` will now work correctly
@@ -810,7 +810,7 @@
 asyncio
 -------
 
-Starting with Python 3.6 the ``asyncio`` is no longer provisional and its
+Starting with Python 3.6 the ``asyncio`` module is no longer provisional and its
 API is considered stable.
 
 Notable changes in the :mod:`asyncio` module since Python 3.5.0
@@ -871,7 +871,7 @@
 
 * :meth:`Future.set_exception <asyncio.futures.Future.set_exception>`
   will now raise :exc:`TypeError` when passed an instance of
-  :exc:`StopIteration` exception.
+  the :exc:`StopIteration` exception.
   (Contributed by Chris Angelico in :issue:`26221`.)
 
 * New :meth:`Loop.connect_accepted_socket() <asyncio.BaseEventLoop.connect_accepted_socket>`
@@ -909,7 +909,7 @@
 (Contributed by Ivan Levkivskyi, docs by Neil Girdhar in :issue:`27598`.)
 
 The new :class:`~collections.abc.Reversible` abstract base class represents
-iterable classes that also provide the :meth:`__reversed__`.
+iterable classes that also provide the :meth:`__reversed__` method.
 (Contributed by Ivan Levkivskyi in :issue:`25987`.)
 
 The new :class:`~collections.abc.AsyncGenerator` abstract base class represents
@@ -932,7 +932,7 @@
 concurrent.futures
 ------------------
 
-The :class:`ThreadPoolExecutor <concurrent.futures.thread.ThreadPoolExecutor`
+The :class:`ThreadPoolExecutor <concurrent.futures.ThreadPoolExecutor>`
 class constructor now accepts an optional *thread_name_prefix* argument
 to make it possible to customize the names of the threads created by the
 pool.
@@ -998,7 +998,7 @@
 The ``default_format`` attribute has been removed from
 :class:`distutils.command.sdist.sdist` and the ``formats``
 attribute defaults to ``['gztar']``. Although not anticipated,
-Any code relying on the presence of ``default_format`` may
+any code relying on the presence of ``default_format`` may
 need to be adapted. See :issue:`27819` for more details.
 
 
@@ -1027,7 +1027,7 @@
 encodings
 ---------
 
-On Windows, added the ``'oem'`` encoding to use ``CP_OEMCP`` and the ``'ansi'``
+On Windows, added the ``'oem'`` encoding to use ``CP_OEMCP``, and the ``'ansi'``
 alias for the existing ``'mbcs'`` encoding, which uses the ``CP_ACP`` code page.
 (Contributed by Steve Dower in :issue:`27959`.)
 
@@ -1192,7 +1192,7 @@
 os
 --
 
-See the summary for :ref:`PEP 519 <whatsnew36-pep519>` for details on how the
+See the summary of :ref:`PEP 519 <whatsnew36-pep519>` for details on how the
 :mod:`os` and :mod:`os.path` modules now support
 :term:`path-like objects <path-like object>`.
 
@@ -1219,7 +1219,7 @@
 :mod:`pathlib` now supports :term:`path-like objects <path-like object>`.
 (Contributed by Brett Cannon in :issue:`27186`.)
 
-See the summary for :ref:`PEP 519 <whatsnew36-pep519>` for details.
+See the summary of :ref:`PEP 519 <whatsnew36-pep519>` for details.
 
 
 pdb
@@ -1232,7 +1232,7 @@
 pickle
 ------
 
-Objects that need calling ``__new__`` with keyword arguments can now be pickled
+Objects that need ``__new__`` called with keyword arguments can now be pickled
 using :ref:`pickle protocols <pickle-protocols>` older than protocol version 4.
 Protocol version 4 already supports this case.  (Contributed by Serhiy
 Storchaka in :issue:`24164`.)
@@ -1241,7 +1241,7 @@
 pickletools
 -----------
 
-:func:`pickletools.dis()` now outputs implicit memo index for the
+:func:`pickletools.dis()` now outputs the implicit memo index for the
 ``MEMOIZE`` opcode.
 (Contributed by Serhiy Storchaka in :issue:`25382`.)
 
@@ -1278,7 +1278,7 @@
 equivalent to ``group()``.  So ``mo['name']`` is now equivalent to
 ``mo.group('name')``.  (Contributed by Eric Smith in :issue:`24454`.)
 
-:class:`~re.Match` objects in the now support
+:class:`~re.Match` objects now support
 :meth:`index-like objects <object.__index__>` as group
 indices.
 (Contributed by Jeroen Demeyer and Xiang Zhang in :issue:`27177`.)
@@ -1338,7 +1338,7 @@
 
 The :meth:`~socket.socket.setsockopt` now supports the
 ``setsockopt(level, optname, None, optlen: int)`` form.
-(Contributed by Christian Heimes in issue:`27744`.)
+(Contributed by Christian Heimes in :issue:`27744`.)
 
 The socket module now supports the address family
 :data:`~socket.AF_ALG` to interface with Linux Kernel crypto API. ``ALG_*``,
@@ -1415,9 +1415,9 @@
 
 :class:`subprocess.Popen` destructor now emits a :exc:`ResourceWarning` warning
 if the child process is still running. Use the context manager protocol (``with
-proc: ...``) or call explicitly the :meth:`~subprocess.Popen.wait` method to
-read the exit status of the child process (Contributed by Victor Stinner in
-:issue:`26741`).
+proc: ...``) or explicitly call the :meth:`~subprocess.Popen.wait` method to
+read the exit status of the child process. (Contributed by Victor Stinner in
+:issue:`26741`.)
 
 The :class:`subprocess.Popen` constructor and all functions that pass arguments
 through to it now accept *encoding* and *errors* arguments. Specifying either
@@ -1625,8 +1625,8 @@
 :class:`warnings.WarningMessage` (contributed by Victor Stinner in
 :issue:`26568` and :issue:`26567`).
 
-When a :exc:`ResourceWarning` warning is logged, the :mod:`tracemalloc` is now
-used to try to retrieve the traceback where the detroyed object was allocated.
+When a :exc:`ResourceWarning` warning is logged, the :mod:`tracemalloc` module is now
+used to try to retrieve the traceback where the destroyed object was allocated.
 
 Example with the script ``example.py``::
 
@@ -1648,9 +1648,9 @@
       File "example.py", lineno 6
         f = func()
 
-The "Object allocated at" traceback is new and only displayed if
+The "Object allocated at" traceback is new and is only displayed if
 :mod:`tracemalloc` is tracing Python memory allocations and if the
-:mod:`warnings` was already imported.
+:mod:`warnings` module was already imported.
 
 
 winreg
@@ -1672,7 +1672,7 @@
 -------------
 
 The :mod:`xmlrpc.client` module now supports unmarshalling
-additional data types used by Apache XML-RPC implementation
+additional data types used by the Apache XML-RPC implementation
 for numerics and ``None``.
 (Contributed by Serhiy Storchaka in :issue:`26885`.)
 
@@ -1703,16 +1703,16 @@
 Optimizations
 =============
 
-* Python interpreter now uses 16-bit wordcode instead of bytecode which
+* The Python interpreter now uses a 16-bit wordcode instead of bytecode which
   made a number of opcode optimizations possible.
   (Contributed by Demur Rumed with input and reviews from
   Serhiy Storchaka and Victor Stinner in :issue:`26647` and :issue:`28050`.)
 
-* The :class:`Future <asyncio.futures.Future>` now has an optimized
+* The :class:`Future <asyncio.futures.Future>` class now has an optimized
   C implementation.
   (Contributed by Yury Selivanov and INADA Naoki in :issue:`26801`.)
 
-* The :class:`Task <asyncio.tasks.Task>` now has an optimized
+* The :class:`Task <asyncio.tasks.Task>` class now has an optimized
   C implementation. (Contributed by Yury Selivanov in :issue:`28544`.)
 
 * Various implementation improvements in the :mod:`typing` module
@@ -1758,7 +1758,7 @@
   deserializing many small objects (Contributed by Victor Stinner in
   :issue:`27056`).
 
-- Passing :term:`keyword arguments <keyword argument>` to a function has an
+* Passing :term:`keyword arguments <keyword argument>` to a function has an
   overhead in comparison with passing :term:`positional arguments
   <positional argument>`.  Now in extension functions implemented with using
   Argument Clinic this overhead is significantly decreased.
@@ -1791,7 +1791,7 @@
   For more information, see :pep:`7` and :issue:`17884`.
 
 * Cross-compiling CPython with the Android NDK and the Android API level set to
-  21 (Android 5.0 Lollilop) or greater, runs successfully. While Android is not
+  21 (Android 5.0 Lollilop) or greater runs successfully. While Android is not
   yet a supported platform, the Python test suite runs on the Android emulator
   with only about 16 tests failures. See the Android meta-issue :issue:`26865`.
 
@@ -1821,7 +1821,7 @@
   (Contributed by Eric Snow in :issue:`15767`.)
 
 * The new :c:func:`PyErr_ResourceWarning` function can be used to generate
-  the :exc:`ResourceWarning` providing the source of the resource allocation.
+  a :exc:`ResourceWarning` providing the source of the resource allocation.
   (Contributed by Victor Stinner in :issue:`26567`.)
 
 * The new :c:func:`PyOS_FSPath` function returns the file system
@@ -1977,7 +1977,7 @@
 Undocumented functions :c:func:`PyUnicode_AsEncodedObject`,
 :c:func:`PyUnicode_AsDecodedObject`, :c:func:`PyUnicode_AsEncodedUnicode`
 and :c:func:`PyUnicode_AsDecodedUnicode` are deprecated now.
-Use :ref:`generic codec based API <codec-registry>` instead.
+Use the :ref:`generic codec based API <codec-registry>` instead.
 
 
 Deprecated Build Options
@@ -2054,7 +2054,7 @@
   with ``'+'``.
   (Contributed by Jeff Balogh and John O'Connor in :issue:`2091`.)
 
-* :mod:`sqlite3` no longer implicitly commit an open transaction before DDL
+* :mod:`sqlite3` no longer implicitly commits an open transaction before DDL
   statements.
 
 * On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool
@@ -2067,12 +2067,12 @@
   argument is not set. Previously only ``NULL`` was returned.
 
 * The format of the ``co_lnotab`` attribute of code objects changed to support
-  negative line number delta. By default, Python does not emit bytecode with
-  negative line number delta. Functions using ``frame.f_lineno``,
+  a negative line number delta. By default, Python does not emit bytecode with
+  a negative line number delta. Functions using ``frame.f_lineno``,
   ``PyFrame_GetLineNumber()`` or ``PyCode_Addr2Line()`` are not affected.
-  Functions decoding directly ``co_lnotab`` should be updated to use a signed
-  8-bit integer type for the line number delta, but it's only required to
-  support applications using negative line number delta. See
+  Functions directly decoding ``co_lnotab`` should be updated to use a signed
+  8-bit integer type for the line number delta, but this is only required to
+  support applications using a negative line number delta. See
   ``Objects/lnotab_notes.txt`` for the ``co_lnotab`` format and how to decode
   it, and see the :pep:`511` for the rationale.
 
@@ -2124,7 +2124,7 @@
   an error (e.g. ``EBADF``) was reported by the underlying system call.
   (Contributed by Martin Panter in :issue:`26685`.)
 
-* The *decode_data* argument for :class:`smtpd.SMTPChannel` and
+* The *decode_data* argument for the :class:`smtpd.SMTPChannel` and
   :class:`smtpd.SMTPServer` constructors is now ``False`` by default.
   This means that the argument passed to
   :meth:`~smtpd.SMTPServer.process_message` is now a bytes object by
@@ -2204,15 +2204,15 @@
   (Contributed by Ramchandra Apte in :issue:`17211`.)
 
 * :func:`re.sub` now raises an error for invalid numerical group
-  reference in replacement template even if the pattern is not
-  found in the string.  Error message for invalid group reference
+  references in replacement templates even if the pattern is not
+  found in the string.  The error message for invalid group references
   now includes the group index and the position of the reference.
   (Contributed by SilentGhost, Serhiy Storchaka in :issue:`25953`.)
 
 * :class:`zipfile.ZipFile` will now raise :exc:`NotImplementedError` for
   unrecognized compression values.  Previously a plain :exc:`RuntimeError`
-  was raised.  Additionally, calling :class:`~zipfile.ZipFile` methods or
-  on a closed ZipFile or calling :meth:`~zipfile.ZipFile.write` methods
+  was raised.  Additionally, calling :class:`~zipfile.ZipFile` methods
+  on a closed ZipFile or calling the :meth:`~zipfile.ZipFile.write` method
   on a ZipFile created with mode ``'r'`` will raise a :exc:`ValueError`.
   Previously, a :exc:`RuntimeError` was raised in those scenarios.
 
@@ -2220,8 +2220,8 @@
 Changes in the C API
 --------------------
 
-* :c:func:`PyMem_Malloc` allocator family now uses the :ref:`pymalloc allocator
-  <pymalloc>` rather than system :c:func:`malloc`. Applications calling
+* The :c:func:`PyMem_Malloc` allocator family now uses the :ref:`pymalloc allocator
+  <pymalloc>` rather than the system :c:func:`malloc`. Applications calling
   :c:func:`PyMem_Malloc` without holding the GIL can now crash. Set the
   :envvar:`PYTHONMALLOC` environment variable to ``debug`` to validate the
   usage of memory allocators in your application. See :issue:`26249`.

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Mon Nov 21 10:33:02 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Mon, 21 Nov 2016 15:33:02 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python 2.7 2016-11-21
Message-ID: <254e938e-8940-4dbf-94da-42b4d719fd9e@irsmsx151.ger.corp.intel.com>

Results for project Python 2.7, build date 2016-11-21 03:49:20 +0000
commit:		4de956751cf1
previous commit:	ea91bb92c28b
revision date:	2016-11-20 09:35:06 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.14%         -0.16%          4.41%             7.05%
:-)             pybench      0.14%          0.03%          5.76%             3.75%
:-(            regex_v8      1.08%         -0.47%         -2.64%            11.45%
:-)               nbody      0.10%         -0.01%          8.41%             2.57%
:-)        json_dump_v2      0.34%         -0.57%          3.09%             8.67%
:-|      normal_startup      1.17%          1.41%          0.53%             1.35%
:-)             ssbench      0.15%          0.21%          2.52%             1.93%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-2-7-2016-11-21/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Mon Nov 21 10:39:07 2016
From: python-checkins at python.org (victor.stinner)
Date: Mon, 21 Nov 2016 15:39:07 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Implement_rich?=
 =?utf-8?q?_comparison_for_=5Fsre=2ESRE=5FPattern?=
Message-ID: <20161121153907.8083.33335.DA3359A9@psf.io>

https://hg.python.org/cpython/rev/5e8ef1493843
changeset:   105281:5e8ef1493843
branch:      3.6
parent:      105279:a429d29eafbf
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Nov 21 16:35:08 2016 +0100
summary:
  Implement rich comparison for _sre.SRE_Pattern

Issue #28727: Regular expression patterns, _sre.SRE_Pattern objects created by
re.compile(), become comparable (only x==y and x!=y operators). This change
should fix the issue #18383: don't duplicate warning filters when the warnings
module is reloaded (thing usually only done in unit tests).

files:
  Lib/test/test_re.py |  47 ++++++++++++++++++++-
  Misc/NEWS           |   7 ++-
  Modules/_sre.c      |  73 ++++++++++++++++++++++++++++++--
  3 files changed, 118 insertions(+), 9 deletions(-)


diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py
--- a/Lib/test/test_re.py
+++ b/Lib/test/test_re.py
@@ -3,12 +3,13 @@
 import io
 import locale
 import re
-from re import Scanner
 import sre_compile
+import string
 import sys
-import string
 import traceback
 import unittest
+import warnings
+from re import Scanner
 from weakref import proxy
 
 # Misc tests from Tim Peters' re.doc
@@ -1777,6 +1778,48 @@
         self.assertIn('ASCII', str(re.A))
         self.assertIn('DOTALL', str(re.S))
 
+    def test_pattern_compare(self):
+        pattern1 = re.compile('abc', re.IGNORECASE)
+
+        # equal
+        re.purge()
+        pattern2 = re.compile('abc', re.IGNORECASE)
+        self.assertEqual(hash(pattern2), hash(pattern1))
+        self.assertEqual(pattern2, pattern1)
+
+        # not equal: different pattern
+        re.purge()
+        pattern3 = re.compile('XYZ', re.IGNORECASE)
+        # Don't test hash(pattern3) != hash(pattern1) because there is no
+        # warranty that hash values are different
+        self.assertNotEqual(pattern3, pattern1)
+
+        # not equal: different flag (flags=0)
+        re.purge()
+        pattern4 = re.compile('abc')
+        self.assertNotEqual(pattern4, pattern1)
+
+        # only == and != comparison operators are supported
+        with self.assertRaises(TypeError):
+            pattern1 < pattern2
+
+    def test_pattern_compare_bytes(self):
+        pattern1 = re.compile(b'abc')
+
+        # equal: test bytes patterns
+        re.purge()
+        pattern2 = re.compile(b'abc')
+        self.assertEqual(hash(pattern2), hash(pattern1))
+        self.assertEqual(pattern2, pattern1)
+
+        # not equal: pattern of a different types (str vs bytes),
+        # comparison must not raise a BytesWarning
+        re.purge()
+        pattern3 = re.compile('abc')
+        with warnings.catch_warnings():
+            warnings.simplefilter('error', BytesWarning)
+            self.assertNotEqual(pattern3, pattern1)
+
 
 class PatternReprTests(unittest.TestCase):
     def check(self, pattern, expected):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -42,6 +42,11 @@
 Library
 -------
 
+- Issue #28727: Regular expression patterns, _sre.SRE_Pattern objects created
+  by re.compile(), become comparable (only x==y and x!=y operators). This
+  change should fix the issue #18383: don't duplicate warning filters when the
+  warnings module is reloaded (thing usually only done in unit tests).
+
 - Issue #20572: The subprocess.Popen.wait method's undocumented
   endtime parameter now raises a DeprecationWarning.
 
@@ -77,7 +82,7 @@
 
 - Issue #28703: Fix asyncio.iscoroutinefunction to handle Mock objects.
 
-- Issue #28704: Fix create_unix_server to support Path-like objects 
+- Issue #28704: Fix create_unix_server to support Path-like objects
   (PEP 519).
 
 - Issue #28720: Add collections.abc.AsyncGenerator.
diff --git a/Modules/_sre.c b/Modules/_sre.c
--- a/Modules/_sre.c
+++ b/Modules/_sre.c
@@ -1506,14 +1506,12 @@
 
     self->groups = groups;
 
-    Py_XINCREF(groupindex);
+    Py_INCREF(groupindex);
     self->groupindex = groupindex;
 
-    Py_XINCREF(indexgroup);
+    Py_INCREF(indexgroup);
     self->indexgroup = indexgroup;
 
-    self->weakreflist = NULL;
-
     if (!_validate(self)) {
         Py_DECREF(self);
         return NULL;
@@ -2649,6 +2647,69 @@
     return (PyObject*) scanner;
 }
 
+static Py_hash_t
+pattern_hash(PatternObject *self)
+{
+    Py_hash_t hash, hash2;
+
+    hash = PyObject_Hash(self->pattern);
+    if (hash == -1) {
+        return -1;
+    }
+
+    hash2 = _Py_HashBytes(self->code, sizeof(self->code[0]) * self->codesize);
+    hash ^= hash2;
+
+    hash ^= self->flags;
+    hash ^= self->isbytes;
+    hash ^= self->codesize;
+
+    if (hash == -1) {
+        hash = -2;
+    }
+    return hash;
+}
+
+static PyObject*
+pattern_richcompare(PyObject *lefto, PyObject *righto, int op)
+{
+    PatternObject *left, *right;
+    int cmp;
+
+    if (op != Py_EQ && op != Py_NE) {
+        Py_RETURN_NOTIMPLEMENTED;
+    }
+
+    if (Py_TYPE(lefto) != &Pattern_Type || Py_TYPE(righto) != &Pattern_Type) {
+        Py_RETURN_NOTIMPLEMENTED;
+    }
+    left = (PatternObject *)lefto;
+    right = (PatternObject *)righto;
+
+    cmp = (left->flags == right->flags
+           && left->isbytes == right->isbytes
+           && left->codesize && right->codesize);
+    if (cmp) {
+        /* Compare the code and the pattern because the same pattern can
+           produce different codes depending on the locale used to compile the
+           pattern when the re.LOCALE flag is used. Don't compare groups,
+           indexgroup nor groupindex: they are derivated from the pattern. */
+        cmp = (memcmp(left->code, right->code,
+                      sizeof(left->code[0]) * left->codesize) == 0);
+    }
+    if (cmp) {
+        cmp = PyObject_RichCompareBool(left->pattern, right->pattern,
+                                       Py_EQ);
+        if (cmp < 0) {
+            return NULL;
+        }
+    }
+    if (op == Py_NE) {
+        cmp = !cmp;
+    }
+    return PyBool_FromLong(cmp);
+}
+
 #include "clinic/_sre.c.h"
 
 static PyMethodDef pattern_methods[] = {
@@ -2693,7 +2754,7 @@
     0,                                  /* tp_as_number */
     0,                                  /* tp_as_sequence */
     0,                                  /* tp_as_mapping */
-    0,                                  /* tp_hash */
+    (hashfunc)pattern_hash,             /* tp_hash */
     0,                                  /* tp_call */
     0,                                  /* tp_str */
     0,                                  /* tp_getattro */
@@ -2703,7 +2764,7 @@
     pattern_doc,                        /* tp_doc */
     0,                                  /* tp_traverse */
     0,                                  /* tp_clear */
-    0,                                  /* tp_richcompare */
+    pattern_richcompare,                /* tp_richcompare */
     offsetof(PatternObject, weakreflist),       /* tp_weaklistoffset */
     0,                                  /* tp_iter */
     0,                                  /* tp_iternext */

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 10:39:07 2016
From: python-checkins at python.org (victor.stinner)
Date: Mon, 21 Nov 2016 15:39:07 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42?=
Message-ID: <20161121153907.7241.14658.8BEE4A63@psf.io>

https://hg.python.org/cpython/rev/9c87c5632560
changeset:   105282:9c87c5632560
parent:      105280:acde821520fc
parent:      105281:5e8ef1493843
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Nov 21 16:39:01 2016 +0100
summary:
  Merge 3.6

files:
  Lib/test/test_re.py |  47 ++++++++++++++++++++-
  Misc/NEWS           |   5 ++
  Modules/_sre.c      |  73 ++++++++++++++++++++++++++++++--
  3 files changed, 117 insertions(+), 8 deletions(-)


diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py
--- a/Lib/test/test_re.py
+++ b/Lib/test/test_re.py
@@ -3,12 +3,13 @@
 import io
 import locale
 import re
-from re import Scanner
 import sre_compile
+import string
 import sys
-import string
 import traceback
 import unittest
+import warnings
+from re import Scanner
 from weakref import proxy
 
 # Misc tests from Tim Peters' re.doc
@@ -1777,6 +1778,48 @@
         self.assertIn('ASCII', str(re.A))
         self.assertIn('DOTALL', str(re.S))
 
+    def test_pattern_compare(self):
+        pattern1 = re.compile('abc', re.IGNORECASE)
+
+        # equal
+        re.purge()
+        pattern2 = re.compile('abc', re.IGNORECASE)
+        self.assertEqual(hash(pattern2), hash(pattern1))
+        self.assertEqual(pattern2, pattern1)
+
+        # not equal: different pattern
+        re.purge()
+        pattern3 = re.compile('XYZ', re.IGNORECASE)
+        # Don't test hash(pattern3) != hash(pattern1) because there is no
+        # warranty that hash values are different
+        self.assertNotEqual(pattern3, pattern1)
+
+        # not equal: different flag (flags=0)
+        re.purge()
+        pattern4 = re.compile('abc')
+        self.assertNotEqual(pattern4, pattern1)
+
+        # only == and != comparison operators are supported
+        with self.assertRaises(TypeError):
+            pattern1 < pattern2
+
+    def test_pattern_compare_bytes(self):
+        pattern1 = re.compile(b'abc')
+
+        # equal: test bytes patterns
+        re.purge()
+        pattern2 = re.compile(b'abc')
+        self.assertEqual(hash(pattern2), hash(pattern1))
+        self.assertEqual(pattern2, pattern1)
+
+        # not equal: pattern of a different types (str vs bytes),
+        # comparison must not raise a BytesWarning
+        re.purge()
+        pattern3 = re.compile('abc')
+        with warnings.catch_warnings():
+            warnings.simplefilter('error', BytesWarning)
+            self.assertNotEqual(pattern3, pattern1)
+
 
 class PatternReprTests(unittest.TestCase):
     def check(self, pattern, expected):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -133,6 +133,11 @@
 Library
 -------
 
+- Issue #28727: Regular expression patterns, _sre.SRE_Pattern objects created
+  by re.compile(), become comparable (only x==y and x!=y operators). This
+  change should fix the issue #18383: don't duplicate warning filters when the
+  warnings module is reloaded (thing usually only done in unit tests).
+
 - Issue #20572: Remove the subprocess.Popen.wait endtime parameter.  It was
   deprecated in 3.4 and undocumented prior to that.
 
diff --git a/Modules/_sre.c b/Modules/_sre.c
--- a/Modules/_sre.c
+++ b/Modules/_sre.c
@@ -1506,14 +1506,12 @@
 
     self->groups = groups;
 
-    Py_XINCREF(groupindex);
+    Py_INCREF(groupindex);
     self->groupindex = groupindex;
 
-    Py_XINCREF(indexgroup);
+    Py_INCREF(indexgroup);
     self->indexgroup = indexgroup;
 
-    self->weakreflist = NULL;
-
     if (!_validate(self)) {
         Py_DECREF(self);
         return NULL;
@@ -2649,6 +2647,69 @@
     return (PyObject*) scanner;
 }
 
+static Py_hash_t
+pattern_hash(PatternObject *self)
+{
+    Py_hash_t hash, hash2;
+
+    hash = PyObject_Hash(self->pattern);
+    if (hash == -1) {
+        return -1;
+    }
+
+    hash2 = _Py_HashBytes(self->code, sizeof(self->code[0]) * self->codesize);
+    hash ^= hash2;
+
+    hash ^= self->flags;
+    hash ^= self->isbytes;
+    hash ^= self->codesize;
+
+    if (hash == -1) {
+        hash = -2;
+    }
+    return hash;
+}
+
+static PyObject*
+pattern_richcompare(PyObject *lefto, PyObject *righto, int op)
+{
+    PatternObject *left, *right;
+    int cmp;
+
+    if (op != Py_EQ && op != Py_NE) {
+        Py_RETURN_NOTIMPLEMENTED;
+    }
+
+    if (Py_TYPE(lefto) != &Pattern_Type || Py_TYPE(righto) != &Pattern_Type) {
+        Py_RETURN_NOTIMPLEMENTED;
+    }
+    left = (PatternObject *)lefto;
+    right = (PatternObject *)righto;
+
+    cmp = (left->flags == right->flags
+           && left->isbytes == right->isbytes
+           && left->codesize && right->codesize);
+    if (cmp) {
+        /* Compare the code and the pattern because the same pattern can
+           produce different codes depending on the locale used to compile the
+           pattern when the re.LOCALE flag is used. Don't compare groups,
+           indexgroup nor groupindex: they are derivated from the pattern. */
+        cmp = (memcmp(left->code, right->code,
+                      sizeof(left->code[0]) * left->codesize) == 0);
+    }
+    if (cmp) {
+        cmp = PyObject_RichCompareBool(left->pattern, right->pattern,
+                                       Py_EQ);
+        if (cmp < 0) {
+            return NULL;
+        }
+    }
+    if (op == Py_NE) {
+        cmp = !cmp;
+    }
+    return PyBool_FromLong(cmp);
+}
+
 #include "clinic/_sre.c.h"
 
 static PyMethodDef pattern_methods[] = {
@@ -2693,7 +2754,7 @@
     0,                                  /* tp_as_number */
     0,                                  /* tp_as_sequence */
     0,                                  /* tp_as_mapping */
-    0,                                  /* tp_hash */
+    (hashfunc)pattern_hash,             /* tp_hash */
     0,                                  /* tp_call */
     0,                                  /* tp_str */
     0,                                  /* tp_getattro */
@@ -2703,7 +2764,7 @@
     pattern_doc,                        /* tp_doc */
     0,                                  /* tp_traverse */
     0,                                  /* tp_clear */
-    0,                                  /* tp_richcompare */
+    pattern_richcompare,                /* tp_richcompare */
     offsetof(PatternObject, weakreflist),       /* tp_weaklistoffset */
     0,                                  /* tp_iter */
     0,                                  /* tp_iternext */

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Mon Nov 21 10:43:36 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Mon, 21 Nov 2016 15:43:36 +0000
Subject: [Python-checkins] UGLY Benchmark Results for Python Default
 2016-11-21
Message-ID: <dc57af1b-7e82-422e-8c1b-7574536ec7c3@irsmsx151.ger.corp.intel.com>

Results for project Python default, build date 2016-11-21 03:03:19 +0000
commit:		30f154d9abf0
previous commit:	53f5e16695d0
revision date:	2016-11-21 02:01:37 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.18%          2.82%          7.36%            13.85%
:-)             pybench      0.20%          0.32%          6.05%             4.36%
:-|            regex_v8      3.91%          0.49%         -1.74%             2.66%
:-(               nbody      0.10%         -5.02%          0.76%             6.06%
:-(        json_dump_v2      0.27%         -2.06%         -8.53%            13.56%
:-|      normal_startup      1.05%          0.82%          0.60%             6.30%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-default-2016-11-21/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Mon Nov 21 10:46:26 2016
From: python-checkins at python.org (victor.stinner)
Date: Mon, 21 Nov 2016 15:46:26 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4Njg4?=
 =?utf-8?q?=3A_Remove_warnings=2Efilters_check_from_regrtest?=
Message-ID: <20161121154623.14486.5799.A0091CD0@psf.io>

https://hg.python.org/cpython/rev/75b1091594f8
changeset:   105283:75b1091594f8
branch:      3.5
parent:      105277:4d2888039631
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Nov 21 16:44:01 2016 +0100
summary:
  Issue #28688: Remove warnings.filters check from regrtest

Reloading the warnings module duplicates filters in warnings.filters. Fixing
the issue is tricky. It was decided to simply remove the check from Python 3.5,
since the bug only impacts Python unit tests, not real applications.

The check is kept in Python 3.6 and newer.

files:
  Lib/test/regrtest.py |  8 +-------
  1 files changed, 1 insertions(+), 7 deletions(-)


diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
--- a/Lib/test/regrtest.py
+++ b/Lib/test/regrtest.py
@@ -1052,7 +1052,7 @@
 
     resources = ('sys.argv', 'cwd', 'sys.stdin', 'sys.stdout', 'sys.stderr',
                  'os.environ', 'sys.path', 'sys.path_hooks', '__import__',
-                 'warnings.filters', 'asyncore.socket_map',
+                 'asyncore.socket_map',
                  'logging._handlers', 'logging._handlerList', 'sys.gettrace',
                  'sys.warnoptions',
                  # multiprocessing.process._cleanup() may release ref
@@ -1118,12 +1118,6 @@
     def restore___import__(self, import_):
         builtins.__import__ = import_
 
-    def get_warnings_filters(self):
-        return id(warnings.filters), warnings.filters, warnings.filters[:]
-    def restore_warnings_filters(self, saved_filters):
-        warnings.filters = saved_filters[1]
-        warnings.filters[:] = saved_filters[2]
-
     def get_asyncore_socket_map(self):
         asyncore = sys.modules.get('asyncore')
         # XXX Making a copy keeps objects alive until __exit__ gets called.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 10:46:26 2016
From: python-checkins at python.org (victor.stinner)
Date: Mon, 21 Nov 2016 15:46:26 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328688=3A_Null_merge_3=2E5?=
Message-ID: <20161121154624.14017.93630.54F74A99@psf.io>

https://hg.python.org/cpython/rev/a2616863de06
changeset:   105284:a2616863de06
branch:      3.6
parent:      105281:5e8ef1493843
parent:      105283:75b1091594f8
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Nov 21 16:45:19 2016 +0100
summary:
  Issue #28688: Null merge 3.5

The Python 3.5 change must not be applied to Python 3.6.

files:



-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 10:46:26 2016
From: python-checkins at python.org (victor.stinner)
Date: Mon, 21 Nov 2016 15:46:26 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328688=3A_Null_merge_3=2E6?=
Message-ID: <20161121154624.14252.19284.1A77D485@psf.io>

https://hg.python.org/cpython/rev/da042eec6743
changeset:   105285:da042eec6743
parent:      105282:9c87c5632560
parent:      105284:a2616863de06
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Nov 21 16:45:54 2016 +0100
summary:
  Issue #28688: Null merge 3.6

files:



-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 10:57:10 2016
From: python-checkins at python.org (inada.naoki)
Date: Mon, 21 Nov 2016 15:57:10 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328731=3A_Optimize?=
 =?utf-8?q?_=5FPyDict=5FNewPresized=28=29_to_create_correct_size_dict?=
Message-ID: <20161121155709.41629.36433.D87A2DBE@psf.io>

https://hg.python.org/cpython/rev/b708b3190ecb
changeset:   105286:b708b3190ecb
user:        INADA Naoki <songofacandy at gmail.com>
date:        Tue Nov 22 00:57:02 2016 +0900
summary:
  Issue #28731: Optimize _PyDict_NewPresized() to create correct size dict

Improve speed of dict literal with constant keys up to 30%.

files:
  Misc/NEWS            |   3 +++
  Objects/dictobject.c |  24 +++++++++++++++++++-----
  2 files changed, 22 insertions(+), 5 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #28731: Optimize _PyDict_NewPresized() to create correct size dict.
+  Improve speed of dict literal with constant keys up to 30%.
+
 - Issue #28532: Show sys.version when -V option is supplied twice.
 
 - Issue #28746: Fix the set_inheritable() file descriptor method on platforms
diff --git a/Objects/dictobject.c b/Objects/dictobject.c
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -389,7 +389,7 @@
  * This can be used to reserve enough size to insert n entries without
  * resizing.
  */
-#define ESTIMATE_SIZE(n)  (((n)*3) >> 1)
+#define ESTIMATE_SIZE(n)  (((n)*3+1) >> 1)
 
 /* Alternative fraction that is otherwise close enough to 2n/3 to make
  * little difference. 8 * 2/3 == 8 * 5/8 == 5. 16 * 2/3 == 16 * 5/8 == 10.
@@ -1361,12 +1361,26 @@
 PyObject *
 _PyDict_NewPresized(Py_ssize_t minused)
 {
+    const Py_ssize_t max_presize = 128 * 1024;
     Py_ssize_t newsize;
     PyDictKeysObject *new_keys;
-    for (newsize = PyDict_MINSIZE;
-         newsize <= minused && newsize > 0;
-         newsize <<= 1)
-        ;
+
+    /* There are no strict guarantee that returned dict can contain minused
+     * items without resize.  So we create medium size dict instead of very
+     * large dict or MemoryError.
+     */
+    if (minused > USABLE_FRACTION(max_presize)) {
+        newsize = max_presize;
+    }
+    else {
+        Py_ssize_t minsize = ESTIMATE_SIZE(minused);
+        newsize = PyDict_MINSIZE;
+        while (newsize < minsize) {
+            newsize <<= 1;
+        }
+    }
+    assert(IS_POWER_OF_2(newsize));
+
     new_keys = new_keys_object(newsize);
     if (new_keys == NULL)
         return NULL;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 11:31:12 2016
From: python-checkins at python.org (ethan.furman)
Date: Mon, 21 Nov 2016 16:31:12 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_closes_issue23?=
 =?utf-8?q?591=3A_add_NEWS_entry?=
Message-ID: <20161121163112.41827.84745.9953C7A9@psf.io>

https://hg.python.org/cpython/rev/f404a59d834e
changeset:   105287:f404a59d834e
branch:      3.6
parent:      105284:a2616863de06
user:        Ethan Furman <ethan at stoneleaf.us>
date:        Mon Nov 21 08:28:56 2016 -0800
summary:
  closes issue23591: add NEWS entry

files:
  Misc/NEWS |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -730,6 +730,8 @@
 - Issue #28025: Convert all ssl module constants to IntEnum and IntFlags.
   SSLContext properties now return flags and enums.
 
+- Issue #23591: Add Flag, IntFlag, and auto() to enum module.
+
 - Issue #433028: Added support of modifier spans in regular expressions.
 
 - Issue #24594: Validates persist parameter when opening MSI database

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 11:31:12 2016
From: python-checkins at python.org (ethan.furman)
Date: Mon, 21 Nov 2016 16:31:12 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_update_news_and_re_doc?=
Message-ID: <20161121163112.45206.2169.87517EAC@psf.io>

https://hg.python.org/cpython/rev/12b4e7209d03
changeset:   105289:12b4e7209d03
parent:      105286:b708b3190ecb
parent:      105288:176fc21f8430
user:        Ethan Furman <ethan at stoneleaf.us>
date:        Mon Nov 21 08:30:55 2016 -0800
summary:
  update news and re doc

files:
  Doc/library/re.rst |  3 +++
  Misc/NEWS          |  2 ++
  2 files changed, 5 insertions(+), 0 deletions(-)


diff --git a/Doc/library/re.rst b/Doc/library/re.rst
--- a/Doc/library/re.rst
+++ b/Doc/library/re.rst
@@ -478,6 +478,9 @@
 regular expressions.  Most non-trivial applications always use the compiled
 form.
 
+.. versionchanged:: 3.6
+   Flag constants are now instances of :class:`RegexFlag`, which is a subclass of
+   :class:`enum.IntFlag`.
 
 .. function:: compile(pattern, flags=0)
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -726,6 +726,8 @@
 - Issue #28025: Convert all ssl module constants to IntEnum and IntFlags.
   SSLContext properties now return flags and enums.
 
+- Issue #23591: Add Flag, IntFlag, and auto() to enum module.
+
 - Issue #433028: Added support of modifier spans in regular expressions.
 
 - Issue #24594: Validates persist parameter when opening MSI database

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 11:31:12 2016
From: python-checkins at python.org (ethan.furman)
Date: Mon, 21 Nov 2016 16:31:12 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_closes_issue28?=
 =?utf-8?q?082=3A_doc_update_and_NEWS_entry?=
Message-ID: <20161121163112.7660.68313.6D2FAA7E@psf.io>

https://hg.python.org/cpython/rev/176fc21f8430
changeset:   105288:176fc21f8430
branch:      3.6
user:        Ethan Furman <ethan at stoneleaf.us>
date:        Mon Nov 21 08:29:31 2016 -0800
summary:
  closes issue28082: doc update and NEWS entry

files:
  Doc/library/re.rst |  3 +++
  1 files changed, 3 insertions(+), 0 deletions(-)


diff --git a/Doc/library/re.rst b/Doc/library/re.rst
--- a/Doc/library/re.rst
+++ b/Doc/library/re.rst
@@ -478,6 +478,9 @@
 regular expressions.  Most non-trivial applications always use the compiled
 form.
 
+.. versionchanged:: 3.6
+   Flag constants are now instances of :class:`RegexFlag`, which is a subclass of
+   :class:`enum.IntFlag`.
 
 .. function:: compile(pattern, flags=0)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 11:40:22 2016
From: python-checkins at python.org (ethan.furman)
Date: Mon, 21 Nov 2016 16:40:22 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogaXNzdWUyODA4Mjog?=
 =?utf-8?q?actually_include_NEWS_entry?=
Message-ID: <20161121164021.5443.58927.4A20B0BA@psf.io>

https://hg.python.org/cpython/rev/493359386360
changeset:   105290:493359386360
branch:      3.6
parent:      105288:176fc21f8430
user:        Ethan Furman <ethan at stoneleaf.us>
date:        Mon Nov 21 08:39:32 2016 -0800
summary:
  issue28082: actually include NEWS entry

files:
  Misc/NEWS |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -727,6 +727,8 @@
 - Issue #14977: mailcap now respects the order of the lines in the mailcap
   files ("first match"), as required by RFC 1542.  Patch by Michael Lazar.
 
+- Issue #28082: Convert re flag constants to IntFlag.
+
 - Issue #28025: Convert all ssl module constants to IntEnum and IntFlags.
   SSLContext properties now return flags and enums.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 11:40:22 2016
From: python-checkins at python.org (ethan.furman)
Date: Mon, 21 Nov 2016 16:40:22 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_NEWS_update?=
Message-ID: <20161121164021.14116.81932.3DF24943@psf.io>

https://hg.python.org/cpython/rev/c19383fad567
changeset:   105291:c19383fad567
parent:      105289:12b4e7209d03
parent:      105290:493359386360
user:        Ethan Furman <ethan at stoneleaf.us>
date:        Mon Nov 21 08:39:57 2016 -0800
summary:
  NEWS update

files:
  Misc/NEWS |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -723,6 +723,8 @@
 - Issue #14977: mailcap now respects the order of the lines in the mailcap
   files ("first match"), as required by RFC 1542.  Patch by Michael Lazar.
 
+- Issue #28082: Convert re flag constants to IntFlag.
+
 - Issue #28025: Convert all ssl module constants to IntEnum and IntFlags.
   SSLContext properties now return flags and enums.
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 12:22:55 2016
From: python-checkins at python.org (ethan.furman)
Date: Mon, 21 Nov 2016 17:22:55 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_close_issue281?=
 =?utf-8?q?72=3A_Change_all_example_enum_member_names_to_uppercase=2C_per_?=
 =?utf-8?q?Guido=3B?=
Message-ID: <20161121172254.7770.90585.F2FB5DA5@psf.io>

https://hg.python.org/cpython/rev/b9801dab214a
changeset:   105292:b9801dab214a
branch:      3.6
parent:      105290:493359386360
user:        Ethan Furman <ethan at stoneleaf.us>
date:        Mon Nov 21 09:22:05 2016 -0800
summary:
  close issue28172: Change all example enum member names to uppercase, per Guido; patch by Chris Angelico.

files:
  Doc/library/enum.rst  |  345 +++++++++++++++--------------
  Lib/enum.py           |    4 +-
  Lib/test/test_enum.py |    6 +-
  3 files changed, 178 insertions(+), 177 deletions(-)


diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst
--- a/Doc/library/enum.rst
+++ b/Doc/library/enum.rst
@@ -70,9 +70,9 @@
 
     >>> from enum import Enum
     >>> class Color(Enum):
-    ...     red = 1
-    ...     green = 2
-    ...     blue = 3
+    ...     RED = 1
+    ...     GREEN = 2
+    ...     BLUE = 3
     ...
 
 .. note:: Enum member values
@@ -85,10 +85,10 @@
 .. note:: Nomenclature
 
   - The class :class:`Color` is an *enumeration* (or *enum*)
-  - The attributes :attr:`Color.red`, :attr:`Color.green`, etc., are
-    *enumeration members* (or *enum members*).
+  - The attributes :attr:`Color.RED`, :attr:`Color.GREEN`, etc., are
+    *enumeration members* (or *enum members*) and are functionally constants.
   - The enum members have *names* and *values* (the name of
-    :attr:`Color.red` is ``red``, the value of :attr:`Color.blue` is
+    :attr:`Color.RED` is ``RED``, the value of :attr:`Color.BLUE` is
     ``3``, etc.)
 
 .. note::
@@ -99,49 +99,49 @@
 
 Enumeration members have human readable string representations::
 
-    >>> print(Color.red)
-    Color.red
+    >>> print(Color.RED)
+    Color.RED
 
 ...while their ``repr`` has more information::
 
-    >>> print(repr(Color.red))
-    <Color.red: 1>
+    >>> print(repr(Color.RED))
+    <Color.RED: 1>
 
 The *type* of an enumeration member is the enumeration it belongs to::
 
-    >>> type(Color.red)
+    >>> type(Color.RED)
     <enum 'Color'>
-    >>> isinstance(Color.green, Color)
+    >>> isinstance(Color.GREEN, Color)
     True
     >>>
 
 Enum members also have a property that contains just their item name::
 
-    >>> print(Color.red.name)
-    red
+    >>> print(Color.RED.name)
+    RED
 
 Enumerations support iteration, in definition order::
 
     >>> class Shake(Enum):
-    ...     vanilla = 7
-    ...     chocolate = 4
-    ...     cookies = 9
-    ...     mint = 3
+    ...     VANILLA = 7
+    ...     CHOCOLATE = 4
+    ...     COOKIES = 9
+    ...     MINT = 3
     ...
     >>> for shake in Shake:
     ...     print(shake)
     ...
-    Shake.vanilla
-    Shake.chocolate
-    Shake.cookies
-    Shake.mint
+    Shake.VANILLA
+    Shake.CHOCOLATE
+    Shake.COOKIES
+    Shake.MINT
 
 Enumeration members are hashable, so they can be used in dictionaries and sets::
 
     >>> apples = {}
-    >>> apples[Color.red] = 'red delicious'
-    >>> apples[Color.green] = 'granny smith'
-    >>> apples == {Color.red: 'red delicious', Color.green: 'granny smith'}
+    >>> apples[Color.RED] = 'red delicious'
+    >>> apples[Color.GREEN] = 'granny smith'
+    >>> apples == {Color.RED: 'red delicious', Color.GREEN: 'granny smith'}
     True
 
 
@@ -149,26 +149,26 @@
 ---------------------------------------------------------------
 
 Sometimes it's useful to access members in enumerations programmatically (i.e.
-situations where ``Color.red`` won't do because the exact color is not known
+situations where ``Color.RED`` won't do because the exact color is not known
 at program-writing time).  ``Enum`` allows such access::
 
     >>> Color(1)
-    <Color.red: 1>
+    <Color.RED: 1>
     >>> Color(3)
-    <Color.blue: 3>
+    <Color.BLUE: 3>
 
 If you want to access enum members by *name*, use item access::
 
-    >>> Color['red']
-    <Color.red: 1>
-    >>> Color['green']
-    <Color.green: 2>
+    >>> Color['RED']
+    <Color.RED: 1>
+    >>> Color['GREEN']
+    <Color.GREEN: 2>
 
 If you have an enum member and need its :attr:`name` or :attr:`value`::
 
-    >>> member = Color.red
+    >>> member = Color.RED
     >>> member.name
-    'red'
+    'RED'
     >>> member.value
     1
 
@@ -179,12 +179,12 @@
 Having two enum members with the same name is invalid::
 
     >>> class Shape(Enum):
-    ...     square = 2
-    ...     square = 3
+    ...     SQUARE = 2
+    ...     SQUARE = 3
     ...
     Traceback (most recent call last):
     ...
-    TypeError: Attempted to reuse key: 'square'
+    TypeError: Attempted to reuse key: 'SQUARE'
 
 However, two enum members are allowed to have the same value.  Given two members
 A and B with the same value (and A defined first), B is an alias to A.  By-value
@@ -192,17 +192,17 @@
 return A::
 
     >>> class Shape(Enum):
-    ...     square = 2
-    ...     diamond = 1
-    ...     circle = 3
-    ...     alias_for_square = 2
+    ...     SQUARE = 2
+    ...     DIAMOND = 1
+    ...     CIRCLE = 3
+    ...     ALIAS_FOR_SQUARE = 2
     ...
-    >>> Shape.square
-    <Shape.square: 2>
-    >>> Shape.alias_for_square
-    <Shape.square: 2>
+    >>> Shape.SQUARE
+    <Shape.SQUARE: 2>
+    >>> Shape.ALIAS_FOR_SQUARE
+    <Shape.SQUARE: 2>
     >>> Shape(2)
-    <Shape.square: 2>
+    <Shape.SQUARE: 2>
 
 .. note::
 
@@ -227,14 +227,14 @@
     >>> from enum import Enum, unique
     >>> @unique
     ... class Mistake(Enum):
-    ...     one = 1
-    ...     two = 2
-    ...     three = 3
-    ...     four = 3
+    ...     ONE = 1
+    ...     TWO = 2
+    ...     THREE = 3
+    ...     FOUR = 3
     ...
     Traceback (most recent call last):
     ...
-    ValueError: duplicate values found in <enum 'Mistake'>: four -> three
+    ValueError: duplicate values found in <enum 'Mistake'>: FOUR -> THREE
 
 
 Using automatic values
@@ -244,12 +244,12 @@
 
     >>> from enum import Enum, auto
     >>> class Color(Enum):
-    ...     red = auto()
-    ...     blue = auto()
-    ...     green = auto()
+    ...     RED = auto()
+    ...     BLUE = auto()
+    ...     GREEN = auto()
     ...
     >>> list(Color)
-    [<Color.red: 1>, <Color.blue: 2>, <Color.green: 3>]
+    [<Color.RED: 1>, <Color.BLUE: 2>, <Color.GREEN: 3>]
 
 The values are chosen by :func:`_generate_next_value_`, which can be
 overridden::
@@ -259,13 +259,13 @@
     ...         return name
     ...
     >>> class Ordinal(AutoName):
-    ...     north = auto()
-    ...     south = auto()
-    ...     east = auto()
-    ...     west = auto()
+    ...     NORTH = auto()
+    ...     SOUTH = auto()
+    ...     EAST = auto()
+    ...     WEST = auto()
     ...
     >>> list(Ordinal)
-    [<Ordinal.north: 'north'>, <Ordinal.south: 'south'>, <Ordinal.east: 'east'>, <Ordinal.west: 'west'>]
+    [<Ordinal.NORTH: 'NORTH'>, <Ordinal.SOUTH: 'SOUTH'>, <Ordinal.EAST: 'EAST'>, <Ordinal.WEST: 'WEST'>]
 
 .. note::
 
@@ -279,7 +279,7 @@
 Iterating over the members of an enum does not provide the aliases::
 
     >>> list(Shape)
-    [<Shape.square: 2>, <Shape.diamond: 1>, <Shape.circle: 3>]
+    [<Shape.SQUARE: 2>, <Shape.DIAMOND: 1>, <Shape.CIRCLE: 3>]
 
 The special attribute ``__members__`` is an ordered dictionary mapping names
 to members.  It includes all names defined in the enumeration, including the
@@ -288,16 +288,16 @@
     >>> for name, member in Shape.__members__.items():
     ...     name, member
     ...
-    ('square', <Shape.square: 2>)
-    ('diamond', <Shape.diamond: 1>)
-    ('circle', <Shape.circle: 3>)
-    ('alias_for_square', <Shape.square: 2>)
+    ('SQUARE', <Shape.SQUARE: 2>)
+    ('DIAMOND', <Shape.DIAMOND: 1>)
+    ('CIRCLE', <Shape.CIRCLE: 3>)
+    ('ALIAS_FOR_SQUARE', <Shape.SQUARE: 2>)
 
 The ``__members__`` attribute can be used for detailed programmatic access to
 the enumeration members.  For example, finding all the aliases::
 
     >>> [name for name, member in Shape.__members__.items() if member.name != name]
-    ['alias_for_square']
+    ['ALIAS_FOR_SQUARE']
 
 
 Comparisons
@@ -305,35 +305,35 @@
 
 Enumeration members are compared by identity::
 
-    >>> Color.red is Color.red
+    >>> Color.RED is Color.RED
     True
-    >>> Color.red is Color.blue
+    >>> Color.RED is Color.BLUE
     False
-    >>> Color.red is not Color.blue
+    >>> Color.RED is not Color.BLUE
     True
 
 Ordered comparisons between enumeration values are *not* supported.  Enum
 members are not integers (but see `IntEnum`_ below)::
 
-    >>> Color.red < Color.blue
+    >>> Color.RED < Color.BLUE
     Traceback (most recent call last):
       File "<stdin>", line 1, in <module>
     TypeError: '<' not supported between instances of 'Color' and 'Color'
 
 Equality comparisons are defined though::
 
-    >>> Color.blue == Color.red
+    >>> Color.BLUE == Color.RED
     False
-    >>> Color.blue != Color.red
+    >>> Color.BLUE != Color.RED
     True
-    >>> Color.blue == Color.blue
+    >>> Color.BLUE == Color.BLUE
     True
 
 Comparisons against non-enumeration values will always compare not equal
 (again, :class:`IntEnum` was explicitly designed to behave differently, see
 below)::
 
-    >>> Color.blue == 2
+    >>> Color.BLUE == 2
     False
 
 
@@ -350,8 +350,8 @@
 usual.  If we have this enumeration::
 
     >>> class Mood(Enum):
-    ...     funky = 1
-    ...     happy = 3
+    ...     FUNKY = 1
+    ...     HAPPY = 3
     ...
     ...     def describe(self):
     ...         # self is the member here
@@ -363,16 +363,16 @@
     ...     @classmethod
     ...     def favorite_mood(cls):
     ...         # cls here is the enumeration
-    ...         return cls.happy
+    ...         return cls.HAPPY
     ...
 
 Then::
 
     >>> Mood.favorite_mood()
-    <Mood.happy: 3>
-    >>> Mood.happy.describe()
-    ('happy', 3)
-    >>> str(Mood.funky)
+    <Mood.HAPPY: 3>
+    >>> Mood.HAPPY.describe()
+    ('HAPPY', 3)
+    >>> str(Mood.FUNKY)
     'my custom str! 1'
 
 The rules for what is allowed are as follows: names that start and end with
@@ -393,7 +393,7 @@
 any members.  So this is forbidden::
 
     >>> class MoreColor(Color):
-    ...     pink = 17
+    ...     PINK = 17
     ...
     Traceback (most recent call last):
     ...
@@ -406,8 +406,8 @@
     ...         pass
     ...
     >>> class Bar(Foo):
-    ...     happy = 1
-    ...     sad = 2
+    ...     HAPPY = 1
+    ...     SAD = 2
     ...
 
 Allowing subclassing of enums that define members would lead to a violation of
@@ -423,7 +423,7 @@
 
     >>> from test.test_enum import Fruit
     >>> from pickle import dumps, loads
-    >>> Fruit.tomato is loads(dumps(Fruit.tomato))
+    >>> Fruit.TOMATO is loads(dumps(Fruit.TOMATO))
     True
 
 The usual restrictions for pickling apply: picklable enums must be defined in
@@ -444,15 +444,15 @@
 
 The :class:`Enum` class is callable, providing the following functional API::
 
-    >>> Animal = Enum('Animal', 'ant bee cat dog')
+    >>> Animal = Enum('Animal', 'ANT BEE CAT DOG')
     >>> Animal
     <enum 'Animal'>
-    >>> Animal.ant
-    <Animal.ant: 1>
-    >>> Animal.ant.value
+    >>> Animal.ANT
+    <Animal.ANT: 1>
+    >>> Animal.ANT.value
     1
     >>> list(Animal)
-    [<Animal.ant: 1>, <Animal.bee: 2>, <Animal.cat: 3>, <Animal.dog: 4>]
+    [<Animal.ANT: 1>, <Animal.BEE: 2>, <Animal.CAT: 3>, <Animal.DOG: 4>]
 
 The semantics of this API resemble :class:`~collections.namedtuple`. The first
 argument of the call to :class:`Enum` is the name of the enumeration.
@@ -467,10 +467,10 @@
 assignment to :class:`Animal` is equivalent to::
 
     >>> class Animal(Enum):
-    ...     ant = 1
-    ...     bee = 2
-    ...     cat = 3
-    ...     dog = 4
+    ...     ANT = 1
+    ...     BEE = 2
+    ...     CAT = 3
+    ...     DOG = 4
     ...
 
 The reason for defaulting to ``1`` as the starting number and not ``0`` is
@@ -483,7 +483,7 @@
 function in separate module, and also may not work on IronPython or Jython).
 The solution is to specify the module name explicitly as follows::
 
-    >>> Animal = Enum('Animal', 'ant bee cat dog', module=__name__)
+    >>> Animal = Enum('Animal', 'ANT BEE CAT DOG', module=__name__)
 
 .. warning::
 
@@ -496,7 +496,7 @@
 to find the class.  For example, if the class was made available in class
 SomeData in the global scope::
 
-    >>> Animal = Enum('Animal', 'ant bee cat dog', qualname='SomeData.Animal')
+    >>> Animal = Enum('Animal', 'ANT BEE CAT DOG', qualname='SomeData.Animal')
 
 The complete signature is::
 
@@ -507,19 +507,19 @@
 :names: The Enum members.  This can be a whitespace or comma separated string
   (values will start at 1 unless otherwise specified)::
 
-    'red green blue' | 'red,green,blue' | 'red, green, blue'
+    'RED GREEN BLUE' | 'RED,GREEN,BLUE' | 'RED, GREEN, BLUE'
 
   or an iterator of names::
 
-    ['red', 'green', 'blue']
+    ['RED', 'GREEN', 'BLUE']
 
   or an iterator of (name, value) pairs::
 
-    [('cyan', 4), ('magenta', 5), ('yellow', 6)]
+    [('CYAN', 4), ('MAGENTA', 5), ('YELLOW', 6)]
 
   or a mapping::
 
-    {'chartreuse': 7, 'sea_green': 11, 'rosemary': 42}
+    {'CHARTREUSE': 7, 'SEA_GREEN': 11, 'ROSEMARY': 42}
 
 :module: name of module where new Enum class can be found.
 
@@ -546,40 +546,40 @@
 
     >>> from enum import IntEnum
     >>> class Shape(IntEnum):
-    ...     circle = 1
-    ...     square = 2
+    ...     CIRCLE = 1
+    ...     SQUARE = 2
     ...
     >>> class Request(IntEnum):
-    ...     post = 1
-    ...     get = 2
+    ...     POST = 1
+    ...     GET = 2
     ...
     >>> Shape == 1
     False
-    >>> Shape.circle == 1
+    >>> Shape.CIRCLE == 1
     True
-    >>> Shape.circle == Request.post
+    >>> Shape.CIRCLE == Request.POST
     True
 
 However, they still can't be compared to standard :class:`Enum` enumerations::
 
     >>> class Shape(IntEnum):
-    ...     circle = 1
-    ...     square = 2
+    ...     CIRCLE = 1
+    ...     SQUARE = 2
     ...
     >>> class Color(Enum):
-    ...     red = 1
-    ...     green = 2
+    ...     RED = 1
+    ...     GREEN = 2
     ...
-    >>> Shape.circle == Color.red
+    >>> Shape.CIRCLE == Color.RED
     False
 
 :class:`IntEnum` values behave like integers in other ways you'd expect::
 
-    >>> int(Shape.circle)
+    >>> int(Shape.CIRCLE)
     1
-    >>> ['a', 'b', 'c'][Shape.circle]
+    >>> ['a', 'b', 'c'][Shape.CIRCLE]
     'b'
-    >>> [i for i in range(Shape.square)]
+    >>> [i for i in range(Shape.SQUARE)]
     [0, 1]
 
 
@@ -656,39 +656,39 @@
 
     >>> from enum import Flag
     >>> class Color(Flag):
-    ...     red = auto()
-    ...     blue = auto()
-    ...     green = auto()
+    ...     RED = auto()
+    ...     BLUE = auto()
+    ...     GREEN = auto()
     ...
-    >>> Color.red & Color.green
+    >>> Color.RED & Color.GREEN
     <Color.0: 0>
-    >>> bool(Color.red & Color.green)
+    >>> bool(Color.RED & Color.GREEN)
     False
 
 Individual flags should have values that are powers of two (1, 2, 4, 8, ...),
 while combinations of flags won't::
 
     >>> class Color(Flag):
-    ...     red = auto()
-    ...     blue = auto()
-    ...     green = auto()
-    ...     white = red | blue | green
+    ...     RED = auto()
+    ...     BLUE = auto()
+    ...     GREEN = auto()
+    ...     WHITE = RED | BLUE | GREEN
     ...
-    >>> Color.white
-    <Color.white: 7>
+    >>> Color.WHITE
+    <Color.WHITE: 7>
 
 Giving a name to the "no flags set" condition does not change its boolean
 value::
 
     >>> class Color(Flag):
-    ...     black = 0
-    ...     red = auto()
-    ...     blue = auto()
-    ...     green = auto()
+    ...     BLACK = 0
+    ...     RED = auto()
+    ...     BLUE = auto()
+    ...     GREEN = auto()
     ...
-    >>> Color.black
-    <Color.black: 0>
-    >>> bool(Color.black)
+    >>> Color.BLACK
+    <Color.BLACK: 0>
+    >>> bool(Color.BLACK)
     False
 
 .. note::
@@ -776,12 +776,12 @@
 Using :class:`object` would look like::
 
     >>> class Color(NoValue):
-    ...     red = auto()
-    ...     blue = auto()
-    ...     green = auto()
+    ...     RED = auto()
+    ...     BLUE = auto()
+    ...     GREEN = auto()
     ...
-    >>> Color.green
-    <Color.green>
+    >>> Color.GREEN
+    <Color.GREEN>
 
 
 Using :class:`object`
@@ -790,12 +790,12 @@
 Using :class:`object` would look like::
 
     >>> class Color(NoValue):
-    ...     red = object()
-    ...     green = object()
-    ...     blue = object()
+    ...     RED = object()
+    ...     GREEN = object()
+    ...     BLUE = object()
     ...
-    >>> Color.green
-    <Color.green>
+    >>> Color.GREEN
+    <Color.GREEN>
 
 
 Using a descriptive string
@@ -804,13 +804,13 @@
 Using a string as the value would look like::
 
     >>> class Color(NoValue):
-    ...     red = 'stop'
-    ...     green = 'go'
-    ...     blue = 'too fast!'
+    ...     RED = 'stop'
+    ...     GREEN = 'go'
+    ...     BLUE = 'too fast!'
     ...
-    >>> Color.green
-    <Color.green>
-    >>> Color.green.value
+    >>> Color.GREEN
+    <Color.GREEN>
+    >>> Color.GREEN.value
     'go'
 
 
@@ -827,13 +827,13 @@
     ...         return obj
     ...
     >>> class Color(AutoNumber):
-    ...     red = ()
-    ...     green = ()
-    ...     blue = ()
+    ...     RED = ()
+    ...     GREEN = ()
+    ...     BLUE = ()
     ...
-    >>> Color.green
-    <Color.green>
-    >>> Color.green.value
+    >>> Color.GREEN
+    <Color.GREEN>
+    >>> Color.GREEN.value
     2
 
 
@@ -897,14 +897,14 @@
     ...                 % (a, e))
     ...
     >>> class Color(DuplicateFreeEnum):
-    ...     red = 1
-    ...     green = 2
-    ...     blue = 3
-    ...     grene = 2
+    ...     RED = 1
+    ...     GREEN = 2
+    ...     BLUE = 3
+    ...     GRENE = 2
     ...
     Traceback (most recent call last):
     ...
-    ValueError: aliases not allowed in DuplicateFreeEnum:  'grene' --> 'green'
+    ValueError: aliases not allowed in DuplicateFreeEnum:  'GRENE' --> 'GREEN'
 
 .. note::
 
@@ -1007,10 +1007,10 @@
 and raise an error if the two do not match::
 
     >>> class Color(Enum):
-    ...     _order_ = 'red green blue'
-    ...     red = 1
-    ...     blue = 3
-    ...     green = 2
+    ...     _order_ = 'RED GREEN BLUE'
+    ...     RED = 1
+    ...     BLUE = 3
+    ...     GREEN = 2
     ...
     Traceback (most recent call last):
     ...
@@ -1028,7 +1028,8 @@
 normally accessed as ``EnumClass.member``.  Under certain circumstances they
 can also be accessed as ``EnumClass.member.member``, but you should never do
 this as that lookup may fail or, worse, return something besides the
-:class:`Enum` member you are looking for::
+:class:`Enum` member you are looking for (this is another good reason to use
+all-uppercase names for members)::
 
     >>> class FieldTypes(Enum):
     ...     name = 0
@@ -1078,15 +1079,15 @@
 all named flags and all named combinations of flags that are in the value::
 
     >>> class Color(Flag):
-    ...     red = auto()
-    ...     green = auto()
-    ...     blue = auto()
-    ...     magenta = red | blue
-    ...     yellow = red | green
-    ...     cyan = green | blue
+    ...     RED = auto()
+    ...     GREEN = auto()
+    ...     BLUE = auto()
+    ...     MAGENTA = RED | BLUE
+    ...     YELLOW = RED | GREEN
+    ...     CYAN = GREEN | BLUE
     ...
     >>> Color(3)  # named combination
-    <Color.yellow: 3>
+    <Color.YELLOW: 3>
     >>> Color(7)      # not named combination
-    <Color.cyan|magenta|blue|yellow|green|red: 7>
+    <Color.CYAN|MAGENTA|BLUE|YELLOW|GREEN|RED: 7>
 
diff --git a/Lib/enum.py b/Lib/enum.py
--- a/Lib/enum.py
+++ b/Lib/enum.py
@@ -267,7 +267,7 @@
 
         This method is used both when an enum class is given a value to match
         to an enumeration member (i.e. Color(3)) and for the functional API
-        (i.e. Color = Enum('Color', names='red green blue')).
+        (i.e. Color = Enum('Color', names='RED GREEN BLUE')).
 
         When used for the functional API:
 
@@ -517,7 +517,7 @@
         # without calling this method; this method is called by the metaclass'
         # __call__ (i.e. Color(3) ), and by pickle
         if type(value) is cls:
-            # For lookups like Color(Color.red)
+            # For lookups like Color(Color.RED)
             return value
         # by-value search for a matching enum member
         # see if it's in the reverse mapping (for hashable values)
diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py
--- a/Lib/test/test_enum.py
+++ b/Lib/test/test_enum.py
@@ -69,9 +69,9 @@
 # for doctests
 try:
     class Fruit(Enum):
-        tomato = 1
-        banana = 2
-        cherry = 3
+        TOMATO = 1
+        BANANA = 2
+        CHERRY = 3
 except Exception:
     pass
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 12:22:55 2016
From: python-checkins at python.org (ethan.furman)
Date: Mon, 21 Nov 2016 17:22:55 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_close_issue28172=3A_Change_all_example_enum_member_names?=
 =?utf-8?q?_to_uppercase=2C_per_Guido=3B?=
Message-ID: <20161121172254.45413.2078.C3B5E836@psf.io>

https://hg.python.org/cpython/rev/5ec8d4c51363
changeset:   105293:5ec8d4c51363
parent:      105291:c19383fad567
parent:      105292:b9801dab214a
user:        Ethan Furman <ethan at stoneleaf.us>
date:        Mon Nov 21 09:22:40 2016 -0800
summary:
  close issue28172: Change all example enum member names to uppercase, per Guido; patch by Chris Angelico.

files:
  Doc/library/enum.rst  |  345 +++++++++++++++--------------
  Lib/enum.py           |    4 +-
  Lib/test/test_enum.py |    6 +-
  3 files changed, 178 insertions(+), 177 deletions(-)


diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst
--- a/Doc/library/enum.rst
+++ b/Doc/library/enum.rst
@@ -70,9 +70,9 @@
 
     >>> from enum import Enum
     >>> class Color(Enum):
-    ...     red = 1
-    ...     green = 2
-    ...     blue = 3
+    ...     RED = 1
+    ...     GREEN = 2
+    ...     BLUE = 3
     ...
 
 .. note:: Enum member values
@@ -85,10 +85,10 @@
 .. note:: Nomenclature
 
   - The class :class:`Color` is an *enumeration* (or *enum*)
-  - The attributes :attr:`Color.red`, :attr:`Color.green`, etc., are
-    *enumeration members* (or *enum members*).
+  - The attributes :attr:`Color.RED`, :attr:`Color.GREEN`, etc., are
+    *enumeration members* (or *enum members*) and are functionally constants.
   - The enum members have *names* and *values* (the name of
-    :attr:`Color.red` is ``red``, the value of :attr:`Color.blue` is
+    :attr:`Color.RED` is ``RED``, the value of :attr:`Color.BLUE` is
     ``3``, etc.)
 
 .. note::
@@ -99,49 +99,49 @@
 
 Enumeration members have human readable string representations::
 
-    >>> print(Color.red)
-    Color.red
+    >>> print(Color.RED)
+    Color.RED
 
 ...while their ``repr`` has more information::
 
-    >>> print(repr(Color.red))
-    <Color.red: 1>
+    >>> print(repr(Color.RED))
+    <Color.RED: 1>
 
 The *type* of an enumeration member is the enumeration it belongs to::
 
-    >>> type(Color.red)
+    >>> type(Color.RED)
     <enum 'Color'>
-    >>> isinstance(Color.green, Color)
+    >>> isinstance(Color.GREEN, Color)
     True
     >>>
 
 Enum members also have a property that contains just their item name::
 
-    >>> print(Color.red.name)
-    red
+    >>> print(Color.RED.name)
+    RED
 
 Enumerations support iteration, in definition order::
 
     >>> class Shake(Enum):
-    ...     vanilla = 7
-    ...     chocolate = 4
-    ...     cookies = 9
-    ...     mint = 3
+    ...     VANILLA = 7
+    ...     CHOCOLATE = 4
+    ...     COOKIES = 9
+    ...     MINT = 3
     ...
     >>> for shake in Shake:
     ...     print(shake)
     ...
-    Shake.vanilla
-    Shake.chocolate
-    Shake.cookies
-    Shake.mint
+    Shake.VANILLA
+    Shake.CHOCOLATE
+    Shake.COOKIES
+    Shake.MINT
 
 Enumeration members are hashable, so they can be used in dictionaries and sets::
 
     >>> apples = {}
-    >>> apples[Color.red] = 'red delicious'
-    >>> apples[Color.green] = 'granny smith'
-    >>> apples == {Color.red: 'red delicious', Color.green: 'granny smith'}
+    >>> apples[Color.RED] = 'red delicious'
+    >>> apples[Color.GREEN] = 'granny smith'
+    >>> apples == {Color.RED: 'red delicious', Color.GREEN: 'granny smith'}
     True
 
 
@@ -149,26 +149,26 @@
 ---------------------------------------------------------------
 
 Sometimes it's useful to access members in enumerations programmatically (i.e.
-situations where ``Color.red`` won't do because the exact color is not known
+situations where ``Color.RED`` won't do because the exact color is not known
 at program-writing time).  ``Enum`` allows such access::
 
     >>> Color(1)
-    <Color.red: 1>
+    <Color.RED: 1>
     >>> Color(3)
-    <Color.blue: 3>
+    <Color.BLUE: 3>
 
 If you want to access enum members by *name*, use item access::
 
-    >>> Color['red']
-    <Color.red: 1>
-    >>> Color['green']
-    <Color.green: 2>
+    >>> Color['RED']
+    <Color.RED: 1>
+    >>> Color['GREEN']
+    <Color.GREEN: 2>
 
 If you have an enum member and need its :attr:`name` or :attr:`value`::
 
-    >>> member = Color.red
+    >>> member = Color.RED
     >>> member.name
-    'red'
+    'RED'
     >>> member.value
     1
 
@@ -179,12 +179,12 @@
 Having two enum members with the same name is invalid::
 
     >>> class Shape(Enum):
-    ...     square = 2
-    ...     square = 3
+    ...     SQUARE = 2
+    ...     SQUARE = 3
     ...
     Traceback (most recent call last):
     ...
-    TypeError: Attempted to reuse key: 'square'
+    TypeError: Attempted to reuse key: 'SQUARE'
 
 However, two enum members are allowed to have the same value.  Given two members
 A and B with the same value (and A defined first), B is an alias to A.  By-value
@@ -192,17 +192,17 @@
 return A::
 
     >>> class Shape(Enum):
-    ...     square = 2
-    ...     diamond = 1
-    ...     circle = 3
-    ...     alias_for_square = 2
+    ...     SQUARE = 2
+    ...     DIAMOND = 1
+    ...     CIRCLE = 3
+    ...     ALIAS_FOR_SQUARE = 2
     ...
-    >>> Shape.square
-    <Shape.square: 2>
-    >>> Shape.alias_for_square
-    <Shape.square: 2>
+    >>> Shape.SQUARE
+    <Shape.SQUARE: 2>
+    >>> Shape.ALIAS_FOR_SQUARE
+    <Shape.SQUARE: 2>
     >>> Shape(2)
-    <Shape.square: 2>
+    <Shape.SQUARE: 2>
 
 .. note::
 
@@ -227,14 +227,14 @@
     >>> from enum import Enum, unique
     >>> @unique
     ... class Mistake(Enum):
-    ...     one = 1
-    ...     two = 2
-    ...     three = 3
-    ...     four = 3
+    ...     ONE = 1
+    ...     TWO = 2
+    ...     THREE = 3
+    ...     FOUR = 3
     ...
     Traceback (most recent call last):
     ...
-    ValueError: duplicate values found in <enum 'Mistake'>: four -> three
+    ValueError: duplicate values found in <enum 'Mistake'>: FOUR -> THREE
 
 
 Using automatic values
@@ -244,12 +244,12 @@
 
     >>> from enum import Enum, auto
     >>> class Color(Enum):
-    ...     red = auto()
-    ...     blue = auto()
-    ...     green = auto()
+    ...     RED = auto()
+    ...     BLUE = auto()
+    ...     GREEN = auto()
     ...
     >>> list(Color)
-    [<Color.red: 1>, <Color.blue: 2>, <Color.green: 3>]
+    [<Color.RED: 1>, <Color.BLUE: 2>, <Color.GREEN: 3>]
 
 The values are chosen by :func:`_generate_next_value_`, which can be
 overridden::
@@ -259,13 +259,13 @@
     ...         return name
     ...
     >>> class Ordinal(AutoName):
-    ...     north = auto()
-    ...     south = auto()
-    ...     east = auto()
-    ...     west = auto()
+    ...     NORTH = auto()
+    ...     SOUTH = auto()
+    ...     EAST = auto()
+    ...     WEST = auto()
     ...
     >>> list(Ordinal)
-    [<Ordinal.north: 'north'>, <Ordinal.south: 'south'>, <Ordinal.east: 'east'>, <Ordinal.west: 'west'>]
+    [<Ordinal.NORTH: 'NORTH'>, <Ordinal.SOUTH: 'SOUTH'>, <Ordinal.EAST: 'EAST'>, <Ordinal.WEST: 'WEST'>]
 
 .. note::
 
@@ -279,7 +279,7 @@
 Iterating over the members of an enum does not provide the aliases::
 
     >>> list(Shape)
-    [<Shape.square: 2>, <Shape.diamond: 1>, <Shape.circle: 3>]
+    [<Shape.SQUARE: 2>, <Shape.DIAMOND: 1>, <Shape.CIRCLE: 3>]
 
 The special attribute ``__members__`` is an ordered dictionary mapping names
 to members.  It includes all names defined in the enumeration, including the
@@ -288,16 +288,16 @@
     >>> for name, member in Shape.__members__.items():
     ...     name, member
     ...
-    ('square', <Shape.square: 2>)
-    ('diamond', <Shape.diamond: 1>)
-    ('circle', <Shape.circle: 3>)
-    ('alias_for_square', <Shape.square: 2>)
+    ('SQUARE', <Shape.SQUARE: 2>)
+    ('DIAMOND', <Shape.DIAMOND: 1>)
+    ('CIRCLE', <Shape.CIRCLE: 3>)
+    ('ALIAS_FOR_SQUARE', <Shape.SQUARE: 2>)
 
 The ``__members__`` attribute can be used for detailed programmatic access to
 the enumeration members.  For example, finding all the aliases::
 
     >>> [name for name, member in Shape.__members__.items() if member.name != name]
-    ['alias_for_square']
+    ['ALIAS_FOR_SQUARE']
 
 
 Comparisons
@@ -305,35 +305,35 @@
 
 Enumeration members are compared by identity::
 
-    >>> Color.red is Color.red
+    >>> Color.RED is Color.RED
     True
-    >>> Color.red is Color.blue
+    >>> Color.RED is Color.BLUE
     False
-    >>> Color.red is not Color.blue
+    >>> Color.RED is not Color.BLUE
     True
 
 Ordered comparisons between enumeration values are *not* supported.  Enum
 members are not integers (but see `IntEnum`_ below)::
 
-    >>> Color.red < Color.blue
+    >>> Color.RED < Color.BLUE
     Traceback (most recent call last):
       File "<stdin>", line 1, in <module>
     TypeError: '<' not supported between instances of 'Color' and 'Color'
 
 Equality comparisons are defined though::
 
-    >>> Color.blue == Color.red
+    >>> Color.BLUE == Color.RED
     False
-    >>> Color.blue != Color.red
+    >>> Color.BLUE != Color.RED
     True
-    >>> Color.blue == Color.blue
+    >>> Color.BLUE == Color.BLUE
     True
 
 Comparisons against non-enumeration values will always compare not equal
 (again, :class:`IntEnum` was explicitly designed to behave differently, see
 below)::
 
-    >>> Color.blue == 2
+    >>> Color.BLUE == 2
     False
 
 
@@ -350,8 +350,8 @@
 usual.  If we have this enumeration::
 
     >>> class Mood(Enum):
-    ...     funky = 1
-    ...     happy = 3
+    ...     FUNKY = 1
+    ...     HAPPY = 3
     ...
     ...     def describe(self):
     ...         # self is the member here
@@ -363,16 +363,16 @@
     ...     @classmethod
     ...     def favorite_mood(cls):
     ...         # cls here is the enumeration
-    ...         return cls.happy
+    ...         return cls.HAPPY
     ...
 
 Then::
 
     >>> Mood.favorite_mood()
-    <Mood.happy: 3>
-    >>> Mood.happy.describe()
-    ('happy', 3)
-    >>> str(Mood.funky)
+    <Mood.HAPPY: 3>
+    >>> Mood.HAPPY.describe()
+    ('HAPPY', 3)
+    >>> str(Mood.FUNKY)
     'my custom str! 1'
 
 The rules for what is allowed are as follows: names that start and end with
@@ -393,7 +393,7 @@
 any members.  So this is forbidden::
 
     >>> class MoreColor(Color):
-    ...     pink = 17
+    ...     PINK = 17
     ...
     Traceback (most recent call last):
     ...
@@ -406,8 +406,8 @@
     ...         pass
     ...
     >>> class Bar(Foo):
-    ...     happy = 1
-    ...     sad = 2
+    ...     HAPPY = 1
+    ...     SAD = 2
     ...
 
 Allowing subclassing of enums that define members would lead to a violation of
@@ -423,7 +423,7 @@
 
     >>> from test.test_enum import Fruit
     >>> from pickle import dumps, loads
-    >>> Fruit.tomato is loads(dumps(Fruit.tomato))
+    >>> Fruit.TOMATO is loads(dumps(Fruit.TOMATO))
     True
 
 The usual restrictions for pickling apply: picklable enums must be defined in
@@ -444,15 +444,15 @@
 
 The :class:`Enum` class is callable, providing the following functional API::
 
-    >>> Animal = Enum('Animal', 'ant bee cat dog')
+    >>> Animal = Enum('Animal', 'ANT BEE CAT DOG')
     >>> Animal
     <enum 'Animal'>
-    >>> Animal.ant
-    <Animal.ant: 1>
-    >>> Animal.ant.value
+    >>> Animal.ANT
+    <Animal.ANT: 1>
+    >>> Animal.ANT.value
     1
     >>> list(Animal)
-    [<Animal.ant: 1>, <Animal.bee: 2>, <Animal.cat: 3>, <Animal.dog: 4>]
+    [<Animal.ANT: 1>, <Animal.BEE: 2>, <Animal.CAT: 3>, <Animal.DOG: 4>]
 
 The semantics of this API resemble :class:`~collections.namedtuple`. The first
 argument of the call to :class:`Enum` is the name of the enumeration.
@@ -467,10 +467,10 @@
 assignment to :class:`Animal` is equivalent to::
 
     >>> class Animal(Enum):
-    ...     ant = 1
-    ...     bee = 2
-    ...     cat = 3
-    ...     dog = 4
+    ...     ANT = 1
+    ...     BEE = 2
+    ...     CAT = 3
+    ...     DOG = 4
     ...
 
 The reason for defaulting to ``1`` as the starting number and not ``0`` is
@@ -483,7 +483,7 @@
 function in separate module, and also may not work on IronPython or Jython).
 The solution is to specify the module name explicitly as follows::
 
-    >>> Animal = Enum('Animal', 'ant bee cat dog', module=__name__)
+    >>> Animal = Enum('Animal', 'ANT BEE CAT DOG', module=__name__)
 
 .. warning::
 
@@ -496,7 +496,7 @@
 to find the class.  For example, if the class was made available in class
 SomeData in the global scope::
 
-    >>> Animal = Enum('Animal', 'ant bee cat dog', qualname='SomeData.Animal')
+    >>> Animal = Enum('Animal', 'ANT BEE CAT DOG', qualname='SomeData.Animal')
 
 The complete signature is::
 
@@ -507,19 +507,19 @@
 :names: The Enum members.  This can be a whitespace or comma separated string
   (values will start at 1 unless otherwise specified)::
 
-    'red green blue' | 'red,green,blue' | 'red, green, blue'
+    'RED GREEN BLUE' | 'RED,GREEN,BLUE' | 'RED, GREEN, BLUE'
 
   or an iterator of names::
 
-    ['red', 'green', 'blue']
+    ['RED', 'GREEN', 'BLUE']
 
   or an iterator of (name, value) pairs::
 
-    [('cyan', 4), ('magenta', 5), ('yellow', 6)]
+    [('CYAN', 4), ('MAGENTA', 5), ('YELLOW', 6)]
 
   or a mapping::
 
-    {'chartreuse': 7, 'sea_green': 11, 'rosemary': 42}
+    {'CHARTREUSE': 7, 'SEA_GREEN': 11, 'ROSEMARY': 42}
 
 :module: name of module where new Enum class can be found.
 
@@ -546,40 +546,40 @@
 
     >>> from enum import IntEnum
     >>> class Shape(IntEnum):
-    ...     circle = 1
-    ...     square = 2
+    ...     CIRCLE = 1
+    ...     SQUARE = 2
     ...
     >>> class Request(IntEnum):
-    ...     post = 1
-    ...     get = 2
+    ...     POST = 1
+    ...     GET = 2
     ...
     >>> Shape == 1
     False
-    >>> Shape.circle == 1
+    >>> Shape.CIRCLE == 1
     True
-    >>> Shape.circle == Request.post
+    >>> Shape.CIRCLE == Request.POST
     True
 
 However, they still can't be compared to standard :class:`Enum` enumerations::
 
     >>> class Shape(IntEnum):
-    ...     circle = 1
-    ...     square = 2
+    ...     CIRCLE = 1
+    ...     SQUARE = 2
     ...
     >>> class Color(Enum):
-    ...     red = 1
-    ...     green = 2
+    ...     RED = 1
+    ...     GREEN = 2
     ...
-    >>> Shape.circle == Color.red
+    >>> Shape.CIRCLE == Color.RED
     False
 
 :class:`IntEnum` values behave like integers in other ways you'd expect::
 
-    >>> int(Shape.circle)
+    >>> int(Shape.CIRCLE)
     1
-    >>> ['a', 'b', 'c'][Shape.circle]
+    >>> ['a', 'b', 'c'][Shape.CIRCLE]
     'b'
-    >>> [i for i in range(Shape.square)]
+    >>> [i for i in range(Shape.SQUARE)]
     [0, 1]
 
 
@@ -656,39 +656,39 @@
 
     >>> from enum import Flag
     >>> class Color(Flag):
-    ...     red = auto()
-    ...     blue = auto()
-    ...     green = auto()
+    ...     RED = auto()
+    ...     BLUE = auto()
+    ...     GREEN = auto()
     ...
-    >>> Color.red & Color.green
+    >>> Color.RED & Color.GREEN
     <Color.0: 0>
-    >>> bool(Color.red & Color.green)
+    >>> bool(Color.RED & Color.GREEN)
     False
 
 Individual flags should have values that are powers of two (1, 2, 4, 8, ...),
 while combinations of flags won't::
 
     >>> class Color(Flag):
-    ...     red = auto()
-    ...     blue = auto()
-    ...     green = auto()
-    ...     white = red | blue | green
+    ...     RED = auto()
+    ...     BLUE = auto()
+    ...     GREEN = auto()
+    ...     WHITE = RED | BLUE | GREEN
     ...
-    >>> Color.white
-    <Color.white: 7>
+    >>> Color.WHITE
+    <Color.WHITE: 7>
 
 Giving a name to the "no flags set" condition does not change its boolean
 value::
 
     >>> class Color(Flag):
-    ...     black = 0
-    ...     red = auto()
-    ...     blue = auto()
-    ...     green = auto()
+    ...     BLACK = 0
+    ...     RED = auto()
+    ...     BLUE = auto()
+    ...     GREEN = auto()
     ...
-    >>> Color.black
-    <Color.black: 0>
-    >>> bool(Color.black)
+    >>> Color.BLACK
+    <Color.BLACK: 0>
+    >>> bool(Color.BLACK)
     False
 
 .. note::
@@ -776,12 +776,12 @@
 Using :class:`object` would look like::
 
     >>> class Color(NoValue):
-    ...     red = auto()
-    ...     blue = auto()
-    ...     green = auto()
+    ...     RED = auto()
+    ...     BLUE = auto()
+    ...     GREEN = auto()
     ...
-    >>> Color.green
-    <Color.green>
+    >>> Color.GREEN
+    <Color.GREEN>
 
 
 Using :class:`object`
@@ -790,12 +790,12 @@
 Using :class:`object` would look like::
 
     >>> class Color(NoValue):
-    ...     red = object()
-    ...     green = object()
-    ...     blue = object()
+    ...     RED = object()
+    ...     GREEN = object()
+    ...     BLUE = object()
     ...
-    >>> Color.green
-    <Color.green>
+    >>> Color.GREEN
+    <Color.GREEN>
 
 
 Using a descriptive string
@@ -804,13 +804,13 @@
 Using a string as the value would look like::
 
     >>> class Color(NoValue):
-    ...     red = 'stop'
-    ...     green = 'go'
-    ...     blue = 'too fast!'
+    ...     RED = 'stop'
+    ...     GREEN = 'go'
+    ...     BLUE = 'too fast!'
     ...
-    >>> Color.green
-    <Color.green>
-    >>> Color.green.value
+    >>> Color.GREEN
+    <Color.GREEN>
+    >>> Color.GREEN.value
     'go'
 
 
@@ -827,13 +827,13 @@
     ...         return obj
     ...
     >>> class Color(AutoNumber):
-    ...     red = ()
-    ...     green = ()
-    ...     blue = ()
+    ...     RED = ()
+    ...     GREEN = ()
+    ...     BLUE = ()
     ...
-    >>> Color.green
-    <Color.green>
-    >>> Color.green.value
+    >>> Color.GREEN
+    <Color.GREEN>
+    >>> Color.GREEN.value
     2
 
 
@@ -897,14 +897,14 @@
     ...                 % (a, e))
     ...
     >>> class Color(DuplicateFreeEnum):
-    ...     red = 1
-    ...     green = 2
-    ...     blue = 3
-    ...     grene = 2
+    ...     RED = 1
+    ...     GREEN = 2
+    ...     BLUE = 3
+    ...     GRENE = 2
     ...
     Traceback (most recent call last):
     ...
-    ValueError: aliases not allowed in DuplicateFreeEnum:  'grene' --> 'green'
+    ValueError: aliases not allowed in DuplicateFreeEnum:  'GRENE' --> 'GREEN'
 
 .. note::
 
@@ -1007,10 +1007,10 @@
 and raise an error if the two do not match::
 
     >>> class Color(Enum):
-    ...     _order_ = 'red green blue'
-    ...     red = 1
-    ...     blue = 3
-    ...     green = 2
+    ...     _order_ = 'RED GREEN BLUE'
+    ...     RED = 1
+    ...     BLUE = 3
+    ...     GREEN = 2
     ...
     Traceback (most recent call last):
     ...
@@ -1028,7 +1028,8 @@
 normally accessed as ``EnumClass.member``.  Under certain circumstances they
 can also be accessed as ``EnumClass.member.member``, but you should never do
 this as that lookup may fail or, worse, return something besides the
-:class:`Enum` member you are looking for::
+:class:`Enum` member you are looking for (this is another good reason to use
+all-uppercase names for members)::
 
     >>> class FieldTypes(Enum):
     ...     name = 0
@@ -1078,15 +1079,15 @@
 all named flags and all named combinations of flags that are in the value::
 
     >>> class Color(Flag):
-    ...     red = auto()
-    ...     green = auto()
-    ...     blue = auto()
-    ...     magenta = red | blue
-    ...     yellow = red | green
-    ...     cyan = green | blue
+    ...     RED = auto()
+    ...     GREEN = auto()
+    ...     BLUE = auto()
+    ...     MAGENTA = RED | BLUE
+    ...     YELLOW = RED | GREEN
+    ...     CYAN = GREEN | BLUE
     ...
     >>> Color(3)  # named combination
-    <Color.yellow: 3>
+    <Color.YELLOW: 3>
     >>> Color(7)      # not named combination
-    <Color.cyan|magenta|blue|yellow|green|red: 7>
+    <Color.CYAN|MAGENTA|BLUE|YELLOW|GREEN|RED: 7>
 
diff --git a/Lib/enum.py b/Lib/enum.py
--- a/Lib/enum.py
+++ b/Lib/enum.py
@@ -267,7 +267,7 @@
 
         This method is used both when an enum class is given a value to match
         to an enumeration member (i.e. Color(3)) and for the functional API
-        (i.e. Color = Enum('Color', names='red green blue')).
+        (i.e. Color = Enum('Color', names='RED GREEN BLUE')).
 
         When used for the functional API:
 
@@ -517,7 +517,7 @@
         # without calling this method; this method is called by the metaclass'
         # __call__ (i.e. Color(3) ), and by pickle
         if type(value) is cls:
-            # For lookups like Color(Color.red)
+            # For lookups like Color(Color.RED)
             return value
         # by-value search for a matching enum member
         # see if it's in the reverse mapping (for hashable values)
diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py
--- a/Lib/test/test_enum.py
+++ b/Lib/test/test_enum.py
@@ -69,9 +69,9 @@
 # for doctests
 try:
     class Fruit(Enum):
-        tomato = 1
-        banana = 2
-        cherry = 3
+        TOMATO = 1
+        BANANA = 2
+        CHERRY = 3
 except Exception:
     pass
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 13:16:32 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 18:16:32 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Simplify_code_?=
 =?utf-8?q?in_an_example?=
Message-ID: <20161121181632.45076.96242.DBC98421@psf.io>

https://hg.python.org/cpython/rev/8f9f2033c1f0
changeset:   105294:8f9f2033c1f0
branch:      3.6
parent:      105292:b9801dab214a
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 10:16:01 2016 -0800
summary:
  Simplify code in an example

files:
  Doc/library/random.rst |  12 ++++++------
  1 files changed, 6 insertions(+), 6 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -429,12 +429,12 @@
 
     num_waiting = 0
     arrival = service_end = 0.0
-    for i in range(10000):
-        num_waiting += 1
-        arrival += expovariate(1.0 / average_arrival_interval)
-        print(f'{arrival:6.1f} arrived')
-
-        while arrival > service_end:
+    for i in range(20000):
+        if arrival <= service_end:
+            num_waiting += 1
+            arrival += expovariate(1.0 / average_arrival_interval)
+            print(f'{arrival:6.1f} arrived')
+        else:
             num_waiting -= 1
             service_start = service_end if num_waiting else arrival
             service_time = gauss(average_service_time, stdev_service_time)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 13:16:32 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 18:16:32 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161121181632.40895.91797.84BEDF9B@psf.io>

https://hg.python.org/cpython/rev/8c483030c5e7
changeset:   105295:8c483030c5e7
parent:      105293:5ec8d4c51363
parent:      105294:8f9f2033c1f0
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 10:16:25 2016 -0800
summary:
  merge

files:
  Doc/library/random.rst |  12 ++++++------
  1 files changed, 6 insertions(+), 6 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -429,12 +429,12 @@
 
     num_waiting = 0
     arrival = service_end = 0.0
-    for i in range(10000):
-        num_waiting += 1
-        arrival += expovariate(1.0 / average_arrival_interval)
-        print(f'{arrival:6.1f} arrived')
-
-        while arrival > service_end:
+    for i in range(20000):
+        if arrival <= service_end:
+            num_waiting += 1
+            arrival += expovariate(1.0 / average_arrival_interval)
+            print(f'{arrival:6.1f} arrived')
+        else:
             num_waiting -= 1
             service_start = service_end if num_waiting else arrival
             service_time = gauss(average_service_time, stdev_service_time)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 13:52:27 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 18:52:27 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161121185227.8083.86107.F53DC751@psf.io>

https://hg.python.org/cpython/rev/1c91c3b1a20d
changeset:   105297:1c91c3b1a20d
parent:      105295:8c483030c5e7
parent:      105296:8a9ea7aaab22
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 10:52:20 2016 -0800
summary:
  merge

files:
  Doc/library/random.rst |  20 ++++++++++++++++++++
  1 files changed, 20 insertions(+), 0 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -441,3 +441,23 @@
             service_end = service_start + service_time
             print(f'\t\t{service_start:.1f} to {service_end:.1f} serviced')
 
+.. seealso::
+
+   `Statistics for Hackers <https://www.youtube.com/watch?v=Iq9DzN6mvYA>`_
+   a video tutorial by
+   `Jake Vanderplas <https://us.pycon.org/2016/speaker/profile/295/>`_
+   on statistical analysis using just a few fundamental concepts
+   including simulation, sampling, shuffling, and cross-validation.
+
+   `Economics Simulation
+   <http://nbviewer.jupyter.org/url/norvig.com/ipython/Economics.ipynb>`_
+   a simulation of a marketplace by
+   `Peter Norvig <http://norvig.com/bio.html>`_ that shows effective
+   use of many the tools and distributions provided by this module
+   (gauss, uniform, sample, betavariate, choice, triangular, and randrange).
+
+   `A Concrete Introduction to Probability (using Python)
+   <http://nbviewer.jupyter.org/url/norvig.com/ipython/Probability.ipynb>`_
+   a tutorial by `Peter Norvig <http://norvig.com/bio.html>`_ covering
+   the basics of probability theory, how to write simulations, and
+   performing data analysis using Python.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 13:52:27 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 18:52:27 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Add_a_seealso_?=
 =?utf-8?q?section_for_further_reference_and_skill_building?=
Message-ID: <20161121185227.41261.54325.5D13F746@psf.io>

https://hg.python.org/cpython/rev/8a9ea7aaab22
changeset:   105296:8a9ea7aaab22
branch:      3.6
parent:      105294:8f9f2033c1f0
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 10:52:04 2016 -0800
summary:
  Add a seealso section for further reference and skill building

files:
  Doc/library/random.rst |  20 ++++++++++++++++++++
  1 files changed, 20 insertions(+), 0 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -441,3 +441,23 @@
             service_end = service_start + service_time
             print(f'\t\t{service_start:.1f} to {service_end:.1f} serviced')
 
+.. seealso::
+
+   `Statistics for Hackers <https://www.youtube.com/watch?v=Iq9DzN6mvYA>`_
+   a video tutorial by
+   `Jake Vanderplas <https://us.pycon.org/2016/speaker/profile/295/>`_
+   on statistical analysis using just a few fundamental concepts
+   including simulation, sampling, shuffling, and cross-validation.
+
+   `Economics Simulation
+   <http://nbviewer.jupyter.org/url/norvig.com/ipython/Economics.ipynb>`_
+   a simulation of a marketplace by
+   `Peter Norvig <http://norvig.com/bio.html>`_ that shows effective
+   use of many the tools and distributions provided by this module
+   (gauss, uniform, sample, betavariate, choice, triangular, and randrange).
+
+   `A Concrete Introduction to Probability (using Python)
+   <http://nbviewer.jupyter.org/url/norvig.com/ipython/Probability.ipynb>`_
+   a tutorial by `Peter Norvig <http://norvig.com/bio.html>`_ covering
+   the basics of probability theory, how to write simulations, and
+   performing data analysis using Python.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 15:34:21 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 20:34:21 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Misc_readabili?=
 =?utf-8?q?ty_and_organization_improvements_for_the_random_docs?=
Message-ID: <20161121203421.7580.87067.993292A3@psf.io>

https://hg.python.org/cpython/rev/2546654c41e8
changeset:   105298:2546654c41e8
branch:      3.6
parent:      105296:8a9ea7aaab22
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 12:33:50 2016 -0800
summary:
  Misc readability and organization improvements for the random docs

files:
  Doc/library/random.rst |  57 ++++++++++++++++-------------
  1 files changed, 31 insertions(+), 26 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -49,8 +49,21 @@
    security purposes.  For security or cryptographic uses, see the
    :mod:`secrets` module.
 
+.. seealso::
 
-Bookkeeping functions:
+   M. Matsumoto and T. Nishimura, "Mersenne Twister: A 623-dimensionally
+   equidistributed uniform pseudorandom number generator", ACM Transactions on
+   Modeling and Computer Simulation Vol. 8, No. 1, January pp.3-30 1998.
+
+
+   `Complementary-Multiply-with-Carry recipe
+   <https://code.activestate.com/recipes/576707/>`_ for a compatible alternative
+   random number generator with a long period and comparatively simple update
+   operations.
+
+
+Bookkeeping functions
+---------------------
 
 .. function:: seed(a=None, version=2)
 
@@ -94,7 +107,8 @@
    :meth:`randrange` to handle arbitrarily large ranges.
 
 
-Functions for integers:
+Functions for integers
+----------------------
 
 .. function:: randrange(stop)
               randrange(start, stop[, step])
@@ -117,7 +131,8 @@
    ``randrange(a, b+1)``.
 
 
-Functions for sequences:
+Functions for sequences
+-----------------------
 
 .. function:: choice(seq)
 
@@ -188,6 +203,9 @@
    If the sample size is larger than the population size, a :exc:`ValueError`
    is raised.
 
+Real-valued distributions
+-------------------------
+
 The following functions generate specific real-valued distributions. Function
 parameters are named after the corresponding variables in the distribution's
 equation, as used in common mathematical practice; most of these equations can
@@ -282,7 +300,8 @@
    parameter.
 
 
-Alternative Generator:
+Alternative Generator
+---------------------
 
 .. class:: SystemRandom([seed])
 
@@ -294,19 +313,6 @@
    :exc:`NotImplementedError` if called.
 
 
-.. seealso::
-
-   M. Matsumoto and T. Nishimura, "Mersenne Twister: A 623-dimensionally
-   equidistributed uniform pseudorandom number generator", ACM Transactions on
-   Modeling and Computer Simulation Vol. 8, No. 1, January pp.3-30 1998.
-
-
-   `Complementary-Multiply-with-Carry recipe
-   <https://code.activestate.com/recipes/576707/>`_ for a compatible alternative
-   random number generator with a long period and comparatively simple update
-   operations.
-
-
 Notes on Reproducibility
 ------------------------
 
@@ -333,13 +339,13 @@
    >>> random()                             # Random float:  0.0 <= x < 1.0
    0.37444887175646646
 
-   >>> uniform(2, 10)                       # Random float:  2.0 <= x < 10.0
+   >>> uniform(2.5, 10.0)                   # Random float:  2.5 <= x < 10.0
    3.1800146073117523
 
-   >>> expovariate(1/5)                     # Interval between arrivals averaging 5 seconds
+   >>> expovariate(1 / 5)                   # Interval between arrivals averaging 5 seconds
    5.148957571865031
 
-   >>> randrange(10)                        # Integer from 0 to 9
+   >>> randrange(10)                        # Integer from 0 to 9 inclusive
    7
 
    >>> randrange(0, 101, 2)                 # Even integer from 0 to 100 inclusive
@@ -362,17 +368,16 @@
    >>> choices(['red', 'black', 'green'], [18, 18, 2], k=6)
    ['red', 'green', 'black', 'black', 'red', 'black']
 
-   # Deal 20 cards without replacement from a deck of 52
-   # playing cards and determine the proportion of cards
-   # with a ten-value (i.e. a ten, jack, queen, or king).
+   # Deal 20 cards without replacement from a deck of 52 playing cards
+   # and determine the proportion of cards with a ten-value (i.e. a ten,
+   # jack, queen, or king).
    >>> deck = collections.Counter(tens=16, low_cards=36)
    >>> seen = sample(list(deck.elements()), k=20)
    >>> print(seen.count('tens') / 20)
    0.15
 
-   # Estimate the probability of getting 5 or more heads
-   # from 7 spins of a biased coin that settles on heads
-   # 60% of the time.
+   # Estimate the probability of getting 5 or more heads from 7 spins
+   # of a biased coin that settles on heads 60% of the time.
    >>> n = 10000
    >>> cw = [0.60, 1.00]
    >>> sum(choices('HT', cum_weights=cw, k=7).count('H') >= 5 for i in range(n)) / n

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 15:34:21 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 20:34:21 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161121203421.7898.74195.F1FD733C@psf.io>

https://hg.python.org/cpython/rev/8e59aff28e2a
changeset:   105299:8e59aff28e2a
parent:      105297:1c91c3b1a20d
parent:      105298:2546654c41e8
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 12:34:12 2016 -0800
summary:
  merge

files:
  Doc/library/random.rst |  57 ++++++++++++++++-------------
  1 files changed, 31 insertions(+), 26 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -49,8 +49,21 @@
    security purposes.  For security or cryptographic uses, see the
    :mod:`secrets` module.
 
+.. seealso::
 
-Bookkeeping functions:
+   M. Matsumoto and T. Nishimura, "Mersenne Twister: A 623-dimensionally
+   equidistributed uniform pseudorandom number generator", ACM Transactions on
+   Modeling and Computer Simulation Vol. 8, No. 1, January pp.3-30 1998.
+
+
+   `Complementary-Multiply-with-Carry recipe
+   <https://code.activestate.com/recipes/576707/>`_ for a compatible alternative
+   random number generator with a long period and comparatively simple update
+   operations.
+
+
+Bookkeeping functions
+---------------------
 
 .. function:: seed(a=None, version=2)
 
@@ -94,7 +107,8 @@
    :meth:`randrange` to handle arbitrarily large ranges.
 
 
-Functions for integers:
+Functions for integers
+----------------------
 
 .. function:: randrange(stop)
               randrange(start, stop[, step])
@@ -117,7 +131,8 @@
    ``randrange(a, b+1)``.
 
 
-Functions for sequences:
+Functions for sequences
+-----------------------
 
 .. function:: choice(seq)
 
@@ -188,6 +203,9 @@
    If the sample size is larger than the population size, a :exc:`ValueError`
    is raised.
 
+Real-valued distributions
+-------------------------
+
 The following functions generate specific real-valued distributions. Function
 parameters are named after the corresponding variables in the distribution's
 equation, as used in common mathematical practice; most of these equations can
@@ -282,7 +300,8 @@
    parameter.
 
 
-Alternative Generator:
+Alternative Generator
+---------------------
 
 .. class:: SystemRandom([seed])
 
@@ -294,19 +313,6 @@
    :exc:`NotImplementedError` if called.
 
 
-.. seealso::
-
-   M. Matsumoto and T. Nishimura, "Mersenne Twister: A 623-dimensionally
-   equidistributed uniform pseudorandom number generator", ACM Transactions on
-   Modeling and Computer Simulation Vol. 8, No. 1, January pp.3-30 1998.
-
-
-   `Complementary-Multiply-with-Carry recipe
-   <https://code.activestate.com/recipes/576707/>`_ for a compatible alternative
-   random number generator with a long period and comparatively simple update
-   operations.
-
-
 Notes on Reproducibility
 ------------------------
 
@@ -333,13 +339,13 @@
    >>> random()                             # Random float:  0.0 <= x < 1.0
    0.37444887175646646
 
-   >>> uniform(2, 10)                       # Random float:  2.0 <= x < 10.0
+   >>> uniform(2.5, 10.0)                   # Random float:  2.5 <= x < 10.0
    3.1800146073117523
 
-   >>> expovariate(1/5)                     # Interval between arrivals averaging 5 seconds
+   >>> expovariate(1 / 5)                   # Interval between arrivals averaging 5 seconds
    5.148957571865031
 
-   >>> randrange(10)                        # Integer from 0 to 9
+   >>> randrange(10)                        # Integer from 0 to 9 inclusive
    7
 
    >>> randrange(0, 101, 2)                 # Even integer from 0 to 100 inclusive
@@ -362,17 +368,16 @@
    >>> choices(['red', 'black', 'green'], [18, 18, 2], k=6)
    ['red', 'green', 'black', 'black', 'red', 'black']
 
-   # Deal 20 cards without replacement from a deck of 52
-   # playing cards and determine the proportion of cards
-   # with a ten-value (i.e. a ten, jack, queen, or king).
+   # Deal 20 cards without replacement from a deck of 52 playing cards
+   # and determine the proportion of cards with a ten-value (i.e. a ten,
+   # jack, queen, or king).
    >>> deck = collections.Counter(tens=16, low_cards=36)
    >>> seen = sample(list(deck.elements()), k=20)
    >>> print(seen.count('tens') / 20)
    0.15
 
-   # Estimate the probability of getting 5 or more heads
-   # from 7 spins of a biased coin that settles on heads
-   # 60% of the time.
+   # Estimate the probability of getting 5 or more heads from 7 spins
+   # of a biased coin that settles on heads 60% of the time.
    >>> n = 10000
    >>> cw = [0.60, 1.00]
    >>> sum(choices('HT', cum_weights=cw, k=7).count('H') >= 5 for i in range(n)) / n

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 17:13:34 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 22:13:34 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Add_analysis_s?=
 =?utf-8?q?ection_to_motivate_the_single_server_queue_example?=
Message-ID: <20161121221334.2820.37750.4A216C83@psf.io>

https://hg.python.org/cpython/rev/c508baa300bb
changeset:   105300:c508baa300bb
branch:      3.6
parent:      105298:2546654c41e8
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 14:13:07 2016 -0800
summary:
  Add analysis section to motivate the single server queue example

files:
  Doc/library/random.rst |  13 ++++++++++---
  1 files changed, 10 insertions(+), 3 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -426,25 +426,32 @@
 
 Simulation of arrival times and service deliveries in a single server queue::
 
-    from random import gauss, expovariate
+    from random import expovariate, gauss
+    from statistics import mean, median, stdev
 
     average_arrival_interval = 5.6
     average_service_time = 5.0
     stdev_service_time = 0.5
 
     num_waiting = 0
+    arrivals = []
+    starts = []
     arrival = service_end = 0.0
     for i in range(20000):
         if arrival <= service_end:
             num_waiting += 1
             arrival += expovariate(1.0 / average_arrival_interval)
-            print(f'{arrival:6.1f} arrived')
+            arrivals.append(arrival)
         else:
             num_waiting -= 1
             service_start = service_end if num_waiting else arrival
             service_time = gauss(average_service_time, stdev_service_time)
             service_end = service_start + service_time
-            print(f'\t\t{service_start:.1f} to {service_end:.1f} serviced')
+            starts.append(service_start)
+
+    waits = [start - arrival for arrival, start in zip(arrivals, starts)]
+    print(f'Mean wait: {mean(waits):.1f}.  Stdev wait: {stdev(waits):.1f}.')
+    print(f'Median wait: {median(waits):.1f}.  Max wait: {max(waits):.1f}.')
 
 .. seealso::
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 17:13:34 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 22:13:34 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161121221334.130095.55355.F2593A40@psf.io>

https://hg.python.org/cpython/rev/018c2e0f82c3
changeset:   105301:018c2e0f82c3
parent:      105299:8e59aff28e2a
parent:      105300:c508baa300bb
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 14:13:29 2016 -0800
summary:
  merge

files:
  Doc/library/random.rst |  13 ++++++++++---
  1 files changed, 10 insertions(+), 3 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -426,25 +426,32 @@
 
 Simulation of arrival times and service deliveries in a single server queue::
 
-    from random import gauss, expovariate
+    from random import expovariate, gauss
+    from statistics import mean, median, stdev
 
     average_arrival_interval = 5.6
     average_service_time = 5.0
     stdev_service_time = 0.5
 
     num_waiting = 0
+    arrivals = []
+    starts = []
     arrival = service_end = 0.0
     for i in range(20000):
         if arrival <= service_end:
             num_waiting += 1
             arrival += expovariate(1.0 / average_arrival_interval)
-            print(f'{arrival:6.1f} arrived')
+            arrivals.append(arrival)
         else:
             num_waiting -= 1
             service_start = service_end if num_waiting else arrival
             service_time = gauss(average_service_time, stdev_service_time)
             service_end = service_start + service_time
-            print(f'\t\t{service_start:.1f} to {service_end:.1f} serviced')
+            starts.append(service_start)
+
+    waits = [start - arrival for arrival, start in zip(arrivals, starts)]
+    print(f'Mean wait: {mean(waits):.1f}.  Stdev wait: {stdev(waits):.1f}.')
+    print(f'Median wait: {median(waits):.1f}.  Max wait: {max(waits):.1f}.')
 
 .. seealso::
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 17:24:57 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 22:24:57 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge?=
Message-ID: <20161121222456.18939.51968.D87D6659@psf.io>

https://hg.python.org/cpython/rev/1c3bfc5e12f8
changeset:   105303:1c3bfc5e12f8
parent:      105301:018c2e0f82c3
parent:      105302:88880a916174
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 14:24:51 2016 -0800
summary:
  Merge

files:
  Include/code.h |  7 +++----
  1 files changed, 3 insertions(+), 4 deletions(-)


diff --git a/Include/code.h b/Include/code.h
--- a/Include/code.h
+++ b/Include/code.h
@@ -32,9 +32,8 @@
     PyObject *co_varnames;	/* tuple of strings (local variable names) */
     PyObject *co_freevars;	/* tuple of strings (free variable names) */
     PyObject *co_cellvars;      /* tuple of strings (cell variable names) */
-    /* The rest aren't used in either hash or comparisons, except for
-       co_name (used in both) and co_firstlineno (used only in
-       comparisons).  This is done to preserve the name and line number
+    /* The rest aren't used in either hash or comparisons, except for co_name,
+       used in both. This is done to preserve the name and line number
        for tracebacks and debuggers; otherwise, constant de-duplication
        would collapse identical functions/lambdas defined on different lines.
     */
@@ -45,7 +44,7 @@
 				   Objects/lnotab_notes.txt for details. */
     void *co_zombieframe;     /* for optimization only (see frameobject.c) */
     PyObject *co_weakreflist;   /* to support weakrefs to code objects */
-    /* Scratch space for extra data relating to the code object.__icc_nan
+    /* Scratch space for extra data relating to the code object.
        Type is a void* to keep the format private in codeobject.c to force
        people to go through the proper APIs. */
     void *co_extra;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 17:24:57 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 22:24:57 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgMjg3NTE6?=
 =?utf-8?q?_Fix_comments_in_code=2Eh=2E__=28Contributed_by_Ned_Batchelder?=
 =?utf-8?b?KS4=?=
Message-ID: <20161121222456.130963.2778.563DEE12@psf.io>

https://hg.python.org/cpython/rev/88880a916174
changeset:   105302:88880a916174
branch:      3.6
parent:      105300:c508baa300bb
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 14:24:32 2016 -0800
summary:
  Issue 28751: Fix comments in code.h.  (Contributed by Ned Batchelder).

files:
  Include/code.h |  7 +++----
  1 files changed, 3 insertions(+), 4 deletions(-)


diff --git a/Include/code.h b/Include/code.h
--- a/Include/code.h
+++ b/Include/code.h
@@ -32,9 +32,8 @@
     PyObject *co_varnames;	/* tuple of strings (local variable names) */
     PyObject *co_freevars;	/* tuple of strings (free variable names) */
     PyObject *co_cellvars;      /* tuple of strings (cell variable names) */
-    /* The rest aren't used in either hash or comparisons, except for
-       co_name (used in both) and co_firstlineno (used only in
-       comparisons).  This is done to preserve the name and line number
+    /* The rest aren't used in either hash or comparisons, except for co_name,
+       used in both. This is done to preserve the name and line number
        for tracebacks and debuggers; otherwise, constant de-duplication
        would collapse identical functions/lambdas defined on different lines.
     */
@@ -45,7 +44,7 @@
 				   Objects/lnotab_notes.txt for details. */
     void *co_zombieframe;     /* for optimization only (see frameobject.c) */
     PyObject *co_weakreflist;   /* to support weakrefs to code objects */
-    /* Scratch space for extra data relating to the code object.__icc_nan
+    /* Scratch space for extra data relating to the code object.
        Type is a void* to keep the format private in codeobject.c to force
        people to go through the proper APIs. */
     void *co_extra;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 17:30:43 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 21 Nov 2016 22:30:43 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogSXNzdWUgIzI4NzUyOiBSZXN0b3JlZCB0aGUgX19yZWR1Y2VfXygpIG1l?=
 =?utf-8?q?thods_of_datetime_objects=2E?=
Message-ID: <20161121223043.130239.49690.28766603@psf.io>

https://hg.python.org/cpython/rev/23140bd66d86
changeset:   105305:23140bd66d86
parent:      105303:1c3bfc5e12f8
parent:      105304:0a2a0061e425
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 22 00:30:32 2016 +0200
summary:
  Issue #28752: Restored the __reduce__() methods of datetime objects.

files:
  Lib/datetime.py            |  12 ++++++--
  Lib/test/datetimetester.py |   7 ++++
  Misc/NEWS                  |   2 +
  Modules/_datetimemodule.c  |  38 +++++++++++++++++++------
  4 files changed, 46 insertions(+), 13 deletions(-)


diff --git a/Lib/datetime.py b/Lib/datetime.py
--- a/Lib/datetime.py
+++ b/Lib/datetime.py
@@ -932,7 +932,7 @@
 
     # Pickle support.
 
-    def _getstate(self, protocol=3):
+    def _getstate(self):
         yhi, ylo = divmod(self._year, 256)
         return bytes([yhi, ylo, self._month, self._day]),
 
@@ -940,8 +940,8 @@
         yhi, ylo, self._month, self._day = string
         self._year = yhi * 256 + ylo
 
-    def __reduce_ex__(self, protocol):
-        return (self.__class__, self._getstate(protocol))
+    def __reduce__(self):
+        return (self.__class__, self._getstate())
 
 _date_class = date  # so functions w/ args named "date" can get at the class
 
@@ -1348,6 +1348,9 @@
     def __reduce_ex__(self, protocol):
         return (time, self._getstate(protocol))
 
+    def __reduce__(self):
+        return self.__reduce_ex__(2)
+
 _time_class = time  # so functions w/ args named "time" can get at the class
 
 time.min = time(0, 0, 0)
@@ -1923,6 +1926,9 @@
     def __reduce_ex__(self, protocol):
         return (self.__class__, self._getstate(protocol))
 
+    def __reduce__(self):
+        return self.__reduce_ex__(2)
+
 
 datetime.min = datetime(1, 1, 1)
 datetime.max = datetime(9999, 12, 31, 23, 59, 59, 999999)
diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py
--- a/Lib/test/datetimetester.py
+++ b/Lib/test/datetimetester.py
@@ -1329,6 +1329,7 @@
             green = pickler.dumps(orig, proto)
             derived = unpickler.loads(green)
             self.assertEqual(orig, derived)
+        self.assertEqual(orig.__reduce__(), orig.__reduce_ex__(2))
 
     def test_compare(self):
         t1 = self.theclass(2, 3, 4)
@@ -1830,6 +1831,7 @@
             green = pickler.dumps(orig, proto)
             derived = unpickler.loads(green)
             self.assertEqual(orig, derived)
+        self.assertEqual(orig.__reduce__(), orig.__reduce_ex__(2))
 
     def test_more_pickling(self):
         a = self.theclass(2003, 2, 7, 16, 48, 37, 444116)
@@ -2469,6 +2471,7 @@
             green = pickler.dumps(orig, proto)
             derived = unpickler.loads(green)
             self.assertEqual(orig, derived)
+        self.assertEqual(orig.__reduce__(), orig.__reduce_ex__(2))
 
     def test_pickling_subclass_time(self):
         args = 20, 59, 16, 64**2
@@ -2829,6 +2832,7 @@
             green = pickler.dumps(orig, proto)
             derived = unpickler.loads(green)
             self.assertEqual(orig, derived)
+        self.assertEqual(orig.__reduce__(), orig.__reduce_ex__(2))
 
         # Try one with a tzinfo.
         tinfo = PicklableFixedOffset(-300, 'cookie')
@@ -2840,6 +2844,7 @@
             self.assertIsInstance(derived.tzinfo, PicklableFixedOffset)
             self.assertEqual(derived.utcoffset(), timedelta(minutes=-300))
             self.assertEqual(derived.tzname(), 'cookie')
+        self.assertEqual(orig.__reduce__(), orig.__reduce_ex__(2))
 
     def test_more_bool(self):
         # time is always True.
@@ -3043,6 +3048,7 @@
             green = pickler.dumps(orig, proto)
             derived = unpickler.loads(green)
             self.assertEqual(orig, derived)
+        self.assertEqual(orig.__reduce__(), orig.__reduce_ex__(2))
 
         # Try one with a tzinfo.
         tinfo = PicklableFixedOffset(-300, 'cookie')
@@ -3055,6 +3061,7 @@
             self.assertIsInstance(derived.tzinfo, PicklableFixedOffset)
             self.assertEqual(derived.utcoffset(), timedelta(minutes=-300))
             self.assertEqual(derived.tzname(), 'cookie')
+        self.assertEqual(orig.__reduce__(), orig.__reduce_ex__(2))
 
     def test_extreme_hashes(self):
         # If an attempt is made to hash these via subtracting the offset
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -136,6 +136,8 @@
 Library
 -------
 
+- Issue #28752: Restored the __reduce__() methods of datetime objects.
+
 - Issue #28727: Regular expression patterns, _sre.SRE_Pattern objects created
   by re.compile(), become comparable (only x==y and x!=y operators). This
   change should fix the issue #18383: don't duplicate warning filters when the
diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c
--- a/Modules/_datetimemodule.c
+++ b/Modules/_datetimemodule.c
@@ -3954,15 +3954,21 @@
 }
 
 static PyObject *
-time_reduce(PyDateTime_Time *self, PyObject *args)
-{
-    int proto = 0;
-    if (!PyArg_ParseTuple(args, "|i:__reduce_ex__", &proto))
+time_reduce_ex(PyDateTime_Time *self, PyObject *args)
+{
+    int proto;
+    if (!PyArg_ParseTuple(args, "i:__reduce_ex__", &proto))
         return NULL;
 
     return Py_BuildValue("(ON)", Py_TYPE(self), time_getstate(self, proto));
 }
 
+static PyObject *
+time_reduce(PyDateTime_Time *self, PyObject *arg)
+{
+    return Py_BuildValue("(ON)", Py_TYPE(self), time_getstate(self, 2));
+}
+
 static PyMethodDef time_methods[] = {
 
     {"isoformat",   (PyCFunction)time_isoformat,        METH_VARARGS | METH_KEYWORDS,
@@ -3988,9 +3994,12 @@
     {"replace",     (PyCFunction)time_replace,          METH_VARARGS | METH_KEYWORDS,
      PyDoc_STR("Return time with new specified fields.")},
 
-    {"__reduce_ex__", (PyCFunction)time_reduce,        METH_VARARGS,
+    {"__reduce_ex__", (PyCFunction)time_reduce_ex,        METH_VARARGS,
      PyDoc_STR("__reduce_ex__(proto) -> (cls, state)")},
 
+    {"__reduce__", (PyCFunction)time_reduce,        METH_NOARGS,
+     PyDoc_STR("__reduce__() -> (cls, state)")},
+
     {NULL,      NULL}
 };
 
@@ -5419,15 +5428,21 @@
 }
 
 static PyObject *
-datetime_reduce(PyDateTime_DateTime *self, PyObject *args)
-{
-    int proto = 0;
-    if (!PyArg_ParseTuple(args, "|i:__reduce_ex__", &proto))
+datetime_reduce_ex(PyDateTime_DateTime *self, PyObject *args)
+{
+    int proto;
+    if (!PyArg_ParseTuple(args, "i:__reduce_ex__", &proto))
         return NULL;
 
     return Py_BuildValue("(ON)", Py_TYPE(self), datetime_getstate(self, proto));
 }
 
+static PyObject *
+datetime_reduce(PyDateTime_DateTime *self, PyObject *arg)
+{
+    return Py_BuildValue("(ON)", Py_TYPE(self), datetime_getstate(self, 2));
+}
+
 static PyMethodDef datetime_methods[] = {
 
     /* Class methods: */
@@ -5502,9 +5517,12 @@
     {"astimezone",  (PyCFunction)datetime_astimezone, METH_VARARGS | METH_KEYWORDS,
      PyDoc_STR("tz -> convert to local time in new timezone tz\n")},
 
-    {"__reduce_ex__", (PyCFunction)datetime_reduce,     METH_VARARGS,
+    {"__reduce_ex__", (PyCFunction)datetime_reduce_ex,     METH_VARARGS,
      PyDoc_STR("__reduce_ex__(proto) -> (cls, state)")},
 
+    {"__reduce__", (PyCFunction)datetime_reduce,     METH_NOARGS,
+     PyDoc_STR("__reduce__() -> (cls, state)")},
+
     {NULL,      NULL}
 };
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 17:30:43 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 21 Nov 2016 22:30:43 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NzUy?=
 =?utf-8?q?=3A_Restored_the_=5F=5Freduce=5F=5F=28=29_methods_of_datetime_o?=
 =?utf-8?q?bjects=2E?=
Message-ID: <20161121223043.2600.69670.A7791CEB@psf.io>

https://hg.python.org/cpython/rev/0a2a0061e425
changeset:   105304:0a2a0061e425
branch:      3.6
parent:      105302:88880a916174
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 22 00:29:42 2016 +0200
summary:
  Issue #28752: Restored the __reduce__() methods of datetime objects.

files:
  Lib/datetime.py            |  12 ++++++--
  Lib/test/datetimetester.py |   7 ++++
  Misc/NEWS                  |   2 +
  Modules/_datetimemodule.c  |  38 +++++++++++++++++++------
  4 files changed, 46 insertions(+), 13 deletions(-)


diff --git a/Lib/datetime.py b/Lib/datetime.py
--- a/Lib/datetime.py
+++ b/Lib/datetime.py
@@ -932,7 +932,7 @@
 
     # Pickle support.
 
-    def _getstate(self, protocol=3):
+    def _getstate(self):
         yhi, ylo = divmod(self._year, 256)
         return bytes([yhi, ylo, self._month, self._day]),
 
@@ -940,8 +940,8 @@
         yhi, ylo, self._month, self._day = string
         self._year = yhi * 256 + ylo
 
-    def __reduce_ex__(self, protocol):
-        return (self.__class__, self._getstate(protocol))
+    def __reduce__(self):
+        return (self.__class__, self._getstate())
 
 _date_class = date  # so functions w/ args named "date" can get at the class
 
@@ -1348,6 +1348,9 @@
     def __reduce_ex__(self, protocol):
         return (time, self._getstate(protocol))
 
+    def __reduce__(self):
+        return self.__reduce_ex__(2)
+
 _time_class = time  # so functions w/ args named "time" can get at the class
 
 time.min = time(0, 0, 0)
@@ -1923,6 +1926,9 @@
     def __reduce_ex__(self, protocol):
         return (self.__class__, self._getstate(protocol))
 
+    def __reduce__(self):
+        return self.__reduce_ex__(2)
+
 
 datetime.min = datetime(1, 1, 1)
 datetime.max = datetime(9999, 12, 31, 23, 59, 59, 999999)
diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py
--- a/Lib/test/datetimetester.py
+++ b/Lib/test/datetimetester.py
@@ -1329,6 +1329,7 @@
             green = pickler.dumps(orig, proto)
             derived = unpickler.loads(green)
             self.assertEqual(orig, derived)
+        self.assertEqual(orig.__reduce__(), orig.__reduce_ex__(2))
 
     def test_compare(self):
         t1 = self.theclass(2, 3, 4)
@@ -1830,6 +1831,7 @@
             green = pickler.dumps(orig, proto)
             derived = unpickler.loads(green)
             self.assertEqual(orig, derived)
+        self.assertEqual(orig.__reduce__(), orig.__reduce_ex__(2))
 
     def test_more_pickling(self):
         a = self.theclass(2003, 2, 7, 16, 48, 37, 444116)
@@ -2469,6 +2471,7 @@
             green = pickler.dumps(orig, proto)
             derived = unpickler.loads(green)
             self.assertEqual(orig, derived)
+        self.assertEqual(orig.__reduce__(), orig.__reduce_ex__(2))
 
     def test_pickling_subclass_time(self):
         args = 20, 59, 16, 64**2
@@ -2829,6 +2832,7 @@
             green = pickler.dumps(orig, proto)
             derived = unpickler.loads(green)
             self.assertEqual(orig, derived)
+        self.assertEqual(orig.__reduce__(), orig.__reduce_ex__(2))
 
         # Try one with a tzinfo.
         tinfo = PicklableFixedOffset(-300, 'cookie')
@@ -2840,6 +2844,7 @@
             self.assertIsInstance(derived.tzinfo, PicklableFixedOffset)
             self.assertEqual(derived.utcoffset(), timedelta(minutes=-300))
             self.assertEqual(derived.tzname(), 'cookie')
+        self.assertEqual(orig.__reduce__(), orig.__reduce_ex__(2))
 
     def test_more_bool(self):
         # time is always True.
@@ -3043,6 +3048,7 @@
             green = pickler.dumps(orig, proto)
             derived = unpickler.loads(green)
             self.assertEqual(orig, derived)
+        self.assertEqual(orig.__reduce__(), orig.__reduce_ex__(2))
 
         # Try one with a tzinfo.
         tinfo = PicklableFixedOffset(-300, 'cookie')
@@ -3055,6 +3061,7 @@
             self.assertIsInstance(derived.tzinfo, PicklableFixedOffset)
             self.assertEqual(derived.utcoffset(), timedelta(minutes=-300))
             self.assertEqual(derived.tzname(), 'cookie')
+        self.assertEqual(orig.__reduce__(), orig.__reduce_ex__(2))
 
     def test_extreme_hashes(self):
         # If an attempt is made to hash these via subtracting the offset
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -42,6 +42,8 @@
 Library
 -------
 
+- Issue #28752: Restored the __reduce__() methods of datetime objects.
+
 - Issue #28727: Regular expression patterns, _sre.SRE_Pattern objects created
   by re.compile(), become comparable (only x==y and x!=y operators). This
   change should fix the issue #18383: don't duplicate warning filters when the
diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c
--- a/Modules/_datetimemodule.c
+++ b/Modules/_datetimemodule.c
@@ -3955,15 +3955,21 @@
 }
 
 static PyObject *
-time_reduce(PyDateTime_Time *self, PyObject *args)
-{
-    int proto = 0;
-    if (!PyArg_ParseTuple(args, "|i:__reduce_ex__", &proto))
+time_reduce_ex(PyDateTime_Time *self, PyObject *args)
+{
+    int proto;
+    if (!PyArg_ParseTuple(args, "i:__reduce_ex__", &proto))
         return NULL;
 
     return Py_BuildValue("(ON)", Py_TYPE(self), time_getstate(self, proto));
 }
 
+static PyObject *
+time_reduce(PyDateTime_Time *self, PyObject *arg)
+{
+    return Py_BuildValue("(ON)", Py_TYPE(self), time_getstate(self, 2));
+}
+
 static PyMethodDef time_methods[] = {
 
     {"isoformat",   (PyCFunction)time_isoformat,        METH_VARARGS | METH_KEYWORDS,
@@ -3989,9 +3995,12 @@
     {"replace",     (PyCFunction)time_replace,          METH_VARARGS | METH_KEYWORDS,
      PyDoc_STR("Return time with new specified fields.")},
 
-    {"__reduce_ex__", (PyCFunction)time_reduce,        METH_VARARGS,
+    {"__reduce_ex__", (PyCFunction)time_reduce_ex,        METH_VARARGS,
      PyDoc_STR("__reduce_ex__(proto) -> (cls, state)")},
 
+    {"__reduce__", (PyCFunction)time_reduce,        METH_NOARGS,
+     PyDoc_STR("__reduce__() -> (cls, state)")},
+
     {NULL,      NULL}
 };
 
@@ -5420,15 +5429,21 @@
 }
 
 static PyObject *
-datetime_reduce(PyDateTime_DateTime *self, PyObject *args)
-{
-    int proto = 0;
-    if (!PyArg_ParseTuple(args, "|i:__reduce_ex__", &proto))
+datetime_reduce_ex(PyDateTime_DateTime *self, PyObject *args)
+{
+    int proto;
+    if (!PyArg_ParseTuple(args, "i:__reduce_ex__", &proto))
         return NULL;
 
     return Py_BuildValue("(ON)", Py_TYPE(self), datetime_getstate(self, proto));
 }
 
+static PyObject *
+datetime_reduce(PyDateTime_DateTime *self, PyObject *arg)
+{
+    return Py_BuildValue("(ON)", Py_TYPE(self), datetime_getstate(self, 2));
+}
+
 static PyMethodDef datetime_methods[] = {
 
     /* Class methods: */
@@ -5503,9 +5518,12 @@
     {"astimezone",  (PyCFunction)datetime_astimezone, METH_VARARGS | METH_KEYWORDS,
      PyDoc_STR("tz -> convert to local time in new timezone tz\n")},
 
-    {"__reduce_ex__", (PyCFunction)datetime_reduce,     METH_VARARGS,
+    {"__reduce_ex__", (PyCFunction)datetime_reduce_ex,     METH_VARARGS,
      PyDoc_STR("__reduce_ex__(proto) -> (cls, state)")},
 
+    {"__reduce__", (PyCFunction)datetime_reduce,     METH_NOARGS,
+     PyDoc_STR("__reduce__() -> (cls, state)")},
+
     {NULL,      NULL}
 };
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 17:34:57 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 22:34:57 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgMjg0NzU6?=
 =?utf-8?q?__Improve_error_message_for_random=2Esample=28=29_with_k_=3C_0?=
 =?utf-8?q?=2E?=
Message-ID: <20161121223457.3004.51445.940F3CFC@psf.io>

https://hg.python.org/cpython/rev/89f95c78e0ab
changeset:   105306:89f95c78e0ab
branch:      3.6
parent:      105304:0a2a0061e425
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 14:34:33 2016 -0800
summary:
  Issue 28475:  Improve error message for random.sample() with k < 0. (Contributed by Francisco Couzo).

files:
  Lib/random.py           |  2 +-
  Lib/test/test_random.py |  1 +
  2 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/random.py b/Lib/random.py
--- a/Lib/random.py
+++ b/Lib/random.py
@@ -314,7 +314,7 @@
         randbelow = self._randbelow
         n = len(population)
         if not 0 <= k <= n:
-            raise ValueError("Sample larger than population")
+            raise ValueError("Sample larger than population or is negative")
         result = [None] * k
         setsize = 21        # size of a small set minus size of an empty list
         if k > 5:
diff --git a/Lib/test/test_random.py b/Lib/test/test_random.py
--- a/Lib/test/test_random.py
+++ b/Lib/test/test_random.py
@@ -110,6 +110,7 @@
         self.assertEqual(self.gen.sample([], 0), [])  # test edge case N==k==0
         # Exception raised if size of sample exceeds that of population
         self.assertRaises(ValueError, self.gen.sample, population, N+1)
+        self.assertRaises(ValueError, self.gen.sample, [], -1)
 
     def test_sample_distribution(self):
         # For the entire allowable range of 0 <= k <= N, validate that

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 17:34:57 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 22:34:57 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161121223457.9047.52434.342CCBB8@psf.io>

https://hg.python.org/cpython/rev/671c032e4fd9
changeset:   105307:671c032e4fd9
parent:      105305:23140bd66d86
parent:      105306:89f95c78e0ab
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 14:34:51 2016 -0800
summary:
  merge

files:
  Lib/random.py           |  2 +-
  Lib/test/test_random.py |  1 +
  2 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/random.py b/Lib/random.py
--- a/Lib/random.py
+++ b/Lib/random.py
@@ -314,7 +314,7 @@
         randbelow = self._randbelow
         n = len(population)
         if not 0 <= k <= n:
-            raise ValueError("Sample larger than population")
+            raise ValueError("Sample larger than population or is negative")
         result = [None] * k
         setsize = 21        # size of a small set minus size of an empty list
         if k > 5:
diff --git a/Lib/test/test_random.py b/Lib/test/test_random.py
--- a/Lib/test/test_random.py
+++ b/Lib/test/test_random.py
@@ -110,6 +110,7 @@
         self.assertEqual(self.gen.sample([], 0), [])  # test edge case N==k==0
         # Exception raised if size of sample exceeds that of population
         self.assertRaises(ValueError, self.gen.sample, population, N+1)
+        self.assertRaises(ValueError, self.gen.sample, [], -1)
 
     def test_sample_distribution(self):
         # For the entire allowable range of 0 <= k <= N, validate that

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 17:48:02 2016
From: python-checkins at python.org (yury.selivanov)
Date: Mon, 21 Nov 2016 22:48:02 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NjUy?=
 =?utf-8?q?=3A_Partially_rollback_previous_changes?=
Message-ID: <20161121224802.2934.60440.76C21E2B@psf.io>

https://hg.python.org/cpython/rev/0ee76f3afd70
changeset:   105308:0ee76f3afd70
branch:      3.5
parent:      105283:75b1091594f8
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 21 17:47:27 2016 -0500
summary:
  Issue #28652: Partially rollback previous changes

Allow AF_UNIX in create_server & create_connection

files:
  Lib/asyncio/base_events.py                |  22 +++++------
  Lib/test/test_asyncio/test_base_events.py |  10 ++---
  2 files changed, 14 insertions(+), 18 deletions(-)


diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -98,14 +98,6 @@
     return (sock.type & socket.SOCK_DGRAM) == socket.SOCK_DGRAM
 
 
-def _is_ip_socket(sock):
-    if sock.family == socket.AF_INET:
-        return True
-    if hasattr(socket, 'AF_INET6') and sock.family == socket.AF_INET6:
-        return True
-    return False
-
-
 def _ipaddr_info(host, port, family, type, proto):
     # Try to skip getaddrinfo if "host" is already an IP. Users might have
     # handled name resolution in their own code and pass in resolved IPs.
@@ -795,9 +787,15 @@
             if sock is None:
                 raise ValueError(
                     'host and port was not specified and no sock specified')
-            if not _is_stream_socket(sock) or not _is_ip_socket(sock):
+            if not _is_stream_socket(sock):
+                # We allow AF_INET, AF_INET6, AF_UNIX as long as they
+                # are SOCK_STREAM.
+                # We support passing AF_UNIX sockets even though we have
+                # a dedicated API for that: create_unix_connection.
+                # Disallowing AF_UNIX in this method, breaks backwards
+                # compatibility.
                 raise ValueError(
-                    'A TCP Stream Socket was expected, got {!r}'.format(sock))
+                    'A Stream Socket was expected, got {!r}'.format(sock))
 
         transport, protocol = yield from self._create_connection_transport(
             sock, protocol_factory, ssl, server_hostname)
@@ -1054,9 +1052,9 @@
         else:
             if sock is None:
                 raise ValueError('Neither host/port nor sock were specified')
-            if not _is_stream_socket(sock) or not _is_ip_socket(sock):
+            if not _is_stream_socket(sock):
                 raise ValueError(
-                    'A TCP Stream Socket was expected, got {!r}'.format(sock))
+                    'A Stream Socket was expected, got {!r}'.format(sock))
             sockets = [sock]
 
         server = Server(self, sockets)
diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py
--- a/Lib/test/test_asyncio/test_base_events.py
+++ b/Lib/test/test_asyncio/test_base_events.py
@@ -1047,22 +1047,20 @@
             MyProto, 'example.com', 80, sock=object())
         self.assertRaises(ValueError, self.loop.run_until_complete, coro)
 
-    @unittest.skipUnless(hasattr(socket, 'AF_UNIX'), 'no Unix sockets')
     def test_create_connection_wrong_sock(self):
-        sock = socket.socket(socket.AF_UNIX)
+        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
         with sock:
             coro = self.loop.create_connection(MyProto, sock=sock)
             with self.assertRaisesRegex(ValueError,
-                                        'A TCP Stream Socket was expected'):
+                                        'A Stream Socket was expected'):
                 self.loop.run_until_complete(coro)
 
-    @unittest.skipUnless(hasattr(socket, 'AF_UNIX'), 'no Unix sockets')
     def test_create_server_wrong_sock(self):
-        sock = socket.socket(socket.AF_UNIX)
+        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
         with sock:
             coro = self.loop.create_server(MyProto, sock=sock)
             with self.assertRaisesRegex(ValueError,
-                                        'A TCP Stream Socket was expected'):
+                                        'A Stream Socket was expected'):
                 self.loop.run_until_complete(coro)
 
     @unittest.skipUnless(hasattr(socket, 'SOCK_NONBLOCK'),

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 17:48:02 2016
From: python-checkins at python.org (yury.selivanov)
Date: Mon, 21 Nov 2016 22:48:02 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5_=28issue_=2328652=29?=
Message-ID: <20161121224802.33279.55749.5A29B94E@psf.io>

https://hg.python.org/cpython/rev/a40159071359
changeset:   105309:a40159071359
branch:      3.6
parent:      105306:89f95c78e0ab
parent:      105308:0ee76f3afd70
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 21 17:47:41 2016 -0500
summary:
  Merge 3.5 (issue #28652)

files:
  Lib/asyncio/base_events.py                |  22 +++++------
  Lib/test/test_asyncio/test_base_events.py |  10 ++---
  2 files changed, 14 insertions(+), 18 deletions(-)


diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -98,14 +98,6 @@
     return (sock.type & socket.SOCK_DGRAM) == socket.SOCK_DGRAM
 
 
-def _is_ip_socket(sock):
-    if sock.family == socket.AF_INET:
-        return True
-    if hasattr(socket, 'AF_INET6') and sock.family == socket.AF_INET6:
-        return True
-    return False
-
-
 def _ipaddr_info(host, port, family, type, proto):
     # Try to skip getaddrinfo if "host" is already an IP. Users might have
     # handled name resolution in their own code and pass in resolved IPs.
@@ -796,9 +788,15 @@
             if sock is None:
                 raise ValueError(
                     'host and port was not specified and no sock specified')
-            if not _is_stream_socket(sock) or not _is_ip_socket(sock):
+            if not _is_stream_socket(sock):
+                # We allow AF_INET, AF_INET6, AF_UNIX as long as they
+                # are SOCK_STREAM.
+                # We support passing AF_UNIX sockets even though we have
+                # a dedicated API for that: create_unix_connection.
+                # Disallowing AF_UNIX in this method, breaks backwards
+                # compatibility.
                 raise ValueError(
-                    'A TCP Stream Socket was expected, got {!r}'.format(sock))
+                    'A Stream Socket was expected, got {!r}'.format(sock))
 
         transport, protocol = yield from self._create_connection_transport(
             sock, protocol_factory, ssl, server_hostname)
@@ -1055,9 +1053,9 @@
         else:
             if sock is None:
                 raise ValueError('Neither host/port nor sock were specified')
-            if not _is_stream_socket(sock) or not _is_ip_socket(sock):
+            if not _is_stream_socket(sock):
                 raise ValueError(
-                    'A TCP Stream Socket was expected, got {!r}'.format(sock))
+                    'A Stream Socket was expected, got {!r}'.format(sock))
             sockets = [sock]
 
         server = Server(self, sockets)
diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py
--- a/Lib/test/test_asyncio/test_base_events.py
+++ b/Lib/test/test_asyncio/test_base_events.py
@@ -1047,22 +1047,20 @@
             MyProto, 'example.com', 80, sock=object())
         self.assertRaises(ValueError, self.loop.run_until_complete, coro)
 
-    @unittest.skipUnless(hasattr(socket, 'AF_UNIX'), 'no Unix sockets')
     def test_create_connection_wrong_sock(self):
-        sock = socket.socket(socket.AF_UNIX)
+        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
         with sock:
             coro = self.loop.create_connection(MyProto, sock=sock)
             with self.assertRaisesRegex(ValueError,
-                                        'A TCP Stream Socket was expected'):
+                                        'A Stream Socket was expected'):
                 self.loop.run_until_complete(coro)
 
-    @unittest.skipUnless(hasattr(socket, 'AF_UNIX'), 'no Unix sockets')
     def test_create_server_wrong_sock(self):
-        sock = socket.socket(socket.AF_UNIX)
+        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
         with sock:
             coro = self.loop.create_server(MyProto, sock=sock)
             with self.assertRaisesRegex(ValueError,
-                                        'A TCP Stream Socket was expected'):
+                                        'A Stream Socket was expected'):
                 self.loop.run_until_complete(coro)
 
     @unittest.skipUnless(hasattr(socket, 'SOCK_NONBLOCK'),

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 17:48:02 2016
From: python-checkins at python.org (yury.selivanov)
Date: Mon, 21 Nov 2016 22:48:02 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg2NTIp?=
Message-ID: <20161121224802.33817.49182.00F35A66@psf.io>

https://hg.python.org/cpython/rev/a759fcba1866
changeset:   105310:a759fcba1866
parent:      105307:671c032e4fd9
parent:      105309:a40159071359
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 21 17:47:54 2016 -0500
summary:
  Merge 3.6 (issue #28652)

files:
  Lib/asyncio/base_events.py                |  22 +++++------
  Lib/test/test_asyncio/test_base_events.py |  10 ++---
  2 files changed, 14 insertions(+), 18 deletions(-)


diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -98,14 +98,6 @@
     return (sock.type & socket.SOCK_DGRAM) == socket.SOCK_DGRAM
 
 
-def _is_ip_socket(sock):
-    if sock.family == socket.AF_INET:
-        return True
-    if hasattr(socket, 'AF_INET6') and sock.family == socket.AF_INET6:
-        return True
-    return False
-
-
 def _ipaddr_info(host, port, family, type, proto):
     # Try to skip getaddrinfo if "host" is already an IP. Users might have
     # handled name resolution in their own code and pass in resolved IPs.
@@ -796,9 +788,15 @@
             if sock is None:
                 raise ValueError(
                     'host and port was not specified and no sock specified')
-            if not _is_stream_socket(sock) or not _is_ip_socket(sock):
+            if not _is_stream_socket(sock):
+                # We allow AF_INET, AF_INET6, AF_UNIX as long as they
+                # are SOCK_STREAM.
+                # We support passing AF_UNIX sockets even though we have
+                # a dedicated API for that: create_unix_connection.
+                # Disallowing AF_UNIX in this method, breaks backwards
+                # compatibility.
                 raise ValueError(
-                    'A TCP Stream Socket was expected, got {!r}'.format(sock))
+                    'A Stream Socket was expected, got {!r}'.format(sock))
 
         transport, protocol = yield from self._create_connection_transport(
             sock, protocol_factory, ssl, server_hostname)
@@ -1055,9 +1053,9 @@
         else:
             if sock is None:
                 raise ValueError('Neither host/port nor sock were specified')
-            if not _is_stream_socket(sock) or not _is_ip_socket(sock):
+            if not _is_stream_socket(sock):
                 raise ValueError(
-                    'A TCP Stream Socket was expected, got {!r}'.format(sock))
+                    'A Stream Socket was expected, got {!r}'.format(sock))
             sockets = [sock]
 
         server = Server(self, sockets)
diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py
--- a/Lib/test/test_asyncio/test_base_events.py
+++ b/Lib/test/test_asyncio/test_base_events.py
@@ -1047,22 +1047,20 @@
             MyProto, 'example.com', 80, sock=object())
         self.assertRaises(ValueError, self.loop.run_until_complete, coro)
 
-    @unittest.skipUnless(hasattr(socket, 'AF_UNIX'), 'no Unix sockets')
     def test_create_connection_wrong_sock(self):
-        sock = socket.socket(socket.AF_UNIX)
+        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
         with sock:
             coro = self.loop.create_connection(MyProto, sock=sock)
             with self.assertRaisesRegex(ValueError,
-                                        'A TCP Stream Socket was expected'):
+                                        'A Stream Socket was expected'):
                 self.loop.run_until_complete(coro)
 
-    @unittest.skipUnless(hasattr(socket, 'AF_UNIX'), 'no Unix sockets')
     def test_create_server_wrong_sock(self):
-        sock = socket.socket(socket.AF_UNIX)
+        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
         with sock:
             coro = self.loop.create_server(MyProto, sock=sock)
             with self.assertRaisesRegex(ValueError,
-                                        'A TCP Stream Socket was expected'):
+                                        'A Stream Socket was expected'):
                 self.loop.run_until_complete(coro)
 
     @unittest.skipUnless(hasattr(socket, 'SOCK_NONBLOCK'),

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 18:13:44 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 23:13:44 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgMjg1ODc6?=
 =?utf-8?q?__list=2Eindex_documentation_missing_start_and_stop_arguments?=
 =?utf-8?q?=2E?=
Message-ID: <20161121231344.33323.15742.2555BF6C@psf.io>

https://hg.python.org/cpython/rev/62c16fafa7d4
changeset:   105311:62c16fafa7d4
branch:      3.6
parent:      105309:a40159071359
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 15:12:54 2016 -0800
summary:
  Issue 28587:  list.index documentation missing start and stop arguments. (Contributed by Mariatta Wijaya.)

files:
  Doc/tutorial/datastructures.rst |  13 ++++++++++---
  1 files changed, 10 insertions(+), 3 deletions(-)


diff --git a/Doc/tutorial/datastructures.rst b/Doc/tutorial/datastructures.rst
--- a/Doc/tutorial/datastructures.rst
+++ b/Doc/tutorial/datastructures.rst
@@ -60,11 +60,16 @@
    Remove all items from the list.  Equivalent to ``del a[:]``.
 
 
-.. method:: list.index(x)
+.. method:: list.index(x[, start[, end]])
    :noindex:
 
-   Return the index in the list of the first item whose value is *x*. It is an
-   error if there is no such item.
+   Return zero-based index in the list of the first item whose value is *x*.
+   Raises a :exc:`ValueError` if there is no such item.
+
+   The optional arguments *start* and *end* are interpreted as in the slice
+   notation and are used to limit the search to a particular subsequence of
+   *x*.  The returned index is computed relative to the beginning of the full
+   sequence rather than the *start* argument.
 
 
 .. method:: list.count(x)
@@ -103,6 +108,8 @@
    [66.25, 333, -1, 333, 1, 1234.5, 333]
    >>> a.index(333)
    1
+   >>> a.index(333, 2)  # search for 333 starting at index 2
+   2
    >>> a.remove(333)
    >>> a
    [66.25, -1, 333, 1, 1234.5, 333]

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 18:13:44 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 23:13:44 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Fix_grammar?=
Message-ID: <20161121231344.130124.86597.F74EB32E@psf.io>

https://hg.python.org/cpython/rev/784b6a0bf2bf
changeset:   105312:784b6a0bf2bf
branch:      3.6
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 15:13:18 2016 -0800
summary:
  Fix grammar

files:
  Doc/library/random.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -465,11 +465,11 @@
    <http://nbviewer.jupyter.org/url/norvig.com/ipython/Economics.ipynb>`_
    a simulation of a marketplace by
    `Peter Norvig <http://norvig.com/bio.html>`_ that shows effective
-   use of many the tools and distributions provided by this module
+   use of many of the tools and distributions provided by this module
    (gauss, uniform, sample, betavariate, choice, triangular, and randrange).
 
    `A Concrete Introduction to Probability (using Python)
    <http://nbviewer.jupyter.org/url/norvig.com/ipython/Probability.ipynb>`_
    a tutorial by `Peter Norvig <http://norvig.com/bio.html>`_ covering
    the basics of probability theory, how to write simulations, and
-   performing data analysis using Python.
+   how to perform data analysis using Python.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 18:13:45 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 23:13:45 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161121231344.33205.42456.07EB959F@psf.io>

https://hg.python.org/cpython/rev/5e289a846d9d
changeset:   105313:5e289a846d9d
parent:      105310:a759fcba1866
parent:      105312:784b6a0bf2bf
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 15:13:38 2016 -0800
summary:
  merge

files:
  Doc/library/random.rst          |   4 ++--
  Doc/tutorial/datastructures.rst |  13 ++++++++++---
  2 files changed, 12 insertions(+), 5 deletions(-)


diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -465,11 +465,11 @@
    <http://nbviewer.jupyter.org/url/norvig.com/ipython/Economics.ipynb>`_
    a simulation of a marketplace by
    `Peter Norvig <http://norvig.com/bio.html>`_ that shows effective
-   use of many the tools and distributions provided by this module
+   use of many of the tools and distributions provided by this module
    (gauss, uniform, sample, betavariate, choice, triangular, and randrange).
 
    `A Concrete Introduction to Probability (using Python)
    <http://nbviewer.jupyter.org/url/norvig.com/ipython/Probability.ipynb>`_
    a tutorial by `Peter Norvig <http://norvig.com/bio.html>`_ covering
    the basics of probability theory, how to write simulations, and
-   performing data analysis using Python.
+   how to perform data analysis using Python.
diff --git a/Doc/tutorial/datastructures.rst b/Doc/tutorial/datastructures.rst
--- a/Doc/tutorial/datastructures.rst
+++ b/Doc/tutorial/datastructures.rst
@@ -60,11 +60,16 @@
    Remove all items from the list.  Equivalent to ``del a[:]``.
 
 
-.. method:: list.index(x)
+.. method:: list.index(x[, start[, end]])
    :noindex:
 
-   Return the index in the list of the first item whose value is *x*. It is an
-   error if there is no such item.
+   Return zero-based index in the list of the first item whose value is *x*.
+   Raises a :exc:`ValueError` if there is no such item.
+
+   The optional arguments *start* and *end* are interpreted as in the slice
+   notation and are used to limit the search to a particular subsequence of
+   *x*.  The returned index is computed relative to the beginning of the full
+   sequence rather than the *start* argument.
 
 
 .. method:: list.count(x)
@@ -103,6 +108,8 @@
    [66.25, 333, -1, 333, 1, 1234.5, 333]
    >>> a.index(333)
    1
+   >>> a.index(333, 2)  # search for 333 starting at index 2
+   2
    >>> a.remove(333)
    >>> a
    [66.25, -1, 333, 1, 1234.5, 333]

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 18:32:32 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 23:32:32 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161121233232.33382.83950.DB8C753F@psf.io>

https://hg.python.org/cpython/rev/9c7072af82b4
changeset:   105315:9c7072af82b4
parent:      105313:5e289a846d9d
parent:      105314:3551fca2c6ae
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 15:32:26 2016 -0800
summary:
  merge

files:
  Lib/test/test_random.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_random.py b/Lib/test/test_random.py
--- a/Lib/test/test_random.py
+++ b/Lib/test/test_random.py
@@ -629,7 +629,7 @@
     def test_choices_algorithms(self):
         # The various ways of specifying weights should produce the same results
         choices = self.gen.choices
-        n = 13132817
+        n = 104729
 
         self.gen.seed(8675309)
         a = self.gen.choices(range(n), k=10000)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 18:32:32 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 21 Nov 2016 23:32:32 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NzQz?=
 =?utf-8?q?=3A__Reduce_memory_consumption_for_random_module_tests?=
Message-ID: <20161121233231.2600.84581.34BA875A@psf.io>

https://hg.python.org/cpython/rev/3551fca2c6ae
changeset:   105314:3551fca2c6ae
branch:      3.6
parent:      105312:784b6a0bf2bf
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 15:32:08 2016 -0800
summary:
  Issue #28743:  Reduce memory consumption for random module tests

files:
  Lib/test/test_random.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_random.py b/Lib/test/test_random.py
--- a/Lib/test/test_random.py
+++ b/Lib/test/test_random.py
@@ -633,7 +633,7 @@
     def test_choices_algorithms(self):
         # The various ways of specifying weights should produce the same results
         choices = self.gen.choices
-        n = 13132817
+        n = 104729
 
         self.gen.seed(8675309)
         a = self.gen.choices(range(n), k=10000)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 19:32:03 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 22 Nov 2016 00:32:03 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161122003203.130239.83442.5553FF7A@psf.io>

https://hg.python.org/cpython/rev/9eef500e239c
changeset:   105318:9eef500e239c
parent:      105315:9c7072af82b4
parent:      105317:71dd21a3b9cc
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 16:31:32 2016 -0800
summary:
  merge

files:
  Doc/library/statistics.rst      |  10 ++--
  Doc/tutorial/datastructures.rst |  44 +++++++++-----------
  2 files changed, 25 insertions(+), 29 deletions(-)


diff --git a/Doc/library/statistics.rst b/Doc/library/statistics.rst
--- a/Doc/library/statistics.rst
+++ b/Doc/library/statistics.rst
@@ -69,8 +69,7 @@
 
 .. function:: mean(data)
 
-   Return the sample arithmetic mean of *data*, a sequence or iterator of
-   real-valued numbers.
+   Return the sample arithmetic mean of *data* which can be a sequence or iterator.
 
    The arithmetic mean is the sum of the data divided by the number of data
    points.  It is commonly called "the average", although it is only one of many
@@ -148,6 +147,7 @@
 
    Return the median (middle value) of numeric data, using the common "mean of
    middle two" method.  If *data* is empty, :exc:`StatisticsError` is raised.
+   *data* can be a sequence or iterator.
 
    The median is a robust measure of central location, and is less affected by
    the presence of outliers in your data.  When the number of data points is
@@ -175,7 +175,7 @@
 .. function:: median_low(data)
 
    Return the low median of numeric data.  If *data* is empty,
-   :exc:`StatisticsError` is raised.
+   :exc:`StatisticsError` is raised.  *data* can be a sequence or iterator.
 
    The low median is always a member of the data set.  When the number of data
    points is odd, the middle value is returned.  When it is even, the smaller of
@@ -195,7 +195,7 @@
 .. function:: median_high(data)
 
    Return the high median of data.  If *data* is empty, :exc:`StatisticsError`
-   is raised.
+   is raised.  *data* can be a sequence or iterator.
 
    The high median is always a member of the data set.  When the number of data
    points is odd, the middle value is returned.  When it is even, the larger of
@@ -216,7 +216,7 @@
 
    Return the median of grouped continuous data, calculated as the 50th
    percentile, using interpolation.  If *data* is empty, :exc:`StatisticsError`
-   is raised.
+   is raised.  *data* can be a sequence or iterator.
 
    .. doctest::
 
diff --git a/Doc/tutorial/datastructures.rst b/Doc/tutorial/datastructures.rst
--- a/Doc/tutorial/datastructures.rst
+++ b/Doc/tutorial/datastructures.rst
@@ -99,30 +99,26 @@
 
 An example that uses most of the list methods::
 
-   >>> a = [66.25, 333, 333, 1, 1234.5]
-   >>> print(a.count(333), a.count(66.25), a.count('x'))
-   2 1 0
-   >>> a.insert(2, -1)
-   >>> a.append(333)
-   >>> a
-   [66.25, 333, -1, 333, 1, 1234.5, 333]
-   >>> a.index(333)
-   1
-   >>> a.index(333, 2)  # search for 333 starting at index 2
-   2
-   >>> a.remove(333)
-   >>> a
-   [66.25, -1, 333, 1, 1234.5, 333]
-   >>> a.reverse()
-   >>> a
-   [333, 1234.5, 1, 333, -1, 66.25]
-   >>> a.sort()
-   >>> a
-   [-1, 1, 66.25, 333, 333, 1234.5]
-   >>> a.pop()
-   1234.5
-   >>> a
-   [-1, 1, 66.25, 333, 333]
+    >>> fruits = ['orange', 'apple', 'pear', 'banana', 'kiwi', 'apple', 'banana']
+    >>> fruits.count('apple')
+    2
+    >>> fruits.count('tangerine')
+    0
+    >>> fruits.index('banana')
+    3
+    >>> fruits.index('banana', 4)  # Find next banana starting a position 4
+    6
+    >>> fruits.reverse()
+    >>> fruits
+    ['banana', 'apple', 'kiwi', 'banana', 'pear', 'apple', 'orange']
+    >>> fruits.append('grape')
+    >>> fruits
+    ['banana', 'apple', 'kiwi', 'banana', 'pear', 'apple', 'orange', 'grape']
+    >>> fruits.sort()
+    >>> fruits
+    ['apple', 'apple', 'banana', 'banana', 'grape', 'kiwi', 'orange', 'pear']
+    >>> fruits.pop()
+    'pear'
 
 You might have noticed that methods like ``insert``, ``remove`` or ``sort`` that
 only modify the list have no return value printed -- they return the default

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 19:32:03 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 22 Nov 2016 00:32:03 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NTg3?=
 =?utf-8?q?=3A_Improve_list_examples_in_the_tutorial?=
Message-ID: <20161122003203.130095.69029.0F75D5FB@psf.io>

https://hg.python.org/cpython/rev/efac7ac53933
changeset:   105316:efac7ac53933
branch:      3.6
parent:      105314:3551fca2c6ae
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 16:29:50 2016 -0800
summary:
  Issue #28587: Improve list examples in the tutorial

files:
  Doc/tutorial/datastructures.rst |  44 +++++++++-----------
  1 files changed, 20 insertions(+), 24 deletions(-)


diff --git a/Doc/tutorial/datastructures.rst b/Doc/tutorial/datastructures.rst
--- a/Doc/tutorial/datastructures.rst
+++ b/Doc/tutorial/datastructures.rst
@@ -99,30 +99,26 @@
 
 An example that uses most of the list methods::
 
-   >>> a = [66.25, 333, 333, 1, 1234.5]
-   >>> print(a.count(333), a.count(66.25), a.count('x'))
-   2 1 0
-   >>> a.insert(2, -1)
-   >>> a.append(333)
-   >>> a
-   [66.25, 333, -1, 333, 1, 1234.5, 333]
-   >>> a.index(333)
-   1
-   >>> a.index(333, 2)  # search for 333 starting at index 2
-   2
-   >>> a.remove(333)
-   >>> a
-   [66.25, -1, 333, 1, 1234.5, 333]
-   >>> a.reverse()
-   >>> a
-   [333, 1234.5, 1, 333, -1, 66.25]
-   >>> a.sort()
-   >>> a
-   [-1, 1, 66.25, 333, 333, 1234.5]
-   >>> a.pop()
-   1234.5
-   >>> a
-   [-1, 1, 66.25, 333, 333]
+    >>> fruits = ['orange', 'apple', 'pear', 'banana', 'kiwi', 'apple', 'banana']
+    >>> fruits.count('apple')
+    2
+    >>> fruits.count('tangerine')
+    0
+    >>> fruits.index('banana')
+    3
+    >>> fruits.index('banana', 4)  # Find next banana starting a position 4
+    6
+    >>> fruits.reverse()
+    >>> fruits
+    ['banana', 'apple', 'kiwi', 'banana', 'pear', 'apple', 'orange']
+    >>> fruits.append('grape')
+    >>> fruits
+    ['banana', 'apple', 'kiwi', 'banana', 'pear', 'apple', 'orange', 'grape']
+    >>> fruits.sort()
+    >>> fruits
+    ['apple', 'apple', 'banana', 'banana', 'grape', 'kiwi', 'orange', 'pear']
+    >>> fruits.pop()
+    'pear'
 
 You might have noticed that methods like ``insert``, ``remove`` or ``sort`` that
 only modify the list have no return value printed -- they return the default

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 19:32:03 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 22 Nov 2016 00:32:03 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI3ODI1?=
 =?utf-8?q?=3A_Improve_for_statistics_data_arguments=2E_=28Contributed_by_?=
 =?utf-8?q?Mariatta?=
Message-ID: <20161122003203.3263.43757.09ADCADC@psf.io>

https://hg.python.org/cpython/rev/71dd21a3b9cc
changeset:   105317:71dd21a3b9cc
branch:      3.6
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 16:31:02 2016 -0800
summary:
  Issue #27825: Improve for statistics data arguments. (Contributed by Mariatta Wijaya.)

files:
  Doc/library/statistics.rst |  10 +++++-----
  1 files changed, 5 insertions(+), 5 deletions(-)


diff --git a/Doc/library/statistics.rst b/Doc/library/statistics.rst
--- a/Doc/library/statistics.rst
+++ b/Doc/library/statistics.rst
@@ -69,8 +69,7 @@
 
 .. function:: mean(data)
 
-   Return the sample arithmetic mean of *data*, a sequence or iterator of
-   real-valued numbers.
+   Return the sample arithmetic mean of *data* which can be a sequence or iterator.
 
    The arithmetic mean is the sum of the data divided by the number of data
    points.  It is commonly called "the average", although it is only one of many
@@ -148,6 +147,7 @@
 
    Return the median (middle value) of numeric data, using the common "mean of
    middle two" method.  If *data* is empty, :exc:`StatisticsError` is raised.
+   *data* can be a sequence or iterator.
 
    The median is a robust measure of central location, and is less affected by
    the presence of outliers in your data.  When the number of data points is
@@ -175,7 +175,7 @@
 .. function:: median_low(data)
 
    Return the low median of numeric data.  If *data* is empty,
-   :exc:`StatisticsError` is raised.
+   :exc:`StatisticsError` is raised.  *data* can be a sequence or iterator.
 
    The low median is always a member of the data set.  When the number of data
    points is odd, the middle value is returned.  When it is even, the smaller of
@@ -195,7 +195,7 @@
 .. function:: median_high(data)
 
    Return the high median of data.  If *data* is empty, :exc:`StatisticsError`
-   is raised.
+   is raised.  *data* can be a sequence or iterator.
 
    The high median is always a member of the data set.  When the number of data
    points is odd, the middle value is returned.  When it is even, the larger of
@@ -216,7 +216,7 @@
 
    Return the median of grouped continuous data, calculated as the 50th
    percentile, using interpolation.  If *data* is empty, :exc:`StatisticsError`
-   is raised.
+   is raised.  *data* can be a sequence or iterator.
 
    .. doctest::
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 19:48:30 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 22 Nov 2016 00:48:30 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161122004830.33611.89364.1EF426C6@psf.io>

https://hg.python.org/cpython/rev/1b0ffffcb48e
changeset:   105320:1b0ffffcb48e
parent:      105318:9eef500e239c
parent:      105319:ecc6f7940e02
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 16:48:25 2016 -0800
summary:
  merge

files:
  Lib/test/test_sched.py |  22 ++++++++++++++--------
  1 files changed, 14 insertions(+), 8 deletions(-)


diff --git a/Lib/test/test_sched.py b/Lib/test/test_sched.py
--- a/Lib/test/test_sched.py
+++ b/Lib/test/test_sched.py
@@ -172,17 +172,23 @@
         self.assertEqual(scheduler.queue, [e1, e2, e3, e4, e5])
 
     def test_args_kwargs(self):
-        flag = []
+        seq = []
+        def fun(*a, **b):
+            seq.append((a, b))
 
-        def fun(*a, **b):
-            flag.append(None)
-            self.assertEqual(a, (1,2,3))
-            self.assertEqual(b, {"foo":1})
-
+        now = time.time()
         scheduler = sched.scheduler(time.time, time.sleep)
-        z = scheduler.enterabs(0.01, 1, fun, argument=(1,2,3), kwargs={"foo":1})
+        scheduler.enterabs(now, 1, fun)
+        scheduler.enterabs(now, 1, fun, argument=(1, 2))
+        scheduler.enterabs(now, 1, fun, argument=('a', 'b'))
+        scheduler.enterabs(now, 1, fun, argument=(1, 2), kwargs={"foo": 3})
         scheduler.run()
-        self.assertEqual(flag, [None])
+        self.assertCountEqual(seq, [
+            ((), {}),
+            ((1, 2), {}),
+            (('a', 'b'), {}),
+            ((1, 2), {'foo': 3})
+        ])
 
     def test_run_non_blocking(self):
         l = []

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 19:48:30 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 22 Nov 2016 00:48:30 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzU4MzA6?=
 =?utf-8?q?__Add_test_for_ee476248a74a=2E__=28Contributed_by_Serhiy_Storch?=
 =?utf-8?b?YWthLik=?=
Message-ID: <20161122004830.3046.62041.802078AC@psf.io>

https://hg.python.org/cpython/rev/ecc6f7940e02
changeset:   105319:ecc6f7940e02
branch:      3.6
parent:      105317:71dd21a3b9cc
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 16:48:10 2016 -0800
summary:
  Issue #5830:  Add test for ee476248a74a.  (Contributed by Serhiy Storchaka.)

files:
  Lib/test/test_sched.py |  22 ++++++++++++++--------
  1 files changed, 14 insertions(+), 8 deletions(-)


diff --git a/Lib/test/test_sched.py b/Lib/test/test_sched.py
--- a/Lib/test/test_sched.py
+++ b/Lib/test/test_sched.py
@@ -172,17 +172,23 @@
         self.assertEqual(scheduler.queue, [e1, e2, e3, e4, e5])
 
     def test_args_kwargs(self):
-        flag = []
+        seq = []
+        def fun(*a, **b):
+            seq.append((a, b))
 
-        def fun(*a, **b):
-            flag.append(None)
-            self.assertEqual(a, (1,2,3))
-            self.assertEqual(b, {"foo":1})
-
+        now = time.time()
         scheduler = sched.scheduler(time.time, time.sleep)
-        z = scheduler.enterabs(0.01, 1, fun, argument=(1,2,3), kwargs={"foo":1})
+        scheduler.enterabs(now, 1, fun)
+        scheduler.enterabs(now, 1, fun, argument=(1, 2))
+        scheduler.enterabs(now, 1, fun, argument=('a', 'b'))
+        scheduler.enterabs(now, 1, fun, argument=(1, 2), kwargs={"foo": 3})
         scheduler.run()
-        self.assertEqual(flag, [None])
+        self.assertCountEqual(seq, [
+            ((), {}),
+            ((1, 2), {}),
+            (('a', 'b'), {}),
+            ((1, 2), {'foo': 3})
+        ])
 
     def test_run_non_blocking(self):
         l = []

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 19:59:35 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 22 Nov 2016 00:59:35 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161122005935.33742.43703.DEF5B976@psf.io>

https://hg.python.org/cpython/rev/8b767fa186e7
changeset:   105322:8b767fa186e7
parent:      105320:1b0ffffcb48e
parent:      105321:e0f0211d314d
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 16:59:30 2016 -0800
summary:
  merge

files:
  Lib/test/test_set.py |  5 +----
  1 files changed, 1 insertions(+), 4 deletions(-)


diff --git a/Lib/test/test_set.py b/Lib/test/test_set.py
--- a/Lib/test/test_set.py
+++ b/Lib/test/test_set.py
@@ -730,9 +730,6 @@
             addhashvalue(hash(frozenset([e for e, m in elemmasks if m&i])))
         self.assertEqual(len(hashvalues), 2**n)
 
-        def letter_range(n):
-            return string.ascii_letters[:n]
-
         def zf_range(n):
             # https://en.wikipedia.org/wiki/Set-theoretic_definition_of_natural_numbers
             nums = [frozenset()]
@@ -748,7 +745,7 @@
         for n in range(18):
             t = 2 ** n
             mask = t - 1
-            for nums in (range, letter_range, zf_range):
+            for nums in (range, zf_range):
                 u = len({h & mask for h in map(hash, powerset(nums(n)))})
                 self.assertGreater(4*u, t)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 19:59:36 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 22 Nov 2016 00:59:36 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI2MTYz?=
 =?utf-8?q?=3A__Disable_periodically_failing_test_which_was_overly_demandi?=
 =?utf-8?q?ng_of?=
Message-ID: <20161122005935.9235.27711.CD4AF28E@psf.io>

https://hg.python.org/cpython/rev/e0f0211d314d
changeset:   105321:e0f0211d314d
branch:      3.6
parent:      105319:ecc6f7940e02
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 16:59:04 2016 -0800
summary:
  Issue #26163:  Disable periodically failing test which was overly demanding of the frozenset hash function effectiveness

files:
  Lib/test/test_set.py |  5 +----
  1 files changed, 1 insertions(+), 4 deletions(-)


diff --git a/Lib/test/test_set.py b/Lib/test/test_set.py
--- a/Lib/test/test_set.py
+++ b/Lib/test/test_set.py
@@ -730,9 +730,6 @@
             addhashvalue(hash(frozenset([e for e, m in elemmasks if m&i])))
         self.assertEqual(len(hashvalues), 2**n)
 
-        def letter_range(n):
-            return string.ascii_letters[:n]
-
         def zf_range(n):
             # https://en.wikipedia.org/wiki/Set-theoretic_definition_of_natural_numbers
             nums = [frozenset()]
@@ -748,7 +745,7 @@
         for n in range(18):
             t = 2 ** n
             mask = t - 1
-            for nums in (range, letter_range, zf_range):
+            for nums in (range, zf_range):
                 u = len({h & mask for h in map(hash, powerset(nums(n)))})
                 self.assertGreater(4*u, t)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 20:25:04 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 22 Nov 2016 01:25:04 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI3MTAw?=
 =?utf-8?q?=3A__With_statement_reports_missing_=5F=5Fenter=5F=5F_before_?=
 =?utf-8?b?X19leGl0X18u?=
Message-ID: <20161122012504.400.71295.A6B14433@psf.io>

https://hg.python.org/cpython/rev/3aafb232f2db
changeset:   105323:3aafb232f2db
branch:      3.6
parent:      105321:e0f0211d314d
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 17:24:23 2016 -0800
summary:
  Issue #27100:  With statement reports missing __enter__ before __exit__.  (Contributed by Jonathan Ellington.)

files:
  Lib/test/test_with.py |  15 ++++++++++++---
  Misc/NEWS             |   4 ++++
  Python/ceval.c        |   8 ++++----
  3 files changed, 20 insertions(+), 7 deletions(-)


diff --git a/Lib/test/test_with.py b/Lib/test/test_with.py
--- a/Lib/test/test_with.py
+++ b/Lib/test/test_with.py
@@ -109,7 +109,7 @@
             with foo: pass
         self.assertRaises(NameError, fooNotDeclared)
 
-    def testEnterAttributeError(self):
+    def testEnterAttributeError1(self):
         class LacksEnter(object):
             def __exit__(self, type, value, traceback):
                 pass
@@ -117,7 +117,16 @@
         def fooLacksEnter():
             foo = LacksEnter()
             with foo: pass
-        self.assertRaises(AttributeError, fooLacksEnter)
+        self.assertRaisesRegexp(AttributeError, '__enter__', fooLacksEnter)
+
+    def testEnterAttributeError2(self):
+        class LacksEnterAndExit(object):
+            pass
+
+        def fooLacksEnterAndExit():
+            foo = LacksEnterAndExit()
+            with foo: pass
+        self.assertRaisesRegexp(AttributeError, '__enter__', fooLacksEnterAndExit)
 
     def testExitAttributeError(self):
         class LacksExit(object):
@@ -127,7 +136,7 @@
         def fooLacksExit():
             foo = LacksExit()
             with foo: pass
-        self.assertRaises(AttributeError, fooLacksExit)
+        self.assertRaisesRegexp(AttributeError, '__exit__', fooLacksExit)
 
     def assertRaisesSyntaxError(self, codestr):
         def shouldRaiseSyntaxError(s):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -12,6 +12,10 @@
 
 - Issue #28532: Show sys.version when -V option is supplied twice.
 
+- Issue #27100: The with-statement now checks for __enter__ before it
+  checks for __exit__.  This gives less confusing error messages when
+  both methods are missing. Patch by Jonathan Ellington.
+
 - Issue #28746: Fix the set_inheritable() file descriptor method on platforms
   that do not have the ioctl FIOCLEX and FIONCLEX commands.
 
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -3133,15 +3133,15 @@
             _Py_IDENTIFIER(__exit__);
             _Py_IDENTIFIER(__enter__);
             PyObject *mgr = TOP();
-            PyObject *exit = special_lookup(mgr, &PyId___exit__), *enter;
+            PyObject *enter = special_lookup(mgr, &PyId___enter__), *exit;
             PyObject *res;
+            if (enter == NULL)
+                goto error;
+            exit = special_lookup(mgr, &PyId___exit__);
             if (exit == NULL)
                 goto error;
             SET_TOP(exit);
-            enter = special_lookup(mgr, &PyId___enter__);
             Py_DECREF(mgr);
-            if (enter == NULL)
-                goto error;
             res = PyObject_CallFunctionObjArgs(enter, NULL);
             Py_DECREF(enter);
             if (res == NULL)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 21 20:25:08 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 22 Nov 2016 01:25:08 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161122012504.18803.84923.4CB60923@psf.io>

https://hg.python.org/cpython/rev/a0da176233b9
changeset:   105324:a0da176233b9
parent:      105322:8b767fa186e7
parent:      105323:3aafb232f2db
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Nov 21 17:24:58 2016 -0800
summary:
  merge

files:
  Lib/test/test_with.py |  15 ++++++++++++---
  Misc/NEWS             |   4 ++++
  Python/ceval.c        |   8 ++++----
  3 files changed, 20 insertions(+), 7 deletions(-)


diff --git a/Lib/test/test_with.py b/Lib/test/test_with.py
--- a/Lib/test/test_with.py
+++ b/Lib/test/test_with.py
@@ -109,7 +109,7 @@
             with foo: pass
         self.assertRaises(NameError, fooNotDeclared)
 
-    def testEnterAttributeError(self):
+    def testEnterAttributeError1(self):
         class LacksEnter(object):
             def __exit__(self, type, value, traceback):
                 pass
@@ -117,7 +117,16 @@
         def fooLacksEnter():
             foo = LacksEnter()
             with foo: pass
-        self.assertRaises(AttributeError, fooLacksEnter)
+        self.assertRaisesRegexp(AttributeError, '__enter__', fooLacksEnter)
+
+    def testEnterAttributeError2(self):
+        class LacksEnterAndExit(object):
+            pass
+
+        def fooLacksEnterAndExit():
+            foo = LacksEnterAndExit()
+            with foo: pass
+        self.assertRaisesRegexp(AttributeError, '__enter__', fooLacksEnterAndExit)
 
     def testExitAttributeError(self):
         class LacksExit(object):
@@ -127,7 +136,7 @@
         def fooLacksExit():
             foo = LacksExit()
             with foo: pass
-        self.assertRaises(AttributeError, fooLacksExit)
+        self.assertRaisesRegexp(AttributeError, '__exit__', fooLacksExit)
 
     def assertRaisesSyntaxError(self, codestr):
         def shouldRaiseSyntaxError(s):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -15,6 +15,10 @@
 
 - Issue #28532: Show sys.version when -V option is supplied twice.
 
+- Issue #27100: The with-statement now checks for __enter__ before it
+  checks for __exit__.  This gives less confusing error messages when
+  both methods are missing. Patch by Jonathan Ellington.
+
 - Issue #28746: Fix the set_inheritable() file descriptor method on platforms
   that do not have the ioctl FIOCLEX and FIONCLEX commands.
 
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -3141,15 +3141,15 @@
             _Py_IDENTIFIER(__exit__);
             _Py_IDENTIFIER(__enter__);
             PyObject *mgr = TOP();
-            PyObject *exit = special_lookup(mgr, &PyId___exit__), *enter;
+            PyObject *enter = special_lookup(mgr, &PyId___enter__), *exit;
             PyObject *res;
+            if (enter == NULL)
+                goto error;
+            exit = special_lookup(mgr, &PyId___exit__);
             if (exit == NULL)
                 goto error;
             SET_TOP(exit);
-            enter = special_lookup(mgr, &PyId___enter__);
             Py_DECREF(mgr);
-            if (enter == NULL)
-                goto error;
             res = PyObject_CallFunctionObjArgs(enter, NULL);
             Py_DECREF(enter);
             if (res == NULL)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 00:58:42 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 22 Nov 2016 05:58:42 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328761=3A_The_fiel?=
 =?utf-8?q?ds_name_and_doc_of_structures_PyMemberDef=2C_PyGetSetDef=2C?=
Message-ID: <20161122055842.26749.74126.BDE31D24@psf.io>

https://hg.python.org/cpython/rev/42b0ba372ec2
changeset:   105325:42b0ba372ec2
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 22 07:58:08 2016 +0200
summary:
  Issue #28761: The fields name and doc of structures PyMemberDef, PyGetSetDef,
PyStructSequence_Field, PyStructSequence_Desc, and wrapperbase are now of
type "const char *" rather of "char *".

files:
  Doc/c-api/typeobj.rst      |  10 +++++-----
  Doc/extending/newtypes.rst |  10 +++++-----
  Doc/whatsnew/3.7.rst       |   6 ++++++
  Include/descrobject.h      |   8 ++++----
  Include/structmember.h     |   4 ++--
  Include/structseq.h        |   8 ++++----
  Misc/NEWS                  |   4 ++++
  Objects/structseq.c        |   2 +-
  8 files changed, 31 insertions(+), 21 deletions(-)


diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst
--- a/Doc/c-api/typeobj.rst
+++ b/Doc/c-api/typeobj.rst
@@ -725,11 +725,11 @@
       typedef int (*setter)(PyObject *, PyObject *, void *);
 
       typedef struct PyGetSetDef {
-          char *name;    /* attribute name */
-          getter get;    /* C function to get the attribute */
-          setter set;    /* C function to set or delete the attribute */
-          char *doc;     /* optional doc string */
-          void *closure; /* optional additional data for getter and setter */
+          const char *name; /* attribute name */
+          getter get;       /* C function to get the attribute */
+          setter set;       /* C function to set or delete the attribute */
+          const char *doc;  /* optional doc string */
+          void *closure;    /* optional additional data for getter and setter */
       } PyGetSetDef;
 
 
diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst
--- a/Doc/extending/newtypes.rst
+++ b/Doc/extending/newtypes.rst
@@ -1138,11 +1138,11 @@
 be read-only or read-write.  The structures in the table are defined as::
 
    typedef struct PyMemberDef {
-       char *name;
-       int   type;
-       int   offset;
-       int   flags;
-       char *doc;
+       const char *name;
+       int         type;
+       int         offset;
+       int         flags;
+       const char *doc;
    } PyMemberDef;
 
 For each entry in the table, a :term:`descriptor` will be constructed and added to the
diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst
--- a/Doc/whatsnew/3.7.rst
+++ b/Doc/whatsnew/3.7.rst
@@ -99,6 +99,12 @@
   of libffi is now required when building ``_ctypes`` on such platforms.
   Contributed by Zachary Ware in :issue:`27979`.
 
+* The fields :c:member:`name` and :c:member:`doc` of structures
+  :c:type:`PyMemberDef`, :c:type:`PyGetSetDef`,
+  :c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`,
+  and :c:type:`wrapperbase` are now of type ``const char *`` rather of
+  ``char *``.  (Contributed by Serhiy Storchaka in :issue:`28761`.)
+
 
 Deprecated
 ==========
diff --git a/Include/descrobject.h b/Include/descrobject.h
--- a/Include/descrobject.h
+++ b/Include/descrobject.h
@@ -9,10 +9,10 @@
 typedef int (*setter)(PyObject *, PyObject *, void *);
 
 typedef struct PyGetSetDef {
-    char *name;
+    const char *name;
     getter get;
     setter set;
-    char *doc;
+    const char *doc;
     void *closure;
 } PyGetSetDef;
 
@@ -24,11 +24,11 @@
                                       void *wrapped, PyObject *kwds);
 
 struct wrapperbase {
-    char *name;
+    const char *name;
     int offset;
     void *function;
     wrapperfunc wrapper;
-    char *doc;
+    const char *doc;
     int flags;
     PyObject *name_strobj;
 };
diff --git a/Include/structmember.h b/Include/structmember.h
--- a/Include/structmember.h
+++ b/Include/structmember.h
@@ -16,11 +16,11 @@
    pointer is NULL. */
 
 typedef struct PyMemberDef {
-    char *name;
+    const char *name;
     int type;
     Py_ssize_t offset;
     int flags;
-    char *doc;
+    const char *doc;
 } PyMemberDef;
 
 /* Types */
diff --git a/Include/structseq.h b/Include/structseq.h
--- a/Include/structseq.h
+++ b/Include/structseq.h
@@ -8,13 +8,13 @@
 #endif
 
 typedef struct PyStructSequence_Field {
-    char *name;
-    char *doc;
+    const char *name;
+    const char *doc;
 } PyStructSequence_Field;
 
 typedef struct PyStructSequence_Desc {
-    char *name;
-    char *doc;
+    const char *name;
+    const char *doc;
     struct PyStructSequence_Field *fields;
     int n_in_sequence;
 } PyStructSequence_Desc;
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -432,6 +432,10 @@
 C API
 -----
 
+- Issue #28761: The fields name and doc of structures PyMemberDef, PyGetSetDef,
+  PyStructSequence_Field, PyStructSequence_Desc, and wrapperbase are now of
+  type "const char *" rather of "char *".
+
 - Issue #28748: Private variable _Py_PackageContext is now of type "const char *"
   rather of "char *".
 
diff --git a/Objects/structseq.c b/Objects/structseq.c
--- a/Objects/structseq.c
+++ b/Objects/structseq.c
@@ -256,7 +256,7 @@
     }
 
     for (; i < n_fields; i++) {
-        char *n = Py_TYPE(self)->tp_members[i-n_unnamed_fields].name;
+        const char *n = Py_TYPE(self)->tp_members[i-n_unnamed_fields].name;
         if (PyDict_SetItemString(dict, n, self->ob_item[i]) < 0)
             goto error;
     }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 01:26:37 2016
From: python-checkins at python.org (ned.deily)
Date: Tue, 22 Nov 2016 06:26:37 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Added_tag_v3?=
 =?utf-8?q?=2E6=2E0b4_for_changeset_18496abdb3d5?=
Message-ID: <20161122062637.3230.33646.52BCD011@psf.io>

https://hg.python.org/cpython/rev/38714e702021
changeset:   105328:38714e702021
branch:      3.6
user:        Ned Deily <nad at python.org>
date:        Mon Nov 21 23:31:39 2016 -0500
summary:
  Added tag v3.6.0b4 for changeset 18496abdb3d5

files:
  .hgtags |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -173,3 +173,4 @@
 5b0ca4ed5e2f0669d76ece7ef975c544580f12b4 v3.6.0b1
 b9fadc7d1c3f9c3c77f32f35afbe1a1cc38070e6 v3.6.0b2
 8345e066c0ed713c3e510cbc8fafc1c38d6d306b v3.6.0b3
+18496abdb3d5c2730a659b747a89261b2219fecf v3.6.0b4

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 01:26:37 2016
From: python-checkins at python.org (ned.deily)
Date: Tue, 22 Nov 2016 06:26:37 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Version_bump_f?=
 =?utf-8?q?or_3=2E6=2E0b4?=
Message-ID: <20161122062637.33458.16471.34D46AED@psf.io>

https://hg.python.org/cpython/rev/18496abdb3d5
changeset:   105327:18496abdb3d5
branch:      3.6
tag:         v3.6.0b4
user:        Ned Deily <nad at python.org>
date:        Mon Nov 21 23:30:55 2016 -0500
summary:
  Version bump for 3.6.0b4

files:
  Include/patchlevel.h |  2 +-
  Misc/NEWS            |  2 +-
  README               |  2 +-
  3 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Include/patchlevel.h b/Include/patchlevel.h
--- a/Include/patchlevel.h
+++ b/Include/patchlevel.h
@@ -23,7 +23,7 @@
 #define PY_RELEASE_SERIAL	4
 
 /* Version as a string */
-#define PY_VERSION      	"3.6.0b4+"
+#define PY_VERSION      	"3.6.0b4"
 /*--end constants--*/
 
 /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2.
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -5,7 +5,7 @@
 What's New in Python 3.6.0 beta 4
 =================================
 
-*Release date: XXXX-XX-XX*
+*Release date: 2016-11-21*
 
 Core and Builtins
 -----------------
diff --git a/README b/README
--- a/README
+++ b/README
@@ -1,4 +1,4 @@
-This is Python version 3.6.0 beta 3
+This is Python version 3.6.0 beta 4
 ===================================
 
 Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011,

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 01:26:37 2016
From: python-checkins at python.org (ned.deily)
Date: Tue, 22 Nov 2016 06:26:37 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Update_pydoc_t?=
 =?utf-8?q?opics_for_3=2E6=2E0b4?=
Message-ID: <20161122062637.33414.98935.76E9B6B8@psf.io>

https://hg.python.org/cpython/rev/9d710f4b77ac
changeset:   105326:9d710f4b77ac
branch:      3.6
parent:      105323:3aafb232f2db
user:        Ned Deily <nad at python.org>
date:        Mon Nov 21 23:24:38 2016 -0500
summary:
  Update pydoc topics for 3.6.0b4

files:
  Lib/pydoc_data/topics.py |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py
--- a/Lib/pydoc_data/topics.py
+++ b/Lib/pydoc_data/topics.py
@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-# Autogenerated by Sphinx on Mon Oct 31 20:37:53 2016
+# Autogenerated by Sphinx on Mon Nov 21 23:22:05 2016
 topics = {'assert': '\n'
            'The "assert" statement\n'
            '**********************\n'
@@ -3702,7 +3702,7 @@
              '   end). This is because any time you resume execution (even '
              'with a\n'
              '   simple next or step), you may encounter another '
-             'breakpoint--which\n'
+             'breakpoint?which\n'
              '   could have its own command list, leading to ambiguities about '
              'which\n'
              '   list to execute.\n'

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 01:26:40 2016
From: python-checkins at python.org (ned.deily)
Date: Tue, 22 Nov 2016 06:26:40 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge_tag_from_3=2E6?=
Message-ID: <20161122062637.24316.46842.4A858B72@psf.io>

https://hg.python.org/cpython/rev/2a14385710dc
changeset:   105330:2a14385710dc
parent:      105325:42b0ba372ec2
parent:      105329:88aba26bfc2b
user:        Ned Deily <nad at python.org>
date:        Tue Nov 22 01:25:55 2016 -0500
summary:
  merge tag from 3.6

files:
  .hgtags |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -173,3 +173,4 @@
 5b0ca4ed5e2f0669d76ece7ef975c544580f12b4 v3.6.0b1
 b9fadc7d1c3f9c3c77f32f35afbe1a1cc38070e6 v3.6.0b2
 8345e066c0ed713c3e510cbc8fafc1c38d6d306b v3.6.0b3
+18496abdb3d5c2730a659b747a89261b2219fecf v3.6.0b4

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 01:26:40 2016
From: python-checkins at python.org (ned.deily)
Date: Tue, 22 Nov 2016 06:26:40 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogU3RhcnQgMy42LjBy?=
 =?utf-8?q?c1=2E_=28Note=2C_patch_level_was_incorrectly_set_to_3=2E6=2E0b4?=
 =?utf-8?q?+_following?=
Message-ID: <20161122062637.26602.41531.16673794@psf.io>

https://hg.python.org/cpython/rev/88aba26bfc2b
changeset:   105329:88aba26bfc2b
branch:      3.6
user:        Ned Deily <nad at python.org>
date:        Tue Nov 22 01:21:04 2016 -0500
summary:
  Start 3.6.0rc1. (Note, patch level was incorrectly set to 3.6.0b4+ following b3. It is now b4+ again.)

files:
  Include/patchlevel.h |   2 +-
  Misc/NEWS            |  12 ++++++++++++
  2 files changed, 13 insertions(+), 1 deletions(-)


diff --git a/Include/patchlevel.h b/Include/patchlevel.h
--- a/Include/patchlevel.h
+++ b/Include/patchlevel.h
@@ -23,7 +23,7 @@
 #define PY_RELEASE_SERIAL	4
 
 /* Version as a string */
-#define PY_VERSION      	"3.6.0b4"
+#define PY_VERSION      	"3.6.0b4+"
 /*--end constants--*/
 
 /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2.
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -2,6 +2,18 @@
 Python News
 +++++++++++
 
+What's New in Python 3.6.0 release candidate 1
+==============================================
+
+*Release date: XXXX-XX-XX*
+
+Core and Builtins
+-----------------
+
+Library
+-------
+
+
 What's New in Python 3.6.0 beta 4
 =================================
 

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Tue Nov 22 04:06:07 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Tue, 22 Nov 2016 09:06:07 +0000
Subject: [Python-checkins] Daily reference leaks (a0da176233b9): sum=981
Message-ID: <20161122090607.113512.35695.0444599F@psf.io>

results for a0da176233b9 on branch "default"
--------------------------------------------

test_collections leaked [0, 0, 7] memory blocks, sum=7
test_contextlib leaked [38, 38, 38] references, sum=114
test_contextlib leaked [13, 12, 12] memory blocks, sum=37
test_descr leaked [164, 164, 164] references, sum=492
test_descr leaked [59, 59, 59] memory blocks, sum=177
test_functools leaked [0, 3, 1] memory blocks, sum=4
test_with leaked [37, 37, 37] references, sum=111
test_with leaked [13, 13, 13] memory blocks, sum=39


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogWMjqsJ', '--timeout', '7200']

From python-checkins at python.org  Tue Nov 22 05:43:58 2016
From: python-checkins at python.org (inada.naoki)
Date: Tue, 22 Nov 2016 10:43:58 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4MDIz?=
 =?utf-8?q?=3A_Fix_python-gdb=2Epy_didn=27t_support_new_dict_implementatio?=
 =?utf-8?q?n?=
Message-ID: <20161122104358.18916.61306.EC35C68B@psf.io>

https://hg.python.org/cpython/rev/4f6fb9e47f6b
changeset:   105331:4f6fb9e47f6b
branch:      3.6
parent:      105329:88aba26bfc2b
user:        INADA Naoki <songofacandy at gmail.com>
date:        Tue Nov 22 19:40:58 2016 +0900
summary:
  Issue #28023: Fix python-gdb.py didn't support new dict implementation

files:
  Lib/test/test_gdb.py   |   9 +++------
  Misc/NEWS              |   5 +++++
  Tools/gdb/libpython.py |  28 ++++++++++++++++++++++++++--
  3 files changed, 34 insertions(+), 8 deletions(-)


diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py
--- a/Lib/test/test_gdb.py
+++ b/Lib/test/test_gdb.py
@@ -11,9 +11,6 @@
 import unittest
 import locale
 
-# FIXME: issue #28023
-raise unittest.SkipTest("FIXME: issue #28023, compact dict (issue #27350) broke python-gdb.py")
-
 # Is this Python configured to support threads?
 try:
     import _thread
@@ -296,9 +293,8 @@
         'Verify the pretty-printing of dictionaries'
         self.assertGdbRepr({})
         self.assertGdbRepr({'foo': 'bar'}, "{'foo': 'bar'}")
-        # PYTHONHASHSEED is need to get the exact item order
-        if not sys.flags.ignore_environment:
-            self.assertGdbRepr({'foo': 'bar', 'douglas': 42}, "{'douglas': 42, 'foo': 'bar'}")
+        # Python preserves insertion order since 3.6
+        self.assertGdbRepr({'foo': 'bar', 'douglas': 42}, "{'foo': 'bar', 'douglas': 42}")
 
     def test_lists(self):
         'Verify the pretty-printing of lists'
@@ -819,6 +815,7 @@
                                           )
         self.assertIn('Garbage-collecting', gdb_output)
 
+    @unittest.skip("FIXME: builtin method is not shown in py-bt and py-bt-full")
     @unittest.skipIf(python_is_optimized(),
                      "Python was compiled with optimizations")
     # Some older versions of gdb will fail with
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -13,6 +13,11 @@
 Library
 -------
 
+Tools/Demos
+-----------
+
+- Issue #28023: Fix python-gdb.py didn't support new dict implementation.
+
 
 What's New in Python 3.6.0 beta 4
 =================================
diff --git a/Tools/gdb/libpython.py b/Tools/gdb/libpython.py
--- a/Tools/gdb/libpython.py
+++ b/Tools/gdb/libpython.py
@@ -666,8 +666,9 @@
         '''
         keys = self.field('ma_keys')
         values = self.field('ma_values')
-        for i in safe_range(keys['dk_size']):
-            ep = keys['dk_entries'].address + i
+        entries, nentries = self._get_entries(keys)
+        for i in safe_range(nentries):
+            ep = entries[i]
             if long(values):
                 pyop_value = PyObjectPtr.from_pyobject_ptr(values[i])
             else:
@@ -707,6 +708,29 @@
             pyop_value.write_repr(out, visited)
         out.write('}')
 
+    def _get_entries(self, keys):
+        dk_size = int(keys['dk_size'])
+        try:
+            # <= Python 3.5
+            return keys['dk_entries'], dk_size
+        except gdb.error:
+            # >= Python 3.6
+            pass
+
+        if dk_size <= 0xFF:
+            offset = dk_size
+        elif dk_size <= 0xFFFF:
+            offset = 2 * dk_size
+        elif dk_size <= 0xFFFFFFFF:
+            offset = 4 * dk_size
+        else:
+            offset = 8 * dk_size
+
+        ent_ptr_t = gdb.lookup_type('PyDictKeyEntry').pointer()
+        ent_addr = int(keys['dk_indices']['as_1'].address) + offset
+        return gdb.Value(ent_addr).cast(ent_ptr_t), int(keys['dk_nentries'])
+
+
 class PyListObjectPtr(PyObjectPtr):
     _typename = 'PyListObject'
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 05:43:59 2016
From: python-checkins at python.org (inada.naoki)
Date: Tue, 22 Nov 2016 10:43:59 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328023=3A_Fix_python-gdb=2Epy_didn=27t_support_n?=
 =?utf-8?q?ew_dict_implementation?=
Message-ID: <20161122104358.89278.98376.FF01DF5A@psf.io>

https://hg.python.org/cpython/rev/c51045920410
changeset:   105332:c51045920410
parent:      105330:2a14385710dc
parent:      105331:4f6fb9e47f6b
user:        INADA Naoki <songofacandy at gmail.com>
date:        Tue Nov 22 19:43:11 2016 +0900
summary:
  Issue #28023: Fix python-gdb.py didn't support new dict implementation

files:
  Lib/test/test_gdb.py   |   9 +++------
  Misc/NEWS              |   2 ++
  Tools/gdb/libpython.py |  28 ++++++++++++++++++++++++++--
  3 files changed, 31 insertions(+), 8 deletions(-)


diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py
--- a/Lib/test/test_gdb.py
+++ b/Lib/test/test_gdb.py
@@ -11,9 +11,6 @@
 import unittest
 import locale
 
-# FIXME: issue #28023
-raise unittest.SkipTest("FIXME: issue #28023, compact dict (issue #27350) broke python-gdb.py")
-
 # Is this Python configured to support threads?
 try:
     import _thread
@@ -296,9 +293,8 @@
         'Verify the pretty-printing of dictionaries'
         self.assertGdbRepr({})
         self.assertGdbRepr({'foo': 'bar'}, "{'foo': 'bar'}")
-        # PYTHONHASHSEED is need to get the exact item order
-        if not sys.flags.ignore_environment:
-            self.assertGdbRepr({'foo': 'bar', 'douglas': 42}, "{'douglas': 42, 'foo': 'bar'}")
+        # Python preserves insertion order since 3.6
+        self.assertGdbRepr({'foo': 'bar', 'douglas': 42}, "{'foo': 'bar', 'douglas': 42}")
 
     def test_lists(self):
         'Verify the pretty-printing of lists'
@@ -819,6 +815,7 @@
                                           )
         self.assertIn('Garbage-collecting', gdb_output)
 
+    @unittest.skip("FIXME: builtin method is not shown in py-bt and py-bt-full")
     @unittest.skipIf(python_is_optimized(),
                      "Python was compiled with optimizations")
     # Some older versions of gdb will fail with
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -493,6 +493,8 @@
 Tools/Demos
 -----------
 
+- Issue #28023: Fix python-gdb.py didn't support new dict implementation.
+
 - Issue #15369: The pybench and pystone microbenchmark have been removed from
   Tools. Please use the new Python benchmark suite
   https://github.com/python/performance which is more reliable and includes a
diff --git a/Tools/gdb/libpython.py b/Tools/gdb/libpython.py
--- a/Tools/gdb/libpython.py
+++ b/Tools/gdb/libpython.py
@@ -666,8 +666,9 @@
         '''
         keys = self.field('ma_keys')
         values = self.field('ma_values')
-        for i in safe_range(keys['dk_size']):
-            ep = keys['dk_entries'].address + i
+        entries, nentries = self._get_entries(keys)
+        for i in safe_range(nentries):
+            ep = entries[i]
             if long(values):
                 pyop_value = PyObjectPtr.from_pyobject_ptr(values[i])
             else:
@@ -707,6 +708,29 @@
             pyop_value.write_repr(out, visited)
         out.write('}')
 
+    def _get_entries(self, keys):
+        dk_size = int(keys['dk_size'])
+        try:
+            # <= Python 3.5
+            return keys['dk_entries'], dk_size
+        except gdb.error:
+            # >= Python 3.6
+            pass
+
+        if dk_size <= 0xFF:
+            offset = dk_size
+        elif dk_size <= 0xFFFF:
+            offset = 2 * dk_size
+        elif dk_size <= 0xFFFFFFFF:
+            offset = 4 * dk_size
+        else:
+            offset = 8 * dk_size
+
+        ent_ptr_t = gdb.lookup_type('PyDictKeyEntry').pointer()
+        ent_addr = int(keys['dk_indices']['as_1'].address) + offset
+        return gdb.Value(ent_addr).cast(ent_ptr_t), int(keys['dk_nentries'])
+
+
 class PyListObjectPtr(PyObjectPtr):
     _typename = 'PyListObject'
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 07:11:40 2016
From: python-checkins at python.org (victor.stinner)
Date: Tue, 22 Nov 2016 12:11:40 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42?=
Message-ID: <20161122121138.24294.7292.333839CD@psf.io>

https://hg.python.org/cpython/rev/19adf22d6a5b
changeset:   105334:19adf22d6a5b
parent:      105332:c51045920410
parent:      105333:9b974f988c95
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 22 13:10:02 2016 +0100
summary:
  Merge 3.6

files:
  Tools/gdb/libpython.py |  8 ++++++--
  1 files changed, 6 insertions(+), 2 deletions(-)


diff --git a/Tools/gdb/libpython.py b/Tools/gdb/libpython.py
--- a/Tools/gdb/libpython.py
+++ b/Tools/gdb/libpython.py
@@ -709,6 +709,7 @@
         out.write('}')
 
     def _get_entries(self, keys):
+        dk_nentries = int(keys['dk_nentries'])
         dk_size = int(keys['dk_size'])
         try:
             # <= Python 3.5
@@ -726,9 +727,12 @@
         else:
             offset = 8 * dk_size
 
+        ent_addr = keys['dk_indices']['as_1'].address
+        ent_addr = ent_addr.cast(_type_unsigned_char_ptr()) + offset
         ent_ptr_t = gdb.lookup_type('PyDictKeyEntry').pointer()
-        ent_addr = int(keys['dk_indices']['as_1'].address) + offset
-        return gdb.Value(ent_addr).cast(ent_ptr_t), int(keys['dk_nentries'])
+        ent_addr = ent_addr.cast(ent_ptr_t)
+
+        return ent_addr, dk_nentries
 
 
 class PyListObjectPtr(PyObjectPtr):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 07:11:40 2016
From: python-checkins at python.org (victor.stinner)
Date: Tue, 22 Nov 2016 12:11:40 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4MDIz?=
 =?utf-8?q?=3A_Fix_python-gdb=2Epy_on_old_GDB_versions?=
Message-ID: <20161122121137.9108.71963.930725B9@psf.io>

https://hg.python.org/cpython/rev/9b974f988c95
changeset:   105333:9b974f988c95
branch:      3.6
parent:      105331:4f6fb9e47f6b
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 22 13:09:39 2016 +0100
summary:
  Issue #28023: Fix python-gdb.py on old GDB versions

Replace int(value.address)+offset with value.cast(unsigned char*)+offset.

It seems like int(value.address) fails on old versions of GDB.

files:
  Tools/gdb/libpython.py |  8 ++++++--
  1 files changed, 6 insertions(+), 2 deletions(-)


diff --git a/Tools/gdb/libpython.py b/Tools/gdb/libpython.py
--- a/Tools/gdb/libpython.py
+++ b/Tools/gdb/libpython.py
@@ -709,6 +709,7 @@
         out.write('}')
 
     def _get_entries(self, keys):
+        dk_nentries = int(keys['dk_nentries'])
         dk_size = int(keys['dk_size'])
         try:
             # <= Python 3.5
@@ -726,9 +727,12 @@
         else:
             offset = 8 * dk_size
 
+        ent_addr = keys['dk_indices']['as_1'].address
+        ent_addr = ent_addr.cast(_type_unsigned_char_ptr()) + offset
         ent_ptr_t = gdb.lookup_type('PyDictKeyEntry').pointer()
-        ent_addr = int(keys['dk_indices']['as_1'].address) + offset
-        return gdb.Value(ent_addr).cast(ent_ptr_t), int(keys['dk_nentries'])
+        ent_addr = ent_addr.cast(ent_ptr_t)
+
+        return ent_addr, dk_nentries
 
 
 class PyListObjectPtr(PyObjectPtr):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 08:00:57 2016
From: python-checkins at python.org (xiang.zhang)
Date: Tue, 22 Nov 2016 13:00:57 +0000
Subject: [Python-checkins] =?utf-8?q?test=3A_Test=2E_Add_c?=
Message-ID: <20161122130056.113582.44120.B9245931@psf.io>

https://hg.python.org/test/rev/530f0afa8072
changeset:   235:530f0afa8072
user:        Xiang Zhang <angwerzx at 126.com>
date:        Tue Nov 22 21:00:25 2016 +0800
summary:
  Test. Add c

files:
  c |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/c b/c
new file mode 100644
--- /dev/null
+++ b/c
@@ -0,0 +1,1 @@
+xxx

-- 
Repository URL: https://hg.python.org/test

From lp_benchmark_robot at intel.com  Tue Nov 22 08:53:31 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Tue, 22 Nov 2016 13:53:31 +0000
Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-11-22
Message-ID: <bb9e97ee-1592-4016-bf59-cfff2d4e1de1@irsmsx106.ger.corp.intel.com>

Results for project Python 2.7, build date 2016-11-22 03:47:50 +0000
commit:		6eefdf0fade6
previous commit:	4de956751cf1
revision date:	2016-11-21 09:37:18 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.17%         -1.09%          3.37%             8.32%
:-)             pybench      0.13%         -0.05%          5.71%             3.52%
:-(            regex_v8      0.57%          0.32%         -2.32%            11.28%
:-)               nbody      0.86%         -0.57%          7.89%             5.91%
:-)        json_dump_v2      0.30%          0.72%          3.79%             7.84%
:-|      normal_startup      1.00%         -0.84%         -0.31%             2.71%
:-)             ssbench      0.23%          0.11%          2.63%             1.30%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-11-22/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Tue Nov 22 08:53:59 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Tue, 22 Nov 2016 13:53:59 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python Default
 2016-11-22
Message-ID: <498d5879-d697-4440-8c25-9488b8a3ecad@irsmsx106.ger.corp.intel.com>

Results for project Python default, build date 2016-11-22 03:01:50 +0000
commit:		a0da176233b9
previous commit:	30f154d9abf0
revision date:	2016-11-22 01:24:58 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.19%          0.06%          7.41%            14.04%
:-)             pybench      0.12%         -0.10%          5.96%             4.81%
:-(            regex_v8      3.75%         -0.83%         -2.59%             3.23%
:-)               nbody      0.13%          1.97%          2.71%             6.17%
:-(        json_dump_v2      0.28%          3.62%         -4.60%            10.35%
:-|      normal_startup      0.76%          0.23%          0.74%             6.14%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-11-22/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Tue Nov 22 09:31:47 2016
From: python-checkins at python.org (victor.stinner)
Date: Tue, 22 Nov 2016 14:31:47 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NzI3?=
 =?utf-8?q?=3A_Fix_typo_in_pattern=5Frichcompare=28=29?=
Message-ID: <20161122143146.88420.33018.36888BED@psf.io>

https://hg.python.org/cpython/rev/6b43d15fd2d7
changeset:   105335:6b43d15fd2d7
branch:      3.6
parent:      105333:9b974f988c95
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 22 15:23:00 2016 +0100
summary:
  Issue #28727: Fix typo in pattern_richcompare()

Typo catched by Serhiy Storchaka, thanks!

files:
  Modules/_sre.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/_sre.c b/Modules/_sre.c
--- a/Modules/_sre.c
+++ b/Modules/_sre.c
@@ -2688,7 +2688,7 @@
 
     cmp = (left->flags == right->flags
            && left->isbytes == right->isbytes
-           && left->codesize && right->codesize);
+           && left->codesize == right->codesize);
     if (cmp) {
         /* Compare the code and the pattern because the same pattern can
            produce different codes depending on the locale used to compile the

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 09:31:47 2016
From: python-checkins at python.org (victor.stinner)
Date: Tue, 22 Nov 2016 14:31:47 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NzI3?=
 =?utf-8?q?=3A_Optimize_pattern=5Frichcompare=28=29_for_a=3D=3Da?=
Message-ID: <20161122143146.89721.91656.86BA7D1C@psf.io>

https://hg.python.org/cpython/rev/c2cb70c97163
changeset:   105336:c2cb70c97163
branch:      3.6
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 22 15:30:38 2016 +0100
summary:
  Issue #28727: Optimize pattern_richcompare() for a==a

A pattern is equal to itself.

files:
  Lib/test/test_re.py |  4 ++++
  Modules/_sre.c      |  6 ++++++
  2 files changed, 10 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py
--- a/Lib/test/test_re.py
+++ b/Lib/test/test_re.py
@@ -1781,6 +1781,10 @@
     def test_pattern_compare(self):
         pattern1 = re.compile('abc', re.IGNORECASE)
 
+        # equal to itself
+        self.assertEqual(pattern1, pattern1)
+        self.assertFalse(pattern1 != pattern1)
+
         # equal
         re.purge()
         pattern2 = re.compile('abc', re.IGNORECASE)
diff --git a/Modules/_sre.c b/Modules/_sre.c
--- a/Modules/_sre.c
+++ b/Modules/_sre.c
@@ -2683,6 +2683,12 @@
     if (Py_TYPE(lefto) != &Pattern_Type || Py_TYPE(righto) != &Pattern_Type) {
         Py_RETURN_NOTIMPLEMENTED;
     }
+
+    if (lefto == righto) {
+        /* a pattern is equal to itself */
+        return PyBool_FromLong(op == Py_EQ);
+    }
+
     left = (PatternObject *)lefto;
     right = (PatternObject *)righto;
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 09:32:50 2016
From: python-checkins at python.org (victor.stinner)
Date: Tue, 22 Nov 2016 14:32:50 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42?=
Message-ID: <20161122143146.9068.76488.8743B932@psf.io>

https://hg.python.org/cpython/rev/09e2c4197dbf
changeset:   105337:09e2c4197dbf
parent:      105334:19adf22d6a5b
parent:      105336:c2cb70c97163
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 22 15:30:53 2016 +0100
summary:
  Merge 3.6

files:
  Lib/test/test_re.py |  4 ++++
  Modules/_sre.c      |  8 +++++++-
  2 files changed, 11 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py
--- a/Lib/test/test_re.py
+++ b/Lib/test/test_re.py
@@ -1781,6 +1781,10 @@
     def test_pattern_compare(self):
         pattern1 = re.compile('abc', re.IGNORECASE)
 
+        # equal to itself
+        self.assertEqual(pattern1, pattern1)
+        self.assertFalse(pattern1 != pattern1)
+
         # equal
         re.purge()
         pattern2 = re.compile('abc', re.IGNORECASE)
diff --git a/Modules/_sre.c b/Modules/_sre.c
--- a/Modules/_sre.c
+++ b/Modules/_sre.c
@@ -2683,12 +2683,18 @@
     if (Py_TYPE(lefto) != &Pattern_Type || Py_TYPE(righto) != &Pattern_Type) {
         Py_RETURN_NOTIMPLEMENTED;
     }
+
+    if (lefto == righto) {
+        /* a pattern is equal to itself */
+        return PyBool_FromLong(op == Py_EQ);
+    }
+
     left = (PatternObject *)lefto;
     right = (PatternObject *)righto;
 
     cmp = (left->flags == right->flags
            && left->isbytes == right->isbytes
-           && left->codesize && right->codesize);
+           && left->codesize == right->codesize);
     if (cmp) {
         /* Compare the code and the pattern because the same pattern can
            produce different codes depending on the locale used to compile the

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 14:49:38 2016
From: python-checkins at python.org (steve.dower)
Date: Tue, 22 Nov 2016 19:49:38 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328573=3A_Missing_sys=2E=5Fmercurial_info_and_ot?=
 =?utf-8?q?her_build_issues=2E?=
Message-ID: <20161122194938.88420.60610.ACCFAA97@psf.io>

https://hg.python.org/cpython/rev/d0958078bcb6
changeset:   105339:d0958078bcb6
parent:      105337:09e2c4197dbf
parent:      105338:089886be06df
user:        Steve Dower <steve.dower at microsoft.com>
date:        Tue Nov 22 11:49:09 2016 -0800
summary:
  Issue #28573: Missing sys._mercurial info and other build issues.

files:
  PCbuild/pythoncore.vcxproj |  13 ++++++++++---
  Tools/msi/buildrelease.bat |  19 ++++++++++++++++---
  2 files changed, 26 insertions(+), 6 deletions(-)


diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj
--- a/PCbuild/pythoncore.vcxproj
+++ b/PCbuild/pythoncore.vcxproj
@@ -406,14 +406,21 @@
   <ImportGroup Label="ExtensionTargets">
   </ImportGroup>
   <Target Name="_GetBuildInfo" BeforeTargets="PrepareForBuild">
-    <Exec Command="hg id -b &gt; &quot;$(IntDir)hgbranch.txt&quot;" ContinueOnError="true" />
-    <Exec Command="hg id -i &gt; &quot;$(IntDir)hgversion.txt&quot;" ContinueOnError="true" />
-    <Exec Command="hg id -t &gt; &quot;$(IntDir)hgtag.txt&quot;" ContinueOnError="true" />
+    <PropertyGroup>
+      <HG Condition="$(HG) == ''">hg</HG>
+      <_HG>$(HG)</_HG>
+      <_HG Condition="$(HG.Contains(` `))">"$(HG)"</_HG>
+    </PropertyGroup>
+    <Message Text="Getting build info from $(_HG)" Importance="high" />
+    <Exec Command="$(_HG) id -b &gt; &quot;$(IntDir)hgbranch.txt&quot;" ContinueOnError="true" />
+    <Exec Command="$(_HG) id -i &gt; &quot;$(IntDir)hgversion.txt&quot;" ContinueOnError="true" />
+    <Exec Command="$(_HG) id -t &gt; &quot;$(IntDir)hgtag.txt&quot;" ContinueOnError="true" />
     <PropertyGroup>
       <HgBranch Condition="Exists('$(IntDir)hgbranch.txt')">$([System.IO.File]::ReadAllText('$(IntDir)hgbranch.txt').Trim())</HgBranch>
       <HgVersion Condition="Exists('$(IntDir)hgversion.txt')">$([System.IO.File]::ReadAllText('$(IntDir)hgversion.txt').Trim())</HgVersion>
       <HgTag Condition="Exists('$(IntDir)hgtag.txt')">$([System.IO.File]::ReadAllText('$(IntDir)hgtag.txt').Trim())</HgTag>
     </PropertyGroup>
+    <Message Text="Building $(HgTag):$(HgVersion) $(HgBranch)" Importance="high" />
     <ItemGroup>
       <ClCompile Include="..\Modules\getbuildinfo.c">
         <PreprocessorDefinitions>HGVERSION="$(HgVersion)";HGTAG="$(HgTag)";HGBRANCH="$(HgBranch)";%(PreprocessorDefinitions)</PreprocessorDefinitions>
diff --git a/Tools/msi/buildrelease.bat b/Tools/msi/buildrelease.bat
--- a/Tools/msi/buildrelease.bat
+++ b/Tools/msi/buildrelease.bat
@@ -78,7 +78,8 @@
 if errorlevel 1 goto :eof
 :skipdoc
 
-where hg /q || echo Cannot find Mercurial on PATH && exit /B 1
+where hg > "%TEMP%\hg.loc" 2> nul && set /P HG= < "%TEMP%\hg.loc" & del "%TEMP%\hg.loc"
+if not exist "%HG%" echo Cannot find Mercurial on PATH && exit /B 1
 
 where dlltool /q && goto skipdlltoolsearch
 set _DLLTOOL_PATH=
@@ -128,6 +129,12 @@
     if errorlevel 1 exit /B
 )
 
+if exist "%D%obj\Debug_%OBJDIR_PLAT%" (
+    echo Deleting "%D%obj\Debug_%OBJDIR_PLAT%"
+    rmdir /q/s "%D%obj\Debug_%OBJDIR_PLAT%"
+    if errorlevel 1 exit /B
+)
+
 if exist "%D%obj\Release_%OBJDIR_PLAT%" (
     echo Deleting "%D%obj\Release_%OBJDIR_PLAT%"
     rmdir /q/s "%D%obj\Release_%OBJDIR_PLAT%"
@@ -145,8 +152,14 @@
     set PGOOPTS=
 )
 if not "%SKIPBUILD%" EQU "1" (
-    @echo call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% %PGOOPTS%
-    @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% %PGOOPTS%
+    @echo call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %PGOOPTS% %CERTOPTS%
+    @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %PGOOPTS% %CERTOPTS%
+    @if errorlevel 1 exit /B
+    @rem build.bat turns echo back on, so we disable it again
+    @echo off
+
+    @echo call "%PCBUILD%build.bat" -d -e -p %BUILD_PLAT% -t %TARGET%
+    @call "%PCBUILD%build.bat" -d -e -p %BUILD_PLAT% -t %TARGET%
     @if errorlevel 1 exit /B
     @rem build.bat turns echo back on, so we disable it again
     @echo off

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 14:49:38 2016
From: python-checkins at python.org (steve.dower)
Date: Tue, 22 Nov 2016 19:49:38 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NTcz?=
 =?utf-8?q?=3A_Missing_sys=2E=5Fmercurial_info_and_other_build_issues=2E?=
Message-ID: <20161122194938.24648.66755.04FEDF84@psf.io>

https://hg.python.org/cpython/rev/089886be06df
changeset:   105338:089886be06df
branch:      3.6
parent:      105336:c2cb70c97163
user:        Steve Dower <steve.dower at microsoft.com>
date:        Tue Nov 22 11:48:52 2016 -0800
summary:
  Issue #28573: Missing sys._mercurial info and other build issues.

files:
  PCbuild/pythoncore.vcxproj |  13 ++++++++++---
  Tools/msi/buildrelease.bat |  19 ++++++++++++++++---
  2 files changed, 26 insertions(+), 6 deletions(-)


diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj
--- a/PCbuild/pythoncore.vcxproj
+++ b/PCbuild/pythoncore.vcxproj
@@ -406,14 +406,21 @@
   <ImportGroup Label="ExtensionTargets">
   </ImportGroup>
   <Target Name="_GetBuildInfo" BeforeTargets="PrepareForBuild">
-    <Exec Command="hg id -b &gt; &quot;$(IntDir)hgbranch.txt&quot;" ContinueOnError="true" />
-    <Exec Command="hg id -i &gt; &quot;$(IntDir)hgversion.txt&quot;" ContinueOnError="true" />
-    <Exec Command="hg id -t &gt; &quot;$(IntDir)hgtag.txt&quot;" ContinueOnError="true" />
+    <PropertyGroup>
+      <HG Condition="$(HG) == ''">hg</HG>
+      <_HG>$(HG)</_HG>
+      <_HG Condition="$(HG.Contains(` `))">"$(HG)"</_HG>
+    </PropertyGroup>
+    <Message Text="Getting build info from $(_HG)" Importance="high" />
+    <Exec Command="$(_HG) id -b &gt; &quot;$(IntDir)hgbranch.txt&quot;" ContinueOnError="true" />
+    <Exec Command="$(_HG) id -i &gt; &quot;$(IntDir)hgversion.txt&quot;" ContinueOnError="true" />
+    <Exec Command="$(_HG) id -t &gt; &quot;$(IntDir)hgtag.txt&quot;" ContinueOnError="true" />
     <PropertyGroup>
       <HgBranch Condition="Exists('$(IntDir)hgbranch.txt')">$([System.IO.File]::ReadAllText('$(IntDir)hgbranch.txt').Trim())</HgBranch>
       <HgVersion Condition="Exists('$(IntDir)hgversion.txt')">$([System.IO.File]::ReadAllText('$(IntDir)hgversion.txt').Trim())</HgVersion>
       <HgTag Condition="Exists('$(IntDir)hgtag.txt')">$([System.IO.File]::ReadAllText('$(IntDir)hgtag.txt').Trim())</HgTag>
     </PropertyGroup>
+    <Message Text="Building $(HgTag):$(HgVersion) $(HgBranch)" Importance="high" />
     <ItemGroup>
       <ClCompile Include="..\Modules\getbuildinfo.c">
         <PreprocessorDefinitions>HGVERSION="$(HgVersion)";HGTAG="$(HgTag)";HGBRANCH="$(HgBranch)";%(PreprocessorDefinitions)</PreprocessorDefinitions>
diff --git a/Tools/msi/buildrelease.bat b/Tools/msi/buildrelease.bat
--- a/Tools/msi/buildrelease.bat
+++ b/Tools/msi/buildrelease.bat
@@ -78,7 +78,8 @@
 if errorlevel 1 goto :eof
 :skipdoc
 
-where hg /q || echo Cannot find Mercurial on PATH && exit /B 1
+where hg > "%TEMP%\hg.loc" 2> nul && set /P HG= < "%TEMP%\hg.loc" & del "%TEMP%\hg.loc"
+if not exist "%HG%" echo Cannot find Mercurial on PATH && exit /B 1
 
 where dlltool /q && goto skipdlltoolsearch
 set _DLLTOOL_PATH=
@@ -128,6 +129,12 @@
     if errorlevel 1 exit /B
 )
 
+if exist "%D%obj\Debug_%OBJDIR_PLAT%" (
+    echo Deleting "%D%obj\Debug_%OBJDIR_PLAT%"
+    rmdir /q/s "%D%obj\Debug_%OBJDIR_PLAT%"
+    if errorlevel 1 exit /B
+)
+
 if exist "%D%obj\Release_%OBJDIR_PLAT%" (
     echo Deleting "%D%obj\Release_%OBJDIR_PLAT%"
     rmdir /q/s "%D%obj\Release_%OBJDIR_PLAT%"
@@ -145,8 +152,14 @@
     set PGOOPTS=
 )
 if not "%SKIPBUILD%" EQU "1" (
-    @echo call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% %PGOOPTS%
-    @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% %PGOOPTS%
+    @echo call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %PGOOPTS% %CERTOPTS%
+    @call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %PGOOPTS% %CERTOPTS%
+    @if errorlevel 1 exit /B
+    @rem build.bat turns echo back on, so we disable it again
+    @echo off
+
+    @echo call "%PCBUILD%build.bat" -d -e -p %BUILD_PLAT% -t %TARGET%
+    @call "%PCBUILD%build.bat" -d -e -p %BUILD_PLAT% -t %TARGET%
     @if errorlevel 1 exit /B
     @rem build.bat turns echo back on, so we disable it again
     @echo off

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 14:51:05 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 22 Nov 2016 19:51:05 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI3MTAw?=
 =?utf-8?q?=3A__Fix_ref_leak?=
Message-ID: <20161122195105.24571.84791.ACFF97C2@psf.io>

https://hg.python.org/cpython/rev/749c5d6c4ba5
changeset:   105340:749c5d6c4ba5
branch:      3.6
parent:      105338:089886be06df
user:        Raymond Hettinger <python at rcn.com>
date:        Tue Nov 22 11:50:40 2016 -0800
summary:
  Issue #27100:  Fix ref leak

files:
  Python/ceval.c |  4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -3138,8 +3138,10 @@
             if (enter == NULL)
                 goto error;
             exit = special_lookup(mgr, &PyId___exit__);
-            if (exit == NULL)
+            if (exit == NULL) {
+                Py_DECREF(enter);
                 goto error;
+            }
             SET_TOP(exit);
             Py_DECREF(mgr);
             res = PyObject_CallFunctionObjArgs(enter, NULL);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 14:51:05 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 22 Nov 2016 19:51:05 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161122195105.26881.96991.10118925@psf.io>

https://hg.python.org/cpython/rev/ed7be2ade1bc
changeset:   105341:ed7be2ade1bc
parent:      105339:d0958078bcb6
parent:      105340:749c5d6c4ba5
user:        Raymond Hettinger <python at rcn.com>
date:        Tue Nov 22 11:50:56 2016 -0800
summary:
  merge

files:
  Python/ceval.c |  4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -3146,8 +3146,10 @@
             if (enter == NULL)
                 goto error;
             exit = special_lookup(mgr, &PyId___exit__);
-            if (exit == NULL)
+            if (exit == NULL) {
+                Py_DECREF(enter);
                 goto error;
+            }
             SET_TOP(exit);
             Py_DECREF(mgr);
             res = PyObject_CallFunctionObjArgs(enter, NULL);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 16:55:34 2016
From: python-checkins at python.org (victor.stinner)
Date: Tue, 22 Nov 2016 21:55:34 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328770=3A_Update_p?=
 =?utf-8?q?ython-gdb=2Epy_for_fastcalls?=
Message-ID: <20161122215534.89121.23674.1C55DD45@psf.io>

https://hg.python.org/cpython/rev/752863f96fb8
changeset:   105342:752863f96fb8
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 22 22:53:18 2016 +0100
summary:
  Issue #28770: Update python-gdb.py for fastcalls

Frame.is_other_python_frame() now also handles _PyCFunction_FastCallDict()
frames.

Thanks to the new code to handle fast calls, python-gdb.py is now also able to
detect the <built-in id method of module ...> frame.

files:
  Lib/test/test_gdb.py   |  20 ++++++------
  Tools/gdb/libpython.py |  47 +++++++++++++++++++----------
  2 files changed, 41 insertions(+), 26 deletions(-)


diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py
--- a/Lib/test/test_gdb.py
+++ b/Lib/test/test_gdb.py
@@ -679,7 +679,7 @@
     def test_pyup_command(self):
         'Verify that the "py-up" command works'
         bt = self.get_stack_trace(script=self.get_sample_script(),
-                                  cmds_after_breakpoint=['py-up'])
+                                  cmds_after_breakpoint=['py-up', 'py-up'])
         self.assertMultilineMatches(bt,
                                     r'''^.*
 #[0-9]+ Frame 0x-?[0-9a-f]+, for file .*gdb_sample.py, line 7, in bar \(a=1, b=2, c=3\)
@@ -698,7 +698,7 @@
     def test_up_at_top(self):
         'Verify handling of "py-up" at the top of the stack'
         bt = self.get_stack_trace(script=self.get_sample_script(),
-                                  cmds_after_breakpoint=['py-up'] * 4)
+                                  cmds_after_breakpoint=['py-up'] * 5)
         self.assertEndsWith(bt,
                             'Unable to find an older python frame\n')
 
@@ -708,7 +708,7 @@
     def test_up_then_down(self):
         'Verify "py-up" followed by "py-down"'
         bt = self.get_stack_trace(script=self.get_sample_script(),
-                                  cmds_after_breakpoint=['py-up', 'py-down'])
+                                  cmds_after_breakpoint=['py-up', 'py-up', 'py-down'])
         self.assertMultilineMatches(bt,
                                     r'''^.*
 #[0-9]+ Frame 0x-?[0-9a-f]+, for file .*gdb_sample.py, line 7, in bar \(a=1, b=2, c=3\)
@@ -727,6 +727,7 @@
         self.assertMultilineMatches(bt,
                                     r'''^.*
 Traceback \(most recent call first\):
+  <built-in method id of module object .*>
   File ".*gdb_sample.py", line 10, in baz
     id\(42\)
   File ".*gdb_sample.py", line 7, in bar
@@ -815,7 +816,6 @@
                                           )
         self.assertIn('Garbage-collecting', gdb_output)
 
-    @unittest.skip("FIXME: builtin method is not shown in py-bt and py-bt-full")
     @unittest.skipIf(python_is_optimized(),
                      "Python was compiled with optimizations")
     # Some older versions of gdb will fail with
@@ -854,7 +854,7 @@
     def test_basic_command(self):
         'Verify that the "py-print" command works'
         bt = self.get_stack_trace(script=self.get_sample_script(),
-                                  cmds_after_breakpoint=['py-print args'])
+                                  cmds_after_breakpoint=['py-up', 'py-print args'])
         self.assertMultilineMatches(bt,
                                     r".*\nlocal 'args' = \(1, 2, 3\)\n.*")
 
@@ -863,7 +863,7 @@
     @unittest.skipUnless(HAS_PYUP_PYDOWN, "test requires py-up/py-down commands")
     def test_print_after_up(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
-                                  cmds_after_breakpoint=['py-up', 'py-print c', 'py-print b', 'py-print a'])
+                                  cmds_after_breakpoint=['py-up', 'py-up', 'py-print c', 'py-print b', 'py-print a'])
         self.assertMultilineMatches(bt,
                                     r".*\nlocal 'c' = 3\nlocal 'b' = 2\nlocal 'a' = 1\n.*")
 
@@ -871,7 +871,7 @@
                      "Python was compiled with optimizations")
     def test_printing_global(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
-                                  cmds_after_breakpoint=['py-print __name__'])
+                                  cmds_after_breakpoint=['py-up', 'py-print __name__'])
         self.assertMultilineMatches(bt,
                                     r".*\nglobal '__name__' = '__main__'\n.*")
 
@@ -879,7 +879,7 @@
                      "Python was compiled with optimizations")
     def test_printing_builtin(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
-                                  cmds_after_breakpoint=['py-print len'])
+                                  cmds_after_breakpoint=['py-up', 'py-print len'])
         self.assertMultilineMatches(bt,
                                     r".*\nbuiltin 'len' = <built-in method len of module object at remote 0x-?[0-9a-f]+>\n.*")
 
@@ -888,7 +888,7 @@
                      "Python was compiled with optimizations")
     def test_basic_command(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
-                                  cmds_after_breakpoint=['py-locals'])
+                                  cmds_after_breakpoint=['py-up', 'py-locals'])
         self.assertMultilineMatches(bt,
                                     r".*\nargs = \(1, 2, 3\)\n.*")
 
@@ -897,7 +897,7 @@
                      "Python was compiled with optimizations")
     def test_locals_after_up(self):
         bt = self.get_stack_trace(script=self.get_sample_script(),
-                                  cmds_after_breakpoint=['py-up', 'py-locals'])
+                                  cmds_after_breakpoint=['py-up', 'py-up', 'py-locals'])
         self.assertMultilineMatches(bt,
                                     r".*\na = 1\nb = 2\nc = 3\n.*")
 
diff --git a/Tools/gdb/libpython.py b/Tools/gdb/libpython.py
--- a/Tools/gdb/libpython.py
+++ b/Tools/gdb/libpython.py
@@ -1492,23 +1492,38 @@
          '''
         if self.is_waiting_for_gil():
             return 'Waiting for the GIL'
-        elif self.is_gc_collect():
+
+        if self.is_gc_collect():
             return 'Garbage-collecting'
-        else:
-            # Detect invocations of PyCFunction instances:
-            older = self.older()
-            if older and older._gdbframe.name() == 'PyCFunction_Call':
-                # Within that frame:
-                #   "func" is the local containing the PyObject* of the
-                # PyCFunctionObject instance
-                #   "f" is the same value, but cast to (PyCFunctionObject*)
-                #   "self" is the (PyObject*) of the 'self'
-                try:
-                    # Use the prettyprinter for the func:
-                    func = older._gdbframe.read_var('func')
-                    return str(func)
-                except RuntimeError:
-                    return 'PyCFunction invocation (unable to read "func")'
+
+        # Detect invocations of PyCFunction instances:
+        older = self.older()
+        if not older:
+            return False
+
+        caller = older._gdbframe.name()
+        if not caller:
+            return False
+
+        if caller == 'PyCFunction_Call':
+            # Within that frame:
+            #   "func" is the local containing the PyObject* of the
+            # PyCFunctionObject instance
+            #   "f" is the same value, but cast to (PyCFunctionObject*)
+            #   "self" is the (PyObject*) of the 'self'
+            try:
+                # Use the prettyprinter for the func:
+                func = older._gdbframe.read_var('func')
+                return str(func)
+            except RuntimeError:
+                return 'PyCFunction invocation (unable to read "func")'
+
+        elif caller == '_PyCFunction_FastCallDict':
+            try:
+                func = older._gdbframe.read_var('func_obj')
+                return str(func)
+            except RuntimeError:
+                return 'PyCFunction invocation (unable to read "func_obj")'
 
         # This frame isn't worth reporting:
         return False

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 22 17:20:32 2016
From: python-checkins at python.org (victor.stinner)
Date: Tue, 22 Nov 2016 22:20:32 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328765=3A_=5Fsre?=
 =?utf-8?q?=2Ecompile=28=29_now_checks_the_type_of_groupindex_and_indexgro?=
 =?utf-8?q?up?=
Message-ID: <20161122222032.88454.76632.F6FA89B3@psf.io>

https://hg.python.org/cpython/rev/1addc5d2c246
changeset:   105343:1addc5d2c246
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 22 23:04:39 2016 +0100
summary:
  Issue #28765: _sre.compile() now checks the type of groupindex and indexgroup

groupindex must a dictionary and indexgroup must be a tuple.

Previously, indexgroup was a list. Use a tuple to reduce the memory usage.

files:
  Lib/sre_compile.py      |  2 +-
  Lib/test/test_re.py     |  2 +-
  Modules/_sre.c          |  6 +++---
  Modules/clinic/_sre.c.h |  6 +++---
  Modules/sre.h           |  4 ++--
  5 files changed, 10 insertions(+), 10 deletions(-)


diff --git a/Lib/sre_compile.py b/Lib/sre_compile.py
--- a/Lib/sre_compile.py
+++ b/Lib/sre_compile.py
@@ -576,5 +576,5 @@
     return _sre.compile(
         pattern, flags | p.pattern.flags, code,
         p.pattern.groups-1,
-        groupindex, indexgroup
+        groupindex, tuple(indexgroup)
         )
diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py
--- a/Lib/test/test_re.py
+++ b/Lib/test/test_re.py
@@ -1506,7 +1506,7 @@
         long_overflow = 2**128
         self.assertRaises(TypeError, re.finditer, "a", {})
         with self.assertRaises(OverflowError):
-            _sre.compile("abc", 0, [long_overflow], 0, [], [])
+            _sre.compile("abc", 0, [long_overflow], 0, {}, ())
         with self.assertRaises(TypeError):
             _sre.compile({}, 0, [], 0, [], [])
 
diff --git a/Modules/_sre.c b/Modules/_sre.c
--- a/Modules/_sre.c
+++ b/Modules/_sre.c
@@ -1438,8 +1438,8 @@
     flags: int
     code: object(subclass_of='&PyList_Type')
     groups: Py_ssize_t
-    groupindex: object
-    indexgroup: object
+    groupindex: object(subclass_of='&PyDict_Type')
+    indexgroup: object(subclass_of='&PyTuple_Type')
 
 [clinic start generated code]*/
 
@@ -1447,7 +1447,7 @@
 _sre_compile_impl(PyObject *module, PyObject *pattern, int flags,
                   PyObject *code, Py_ssize_t groups, PyObject *groupindex,
                   PyObject *indexgroup)
-/*[clinic end generated code: output=ef9c2b3693776404 input=7d059ec8ae1edb85]*/
+/*[clinic end generated code: output=ef9c2b3693776404 input=0a68476dbbe5db30]*/
 {
     /* "compile" pattern descriptor to pattern object */
 
diff --git a/Modules/clinic/_sre.c.h b/Modules/clinic/_sre.c.h
--- a/Modules/clinic/_sre.c.h
+++ b/Modules/clinic/_sre.c.h
@@ -438,7 +438,7 @@
 {
     PyObject *return_value = NULL;
     static const char * const _keywords[] = {"pattern", "flags", "code", "groups", "groupindex", "indexgroup", NULL};
-    static _PyArg_Parser _parser = {"OiO!nOO:compile", _keywords, 0};
+    static _PyArg_Parser _parser = {"OiO!nO!O!:compile", _keywords, 0};
     PyObject *pattern;
     int flags;
     PyObject *code;
@@ -447,7 +447,7 @@
     PyObject *indexgroup;
 
     if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser,
-        &pattern, &flags, &PyList_Type, &code, &groups, &groupindex, &indexgroup)) {
+        &pattern, &flags, &PyList_Type, &code, &groups, &PyDict_Type, &groupindex, &PyTuple_Type, &indexgroup)) {
         goto exit;
     }
     return_value = _sre_compile_impl(module, pattern, flags, code, groups, groupindex, indexgroup);
@@ -728,4 +728,4 @@
 {
     return _sre_SRE_Scanner_search_impl(self);
 }
-/*[clinic end generated code: output=a4a246bca1963bc9 input=a9049054013a1b77]*/
+/*[clinic end generated code: output=b74b16d90f207358 input=a9049054013a1b77]*/
diff --git a/Modules/sre.h b/Modules/sre.h
--- a/Modules/sre.h
+++ b/Modules/sre.h
@@ -27,8 +27,8 @@
 typedef struct {
     PyObject_VAR_HEAD
     Py_ssize_t groups; /* must be first! */
-    PyObject* groupindex;
-    PyObject* indexgroup;
+    PyObject* groupindex; /* dict */
+    PyObject* indexgroup; /* tuple */
     /* compatibility */
     PyObject* pattern; /* pattern source (or None) */
     int flags; /* flags used when compiling pattern source */

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Wed Nov 23 04:06:02 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Wed, 23 Nov 2016 09:06:02 +0000
Subject: [Python-checkins] Daily reference leaks (1addc5d2c246): sum=13
Message-ID: <20161123090602.113930.75325.EBA5A7F4@psf.io>

results for 1addc5d2c246 on branch "default"
--------------------------------------------

test_collections leaked [0, 0, 7] memory blocks, sum=7
test_functools leaked [0, 3, 1] memory blocks, sum=4
test_multiprocessing_forkserver leaked [2, -1, 1] memory blocks, sum=2


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog3onegd', '--timeout', '7200']

From python-checkins at python.org  Wed Nov 23 07:19:24 2016
From: python-checkins at python.org (xiang.zhang)
Date: Wed, 23 Nov 2016 12:19:24 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328774=3A_Fix_star?=
 =?utf-8?q?t/end_pos_in_unicode=5Fencode=5Fucs1=28=29=2E?=
Message-ID: <20161123121924.113075.32680.658CD8FC@psf.io>

https://hg.python.org/cpython/rev/3d660ed2a60e
changeset:   105344:3d660ed2a60e
user:        Xiang Zhang <angwerzx at 126.com>
date:        Wed Nov 23 19:34:01 2016 +0800
summary:
  Issue #28774: Fix start/end pos in unicode_encode_ucs1().

Fix error position of the unicode error in ASCII and Latin1
encoders when a string returned by the error handler contains multiple
non-encodable characters (non-ASCII for the ASCII codec, characters out
of the U+0000-U+00FF range for Latin1).

files:
  Misc/NEWS               |  5 +++++
  Objects/unicodeobject.c |  4 ++--
  2 files changed, 7 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,11 @@
 Core and Builtins
 -----------------
 
+- Issue #28774: Fix error position of the unicode error in ASCII and Latin1
+  encoders when a string returned by the error handler contains multiple
+  non-encodable characters (non-ASCII for the ASCII codec, characters out
+  of the U+0000-U+00FF range for Latin1).
+
 - Issue #28731: Optimize _PyDict_NewPresized() to create correct size dict.
   Improve speed of dict literal with constant keys up to 30%.
 
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -6798,7 +6798,7 @@
                     goto onError;
 
                 /* subtract preallocated bytes */
-                writer.min_size -= 1;
+                writer.min_size -= newpos - collstart;
 
                 if (PyBytes_Check(rep)) {
                     /* Directly copy bytes result to output. */
@@ -6835,7 +6835,7 @@
                             ch = PyUnicode_READ_CHAR(rep, i);
                             if (ch >= limit) {
                                 raise_encode_exception(&exc, encoding, unicode,
-                                                       pos, pos+1, reason);
+                                                       collstart, collend, reason);
                                 goto onError;
                             }
                             *str = (char)ch;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 23 08:17:29 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 23 Nov 2016 13:17:29 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328774=3A_Simplifi?=
 =?utf-8?q?ed_encoding_a_str_result_of_an_error_handler_in_ASCII?=
Message-ID: <20161123131728.88335.49255.DA6D8F90@psf.io>

https://hg.python.org/cpython/rev/3addf93f4111
changeset:   105345:3addf93f4111
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Nov 23 15:13:00 2016 +0200
summary:
  Issue #28774: Simplified encoding a str result of an error handler in ASCII
and Latin1 encoders.

files:
  Objects/unicodeobject.c |  38 +++++++++-------------------
  1 files changed, 12 insertions(+), 26 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -6814,33 +6814,19 @@
                     if (PyUnicode_READY(rep) < 0)
                         goto onError;
 
-                    if (PyUnicode_IS_ASCII(rep)) {
-                        /* Fast path: all characters are smaller than limit */
-                        assert(limit >= 128);
-                        assert(PyUnicode_KIND(rep) == PyUnicode_1BYTE_KIND);
-                        str = _PyBytesWriter_WriteBytes(&writer, str,
-                                                        PyUnicode_DATA(rep),
-                                                        PyUnicode_GET_LENGTH(rep));
+                    if (limit == 256 ?
+                        PyUnicode_KIND(rep) != PyUnicode_1BYTE_KIND :
+                        !PyUnicode_IS_ASCII(rep))
+                    {
+                        /* Not all characters are smaller than limit */
+                        raise_encode_exception(&exc, encoding, unicode,
+                                               collstart, collend, reason);
+                        goto onError;
                     }
-                    else {
-                        Py_ssize_t repsize = PyUnicode_GET_LENGTH(rep);
-
-                        str = _PyBytesWriter_Prepare(&writer, str, repsize);
-                        if (str == NULL)
-                            goto onError;
-
-                        /* check if there is anything unencodable in the
-                           replacement and copy it to the output */
-                        for (i = 0; repsize-->0; ++i, ++str) {
-                            ch = PyUnicode_READ_CHAR(rep, i);
-                            if (ch >= limit) {
-                                raise_encode_exception(&exc, encoding, unicode,
-                                                       collstart, collend, reason);
-                                goto onError;
-                            }
-                            *str = (char)ch;
-                        }
-                    }
+                    assert(PyUnicode_KIND(rep) == PyUnicode_1BYTE_KIND);
+                    str = _PyBytesWriter_WriteBytes(&writer, str,
+                                                    PyUnicode_DATA(rep),
+                                                    PyUnicode_GET_LENGTH(rep));
                 }
                 pos = newpos;
                 Py_CLEAR(rep);

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Wed Nov 23 09:57:26 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Wed, 23 Nov 2016 14:57:26 +0000
Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-11-23
Message-ID: <0d3601c6-2ab0-4b1d-ac26-eecaa326fb12@irsmsx106.ger.corp.intel.com>

No new revisions. Here are the previous results:

Results for project Python 2.7, build date 2016-11-23 03:45:49 +0000
commit:		6eefdf0fade6
previous commit:	4de956751cf1
revision date:	2016-11-21 09:37:18 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.17%         -1.09%          3.37%             8.32%
:-)             pybench      0.13%         -0.05%          5.71%             3.52%
:-(            regex_v8      0.57%          0.32%         -2.32%            11.28%
:-)               nbody      0.86%         -0.57%          7.89%             5.91%
:-)        json_dump_v2      0.30%          0.72%          3.79%             7.84%
:-|      normal_startup      1.00%         -0.84%         -0.31%             2.71%
:-)             ssbench      0.23%          0.11%          2.63%             1.30%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-11-23/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Wed Nov 23 09:58:01 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Wed, 23 Nov 2016 14:58:01 +0000
Subject: [Python-checkins] BAD Benchmark Results for Python Default
 2016-11-23
Message-ID: <726ad67f-5a03-442e-83d0-ee62451ee3e7@irsmsx106.ger.corp.intel.com>

Results for project Python default, build date 2016-11-23 03:01:54 +0000
commit:		1addc5d2c246
previous commit:	a0da176233b9
revision date:	2016-11-22 22:04:39 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.20%         -2.50%          5.10%            15.44%
:-)             pybench      0.23%         -0.15%          5.82%             5.79%
:-(            regex_v8      3.89%         -0.43%         -3.03%             3.07%
:-|               nbody      0.09%         -0.96%          1.78%             5.75%
:-(        json_dump_v2      0.26%         -0.13%         -4.74%            10.80%
:-|      normal_startup      0.42%         -1.01%         -0.17%             6.87%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-default-2016-11-23/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Wed Nov 23 15:23:30 2016
From: python-checkins at python.org (steve.dower)
Date: Wed, 23 Nov 2016 20:23:30 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4Nzgz?=
 =?utf-8?q?=3A_Embedded_and_nuget_packages_incorrect_reference_missing?=
Message-ID: <20161123202330.113582.85611.12007058@psf.io>

https://hg.python.org/cpython/rev/f7aa200bed8d
changeset:   105346:f7aa200bed8d
branch:      3.5
parent:      105308:0ee76f3afd70
user:        Steve Dower <steve.dower at microsoft.com>
date:        Wed Nov 23 10:23:47 2016 -0800
summary:
  Issue #28783: Embedded and nuget packages incorrect reference missing bdist_wininst command.

files:
  Tools/msi/distutils.command.__init__.py |  32 +++++++++++++
  Tools/msi/make_zip.py                   |  18 ++++++-
  2 files changed, 47 insertions(+), 3 deletions(-)


diff --git a/Tools/msi/distutils.command.__init__.py b/Tools/msi/distutils.command.__init__.py
new file mode 100644
--- /dev/null
+++ b/Tools/msi/distutils.command.__init__.py
@@ -0,0 +1,32 @@
+"""distutils.command
+
+Package containing implementation of all the standard Distutils
+commands."""
+
+__all__ = ['build',
+           'build_py',
+           'build_ext',
+           'build_clib',
+           'build_scripts',
+           'clean',
+           'install',
+           'install_lib',
+           'install_headers',
+           'install_scripts',
+           'install_data',
+           'sdist',
+           'register',
+           'bdist',
+           'bdist_dumb',
+           'bdist_rpm',
+           # This command is not included in this package
+           #'bdist_wininst',
+           'check',
+           'upload',
+           # These two are reserved for future use:
+           #'bdist_sdux',
+           #'bdist_pkgtool',
+           # Note:
+           # bdist_packager is not included because it only provides
+           # an abstract base class
+          ]
diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py
--- a/Tools/msi/make_zip.py
+++ b/Tools/msi/make_zip.py
@@ -7,6 +7,7 @@
 import os
 import tempfile
 
+from itertools import chain
 from pathlib import Path
 from zipfile import ZipFile, ZIP_DEFLATED
 import subprocess
@@ -74,6 +75,10 @@
     if name in EXCLUDE_FILE_FROM_LIBRARY:
         return False
 
+    # Special code is included below to patch this file back in
+    if [d.lower() for d in p.parts[-3:]] == ['distutils', 'command', '__init__.py']:
+        return False
+
     suffix = p.suffix.lower()
     return suffix not in {'.pyc', '.pyo', '.exe'}
 
@@ -203,10 +208,17 @@
     try:
         for t, s, p, c in layout:
             if s == '$build':
-                s = build
+                fs = build
             else:
-                s = source / s
-            copied = copy_to_layout(temp / t.rstrip('/'), rglob(s, p, c))
+                fs = source / s
+            files = rglob(fs, p, c)
+            extra_files = []
+            if s == 'Lib' and p == '**/*':
+                extra_files.append((
+                    source / 'tools' / 'msi' / 'distutils.command.__init__.py',
+                    Path('distutils') / 'command' / '__init__.py'
+                ))
+            copied = copy_to_layout(temp / t.rstrip('/'), chain(files, extra_files))
             print('Copied {} files'.format(copied))
 
         with open(str(temp / 'pyvenv.cfg'), 'w') as f:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 23 15:23:30 2016
From: python-checkins at python.org (steve.dower)
Date: Wed, 23 Nov 2016 20:23:30 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328783=3A_Embedded_and_nuget_packages_incorrect_?=
 =?utf-8?q?reference_missing?=
Message-ID: <20161123202330.88335.63341.32BF89A3@psf.io>

https://hg.python.org/cpython/rev/831f73efe3a6
changeset:   105348:831f73efe3a6
parent:      105345:3addf93f4111
parent:      105347:a3755890545c
user:        Steve Dower <steve.dower at microsoft.com>
date:        Wed Nov 23 12:23:16 2016 -0800
summary:
  Issue #28783: Embedded and nuget packages incorrect reference missing bdist_wininst command.

files:
  Tools/msi/distutils.command.__init__.py |  32 +++++++++++++
  Tools/msi/make_zip.py                   |  19 ++++++-
  2 files changed, 48 insertions(+), 3 deletions(-)


diff --git a/Tools/msi/distutils.command.__init__.py b/Tools/msi/distutils.command.__init__.py
new file mode 100644
--- /dev/null
+++ b/Tools/msi/distutils.command.__init__.py
@@ -0,0 +1,32 @@
+"""distutils.command
+
+Package containing implementation of all the standard Distutils
+commands."""
+
+__all__ = ['build',
+           'build_py',
+           'build_ext',
+           'build_clib',
+           'build_scripts',
+           'clean',
+           'install',
+           'install_lib',
+           'install_headers',
+           'install_scripts',
+           'install_data',
+           'sdist',
+           'register',
+           'bdist',
+           'bdist_dumb',
+           'bdist_rpm',
+           # This command is not included in this package
+           #'bdist_wininst',
+           'check',
+           'upload',
+           # These two are reserved for future use:
+           #'bdist_sdux',
+           #'bdist_pkgtool',
+           # Note:
+           # bdist_packager is not included because it only provides
+           # an abstract base class
+          ]
diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py
--- a/Tools/msi/make_zip.py
+++ b/Tools/msi/make_zip.py
@@ -7,6 +7,7 @@
 import os
 import tempfile
 
+from itertools import chain
 from pathlib import Path
 from zipfile import ZipFile, ZIP_DEFLATED
 import subprocess
@@ -42,6 +43,7 @@
 }
 
 EXCLUDE_FILE_FROM_LIBS = {
+    'liblzma',
     'ssleay',
     'libeay',
     'python3stub',
@@ -77,6 +79,10 @@
     if name in EXCLUDE_FILE_FROM_LIBRARY:
         return False
 
+    # Special code is included below to patch this file back in
+    if [d.lower() for d in p.parts[-3:]] == ['distutils', 'command', '__init__.py']:
+        return False
+
     suffix = p.suffix.lower()
     return suffix not in {'.pyc', '.pyo', '.exe'}
 
@@ -173,7 +179,7 @@
 def main():
     parser = argparse.ArgumentParser()
     parser.add_argument('-s', '--source', metavar='dir', help='The directory containing the repository root', type=Path)
-    parser.add_argument('-o', '--out', metavar='file', help='The name of the output self-extracting archive', type=Path, default=None)
+    parser.add_argument('-o', '--out', metavar='file', help='The name of the output archive', type=Path, default=None)
     parser.add_argument('-t', '--temp', metavar='dir', help='A directory to temporarily extract files into', type=Path, default=None)
     parser.add_argument('-e', '--embed', help='Create an embedding layout', action='store_true', default=False)
     parser.add_argument('-a', '--arch', help='Specify the architecture to use (win32/amd64)', type=str, default="win32")
@@ -207,8 +213,15 @@
 
     try:
         for t, s, p, c in layout:
-            s = source / s.replace("$arch", arch)
-            copied = copy_to_layout(temp / t.rstrip('/'), rglob(s, p, c))
+            fs = source / s.replace("$arch", arch)
+            files = rglob(fs, p, c)
+            extra_files = []
+            if s == 'Lib' and p == '**/*':
+                extra_files.append((
+                    source / 'tools' / 'msi' / 'distutils.command.__init__.py',
+                    Path('distutils') / 'command' / '__init__.py'
+                ))
+            copied = copy_to_layout(temp / t.rstrip('/'), chain(files, extra_files))
             print('Copied {} files'.format(copied))
 
         if ns.embed:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 23 15:23:30 2016
From: python-checkins at python.org (steve.dower)
Date: Wed, 23 Nov 2016 20:23:30 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328783=3A_Embedded_and_nuget_packages_incorrect_refere?=
 =?utf-8?q?nce_missing?=
Message-ID: <20161123202330.88547.19976.3483E716@psf.io>

https://hg.python.org/cpython/rev/a3755890545c
changeset:   105347:a3755890545c
branch:      3.6
parent:      105340:749c5d6c4ba5
parent:      105346:f7aa200bed8d
user:        Steve Dower <steve.dower at microsoft.com>
date:        Wed Nov 23 11:42:35 2016 -0800
summary:
  Issue #28783: Embedded and nuget packages incorrect reference missing bdist_wininst command.

files:
  Tools/msi/distutils.command.__init__.py |  32 +++++++++++++
  Tools/msi/make_zip.py                   |  19 ++++++-
  2 files changed, 48 insertions(+), 3 deletions(-)


diff --git a/Tools/msi/distutils.command.__init__.py b/Tools/msi/distutils.command.__init__.py
new file mode 100644
--- /dev/null
+++ b/Tools/msi/distutils.command.__init__.py
@@ -0,0 +1,32 @@
+"""distutils.command
+
+Package containing implementation of all the standard Distutils
+commands."""
+
+__all__ = ['build',
+           'build_py',
+           'build_ext',
+           'build_clib',
+           'build_scripts',
+           'clean',
+           'install',
+           'install_lib',
+           'install_headers',
+           'install_scripts',
+           'install_data',
+           'sdist',
+           'register',
+           'bdist',
+           'bdist_dumb',
+           'bdist_rpm',
+           # This command is not included in this package
+           #'bdist_wininst',
+           'check',
+           'upload',
+           # These two are reserved for future use:
+           #'bdist_sdux',
+           #'bdist_pkgtool',
+           # Note:
+           # bdist_packager is not included because it only provides
+           # an abstract base class
+          ]
diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py
--- a/Tools/msi/make_zip.py
+++ b/Tools/msi/make_zip.py
@@ -7,6 +7,7 @@
 import os
 import tempfile
 
+from itertools import chain
 from pathlib import Path
 from zipfile import ZipFile, ZIP_DEFLATED
 import subprocess
@@ -42,6 +43,7 @@
 }
 
 EXCLUDE_FILE_FROM_LIBS = {
+    'liblzma',
     'ssleay',
     'libeay',
     'python3stub',
@@ -77,6 +79,10 @@
     if name in EXCLUDE_FILE_FROM_LIBRARY:
         return False
 
+    # Special code is included below to patch this file back in
+    if [d.lower() for d in p.parts[-3:]] == ['distutils', 'command', '__init__.py']:
+        return False
+
     suffix = p.suffix.lower()
     return suffix not in {'.pyc', '.pyo', '.exe'}
 
@@ -173,7 +179,7 @@
 def main():
     parser = argparse.ArgumentParser()
     parser.add_argument('-s', '--source', metavar='dir', help='The directory containing the repository root', type=Path)
-    parser.add_argument('-o', '--out', metavar='file', help='The name of the output self-extracting archive', type=Path, default=None)
+    parser.add_argument('-o', '--out', metavar='file', help='The name of the output archive', type=Path, default=None)
     parser.add_argument('-t', '--temp', metavar='dir', help='A directory to temporarily extract files into', type=Path, default=None)
     parser.add_argument('-e', '--embed', help='Create an embedding layout', action='store_true', default=False)
     parser.add_argument('-a', '--arch', help='Specify the architecture to use (win32/amd64)', type=str, default="win32")
@@ -207,8 +213,15 @@
 
     try:
         for t, s, p, c in layout:
-            s = source / s.replace("$arch", arch)
-            copied = copy_to_layout(temp / t.rstrip('/'), rglob(s, p, c))
+            fs = source / s.replace("$arch", arch)
+            files = rglob(fs, p, c)
+            extra_files = []
+            if s == 'Lib' and p == '**/*':
+                extra_files.append((
+                    source / 'tools' / 'msi' / 'distutils.command.__init__.py',
+                    Path('distutils') / 'command' / '__init__.py'
+                ))
+            copied = copy_to_layout(temp / t.rstrip('/'), chain(files, extra_files))
             print('Copied {} files'.format(copied))
 
         if ns.embed:

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 24 03:21:56 2016
From: python-checkins at python.org (inada.naoki)
Date: Thu, 24 Nov 2016 08:21:56 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328532=3A_Add_what=27s_new_entry_for_python_-VV_?=
 =?utf-8?q?option?=
Message-ID: <20161124082155.116234.92120.478710CB@psf.io>

https://hg.python.org/cpython/rev/3f9f13077ca8
changeset:   105350:3f9f13077ca8
parent:      105348:831f73efe3a6
parent:      105349:5a29ab5a4785
user:        INADA Naoki <songofacandy at gmail.com>
date:        Thu Nov 24 17:21:47 2016 +0900
summary:
  Issue #28532: Add what's new entry for python -VV option

files:
  Doc/using/cmdline.rst |   2 ++
  Doc/whatsnew/3.6.rst  |  12 ++++++++++++
  2 files changed, 14 insertions(+), 0 deletions(-)


diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst
--- a/Doc/using/cmdline.rst
+++ b/Doc/using/cmdline.rst
@@ -187,6 +187,8 @@
        Python 3.6.0b2+ (3.6:84a3c5003510+, Oct 26 2016, 02:33:55)
        [GCC 6.2.0 20161005]
 
+   .. versionadded:: 3.6
+      The ``-VV`` option.
 
 .. _using-on-misc-options:
 
diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -1832,6 +1832,18 @@
   functions will now accept :term:`path-like objects <path-like object>`.
 
 
+Other Improvements
+==================
+
+* When :option:`--version` (short form: :option:`-V`) is supplied twice,
+  Python prints :data:`sys.version` for detailed information.
+
+  .. code-block:: shell-session
+
+    $ ./python -VV
+    Python 3.6.0b4+ (3.6:223967b49e49+, Nov 21 2016, 20:55:04)
+    [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]
+
 
 Deprecated
 ==========

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 24 03:21:56 2016
From: python-checkins at python.org (inada.naoki)
Date: Thu, 24 Nov 2016 08:21:56 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NTMy?=
 =?utf-8?q?=3A_Add_what=27s_new_entry_for_python_-VV_option?=
Message-ID: <20161124082155.107660.78180.9EFCA3AB@psf.io>

https://hg.python.org/cpython/rev/5a29ab5a4785
changeset:   105349:5a29ab5a4785
branch:      3.6
parent:      105347:a3755890545c
user:        INADA Naoki <songofacandy at gmail.com>
date:        Thu Nov 24 17:20:40 2016 +0900
summary:
  Issue #28532: Add what's new entry for python -VV option

files:
  Doc/using/cmdline.rst |   2 ++
  Doc/whatsnew/3.6.rst  |  12 ++++++++++++
  2 files changed, 14 insertions(+), 0 deletions(-)


diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst
--- a/Doc/using/cmdline.rst
+++ b/Doc/using/cmdline.rst
@@ -187,6 +187,8 @@
        Python 3.6.0b2+ (3.6:84a3c5003510+, Oct 26 2016, 02:33:55)
        [GCC 6.2.0 20161005]
 
+   .. versionadded:: 3.6
+      The ``-VV`` option.
 
 .. _using-on-misc-options:
 
diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -1832,6 +1832,18 @@
   functions will now accept :term:`path-like objects <path-like object>`.
 
 
+Other Improvements
+==================
+
+* When :option:`--version` (short form: :option:`-V`) is supplied twice,
+  Python prints :data:`sys.version` for detailed information.
+
+  .. code-block:: shell-session
+
+    $ ./python -VV
+    Python 3.6.0b4+ (3.6:223967b49e49+, Nov 21 2016, 20:55:04)
+    [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]
+
 
 Deprecated
 ==========

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Thu Nov 24 04:10:12 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Thu, 24 Nov 2016 09:10:12 +0000
Subject: [Python-checkins] Daily reference leaks (831f73efe3a6): sum=7
Message-ID: <20161124091012.107534.49720.9D28DDF1@psf.io>

results for 831f73efe3a6 on branch "default"
--------------------------------------------

test_collections leaked [7, -7, 1] memory blocks, sum=1
test_functools leaked [0, 3, 1] memory blocks, sum=4
test_multiprocessing_spawn leaked [2, 0, 0] memory blocks, sum=2


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogp0wuHx', '--timeout', '7200']

From lp_benchmark_robot at intel.com  Thu Nov 24 07:16:50 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Thu, 24 Nov 2016 12:16:50 +0000
Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-11-24
Message-ID: <00b351cf-9b5f-4f4e-827e-714efbe7e385@irsmsx102.ger.corp.intel.com>

No new revisions. Here are the previous results:

Results for project Python 2.7, build date 2016-11-24 03:45:39 +0000
commit:		6eefdf0fade6
previous commit:	4de956751cf1
revision date:	2016-11-21 09:37:18 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.17%         -1.09%          3.37%             8.32%
:-)             pybench      0.13%         -0.05%          5.71%             3.52%
:-(            regex_v8      0.57%          0.32%         -2.32%            11.28%
:-)               nbody      0.86%         -0.57%          7.89%             5.91%
:-)        json_dump_v2      0.30%          0.72%          3.79%             7.84%
:-|      normal_startup      1.00%         -0.84%         -0.31%             2.71%
:-)             ssbench      0.23%          0.11%          2.63%             1.30%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-11-24/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Thu Nov 24 07:18:26 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Thu, 24 Nov 2016 12:18:26 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python Default
 2016-11-24
Message-ID: <7b448b28-763e-4409-ba05-e51daa524c52@irsmsx102.ger.corp.intel.com>

Results for project Python default, build date 2016-11-24 03:01:50 +0000
commit:		831f73efe3a6
previous commit:	1addc5d2c246
revision date:	2016-11-23 20:23:16 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.15%          1.91%          6.92%            13.84%
:-)             pybench      0.19%         -0.33%          5.51%             5.11%
:-(            regex_v8      3.87%          0.72%         -2.29%             1.57%
:-)               nbody      0.09%          1.03%          2.79%             3.03%
:-(        json_dump_v2      0.32%          1.22%         -3.46%             9.05%
:-|      normal_startup      0.95%         -0.06%         -0.47%             6.83%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-11-24/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Thu Nov 24 13:51:04 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Thu, 24 Nov 2016 18:51:04 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_merge?=
Message-ID: <20161124185104.107562.43314.8C463B05@psf.io>

https://hg.python.org/cpython/rev/971ac6837060
changeset:   105352:971ac6837060
parent:      105350:3f9f13077ca8
parent:      105351:e11df6aa5bf1
user:        Raymond Hettinger <python at rcn.com>
date:        Thu Nov 24 10:50:57 2016 -0800
summary:
  merge

files:
  Lib/test/test_with.py |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_with.py b/Lib/test/test_with.py
--- a/Lib/test/test_with.py
+++ b/Lib/test/test_with.py
@@ -117,7 +117,7 @@
         def fooLacksEnter():
             foo = LacksEnter()
             with foo: pass
-        self.assertRaisesRegexp(AttributeError, '__enter__', fooLacksEnter)
+        self.assertRaisesRegex(AttributeError, '__enter__', fooLacksEnter)
 
     def testEnterAttributeError2(self):
         class LacksEnterAndExit(object):
@@ -126,7 +126,7 @@
         def fooLacksEnterAndExit():
             foo = LacksEnterAndExit()
             with foo: pass
-        self.assertRaisesRegexp(AttributeError, '__enter__', fooLacksEnterAndExit)
+        self.assertRaisesRegex(AttributeError, '__enter__', fooLacksEnterAndExit)
 
     def testExitAttributeError(self):
         class LacksExit(object):
@@ -136,7 +136,7 @@
         def fooLacksExit():
             foo = LacksExit()
             with foo: pass
-        self.assertRaisesRegexp(AttributeError, '__exit__', fooLacksExit)
+        self.assertRaisesRegex(AttributeError, '__exit__', fooLacksExit)
 
     def assertRaisesSyntaxError(self, codestr):
         def shouldRaiseSyntaxError(s):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 24 13:51:04 2016
From: python-checkins at python.org (raymond.hettinger)
Date: Thu, 24 Nov 2016 18:51:04 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI3MTAw?=
 =?utf-8?q?=3A__Silence_deprecation_warning_in_Lib/test/test=5Fwith=2Epy?=
Message-ID: <20161124185104.108103.95953.FB169E20@psf.io>

https://hg.python.org/cpython/rev/e11df6aa5bf1
changeset:   105351:e11df6aa5bf1
branch:      3.6
parent:      105349:5a29ab5a4785
user:        Raymond Hettinger <python at rcn.com>
date:        Thu Nov 24 10:50:34 2016 -0800
summary:
  Issue #27100:  Silence deprecation warning in Lib/test/test_with.py

files:
  Lib/test/test_with.py |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_with.py b/Lib/test/test_with.py
--- a/Lib/test/test_with.py
+++ b/Lib/test/test_with.py
@@ -117,7 +117,7 @@
         def fooLacksEnter():
             foo = LacksEnter()
             with foo: pass
-        self.assertRaisesRegexp(AttributeError, '__enter__', fooLacksEnter)
+        self.assertRaisesRegex(AttributeError, '__enter__', fooLacksEnter)
 
     def testEnterAttributeError2(self):
         class LacksEnterAndExit(object):
@@ -126,7 +126,7 @@
         def fooLacksEnterAndExit():
             foo = LacksEnterAndExit()
             with foo: pass
-        self.assertRaisesRegexp(AttributeError, '__enter__', fooLacksEnterAndExit)
+        self.assertRaisesRegex(AttributeError, '__enter__', fooLacksEnterAndExit)
 
     def testExitAttributeError(self):
         class LacksExit(object):
@@ -136,7 +136,7 @@
         def fooLacksExit():
             foo = LacksExit()
             with foo: pass
-        self.assertRaisesRegexp(AttributeError, '__exit__', fooLacksExit)
+        self.assertRaisesRegex(AttributeError, '__exit__', fooLacksExit)
 
     def assertRaisesSyntaxError(self, codestr):
         def shouldRaiseSyntaxError(s):

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 24 14:56:13 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Thu, 24 Nov 2016 19:56:13 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4Nzcz?=
 =?utf-8?q?=3A_Add_typing=2EFrozenSet_docs=2E_=28Manuel_Krebber=29?=
Message-ID: <20161124195612.108353.61126.82EA2B31@psf.io>

https://hg.python.org/cpython/rev/5d83b6a9b568
changeset:   105353:5d83b6a9b568
branch:      3.5
parent:      105346:f7aa200bed8d
user:        Guido van Rossum <guido at python.org>
date:        Thu Nov 24 11:56:00 2016 -0800
summary:
  Issue #28773: Add typing.FrozenSet docs. (Manuel Krebber)

files:
  Doc/library/typing.rst |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst
--- a/Doc/library/typing.rst
+++ b/Doc/library/typing.rst
@@ -578,6 +578,10 @@
 
    A generic version of :class:`builtins.set <set>`.
 
+.. class:: FrozenSet(frozenset, AbstractSet[T_co])
+
+   A generic version of :class:`builtins.frozenset <frozenset>`.
+
 .. class:: MappingView(Sized, Iterable[T_co])
 
    A generic version of :class:`collections.abc.MappingView`.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 24 14:57:18 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Thu, 24 Nov 2016 19:57:18 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328773=3A_Add_typing=2EFrozenSet_docs=2E_=28Manu?=
 =?utf-8?b?ZWwgS3JlYmJlcikgKDMuNi0+My43KQ==?=
Message-ID: <20161124195718.32417.78849.8A8D27BD@psf.io>

https://hg.python.org/cpython/rev/00bfe5bd3553
changeset:   105355:00bfe5bd3553
parent:      105352:971ac6837060
parent:      105353:5d83b6a9b568
user:        Guido van Rossum <guido at python.org>
date:        Thu Nov 24 11:57:05 2016 -0800
summary:
  Issue #28773: Add typing.FrozenSet docs. (Manuel Krebber) (3.6->3.7)

files:
  Doc/library/typing.rst |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst
--- a/Doc/library/typing.rst
+++ b/Doc/library/typing.rst
@@ -584,6 +584,10 @@
 
    A generic version of :class:`builtins.set <set>`.
 
+.. class:: FrozenSet(frozenset, AbstractSet[T_co])
+
+   A generic version of :class:`builtins.frozenset <frozenset>`.
+
 .. class:: MappingView(Sized, Iterable[T_co])
 
    A generic version of :class:`collections.abc.MappingView`.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 24 14:57:18 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Thu, 24 Nov 2016 19:57:18 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328773=3A_Add_typing=2EFrozenSet_docs=2E_=28Manuel_Kre?=
 =?utf-8?b?YmJlcikgKDMuNS0+My42KQ==?=
Message-ID: <20161124195718.116102.42951.D6D1BBDD@psf.io>

https://hg.python.org/cpython/rev/7059e68db068
changeset:   105354:7059e68db068
branch:      3.6
parent:      105351:e11df6aa5bf1
parent:      105353:5d83b6a9b568
user:        Guido van Rossum <guido at python.org>
date:        Thu Nov 24 11:56:40 2016 -0800
summary:
  Issue #28773: Add typing.FrozenSet docs. (Manuel Krebber) (3.5->3.6)

files:
  Doc/library/typing.rst |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst
--- a/Doc/library/typing.rst
+++ b/Doc/library/typing.rst
@@ -584,6 +584,10 @@
 
    A generic version of :class:`builtins.set <set>`.
 
+.. class:: FrozenSet(frozenset, AbstractSet[T_co])
+
+   A generic version of :class:`builtins.frozenset <frozenset>`.
+
 .. class:: MappingView(Sized, Iterable[T_co])
 
    A generic version of :class:`collections.abc.MappingView`.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 24 16:35:16 2016
From: python-checkins at python.org (victor.stinner)
Date: Thu, 24 Nov 2016 21:35:16 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogRml4IF9QeUdlbl95?=
 =?utf-8?b?Zigp?=
Message-ID: <20161124213515.116823.1196.9256987D@psf.io>

https://hg.python.org/cpython/rev/303cedfb9e7a
changeset:   105357:303cedfb9e7a
branch:      3.6
parent:      105354:7059e68db068
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Thu Nov 24 22:33:01 2016 +0100
summary:
  Fix _PyGen_yf()

Issue #28782: Fix a bug in the implementation ``yield from`` when checking
if the next instruction is YIELD_FROM. Regression introduced by WORDCODE
(issue #26647).

Reviewed by Serhiy Storchaka and Yury Selivanov.

files:
  Misc/NEWS           |  4 ++++
  Objects/genobject.c |  9 +++++++++
  Python/ceval.c      |  1 +
  3 files changed, 14 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,10 @@
 Core and Builtins
 -----------------
 
+- Issue #28782: Fix a bug in the implementation ``yield from`` when checking
+  if the next instruction is YIELD_FROM. Regression introduced by WORDCODE
+  (issue #26647).
+
 Library
 -------
 
diff --git a/Objects/genobject.c b/Objects/genobject.c
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -355,6 +355,14 @@
         PyObject *bytecode = f->f_code->co_code;
         unsigned char *code = (unsigned char *)PyBytes_AS_STRING(bytecode);
 
+        if (f->f_lasti < 0) {
+            /* Return immediately if the frame didn't start yet. YIELD_FROM
+               always come after LOAD_CONST: a code object should not start
+               with YIELD_FROM */
+            assert(code[0] != YIELD_FROM);
+            return NULL;
+        }
+
         if (code[f->f_lasti + sizeof(_Py_CODEUNIT)] != YIELD_FROM)
             return NULL;
         yf = f->f_stacktop[-1];
@@ -463,6 +471,7 @@
             assert(ret == yf);
             Py_DECREF(ret);
             /* Termination repetition of YIELD_FROM */
+            assert(gen->gi_frame->f_lasti >= 0);
             gen->gi_frame->f_lasti += sizeof(_Py_CODEUNIT);
             if (_PyGen_FetchStopIterationValue(&val) == 0) {
                 ret = gen_send_ex(gen, val, 0, 0);
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -2043,6 +2043,7 @@
             f->f_stacktop = stack_pointer;
             why = WHY_YIELD;
             /* and repeat... */
+            assert(f->f_lasti >= (int)sizeof(_Py_CODEUNIT));
             f->f_lasti -= sizeof(_Py_CODEUNIT);
             goto fast_yield;
         }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 24 16:35:16 2016
From: python-checkins at python.org (victor.stinner)
Date: Thu, 24 Nov 2016 21:35:16 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42?=
Message-ID: <20161124213515.108541.75730.8992C53C@psf.io>

https://hg.python.org/cpython/rev/72b8a7423cac
changeset:   105356:72b8a7423cac
parent:      105355:00bfe5bd3553
parent:      105354:7059e68db068
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Thu Nov 24 22:31:25 2016 +0100
summary:
  Merge 3.6

files:



-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 24 16:35:16 2016
From: python-checkins at python.org (victor.stinner)
Date: Thu, 24 Nov 2016 21:35:16 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42?=
Message-ID: <20161124213516.107977.32281.6AAADFA6@psf.io>

https://hg.python.org/cpython/rev/6453ff3328b8
changeset:   105358:6453ff3328b8
parent:      105356:72b8a7423cac
parent:      105357:303cedfb9e7a
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Thu Nov 24 22:33:49 2016 +0100
summary:
  Merge 3.6

files:
  Misc/NEWS           |  4 ++++
  Objects/genobject.c |  9 +++++++++
  Python/ceval.c      |  1 +
  3 files changed, 14 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,10 @@
 Core and Builtins
 -----------------
 
+- Issue #28782: Fix a bug in the implementation ``yield from`` when checking
+  if the next instruction is YIELD_FROM. Regression introduced by WORDCODE
+  (issue #26647).
+
 - Issue #28774: Fix error position of the unicode error in ASCII and Latin1
   encoders when a string returned by the error handler contains multiple
   non-encodable characters (non-ASCII for the ASCII codec, characters out
diff --git a/Objects/genobject.c b/Objects/genobject.c
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -355,6 +355,14 @@
         PyObject *bytecode = f->f_code->co_code;
         unsigned char *code = (unsigned char *)PyBytes_AS_STRING(bytecode);
 
+        if (f->f_lasti < 0) {
+            /* Return immediately if the frame didn't start yet. YIELD_FROM
+               always come after LOAD_CONST: a code object should not start
+               with YIELD_FROM */
+            assert(code[0] != YIELD_FROM);
+            return NULL;
+        }
+
         if (code[f->f_lasti + sizeof(_Py_CODEUNIT)] != YIELD_FROM)
             return NULL;
         yf = f->f_stacktop[-1];
@@ -463,6 +471,7 @@
             assert(ret == yf);
             Py_DECREF(ret);
             /* Termination repetition of YIELD_FROM */
+            assert(gen->gi_frame->f_lasti >= 0);
             gen->gi_frame->f_lasti += sizeof(_Py_CODEUNIT);
             if (_PyGen_FetchStopIterationValue(&val) == 0) {
                 ret = gen_send_ex(gen, val, 0, 0);
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -2049,6 +2049,7 @@
             f->f_stacktop = stack_pointer;
             why = WHY_YIELD;
             /* and repeat... */
+            assert(f->f_lasti >= (int)sizeof(_Py_CODEUNIT));
             f->f_lasti -= sizeof(_Py_CODEUNIT);
             goto fast_yield;
         }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Thu Nov 24 17:35:42 2016
From: python-checkins at python.org (victor.stinner)
Date: Thu, 24 Nov 2016 22:35:42 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328792=3A_Remove_a?=
 =?utf-8?q?liases_from_=5Fbisect?=
Message-ID: <20161124223542.29050.987.FE09BDF5@psf.io>

https://hg.python.org/cpython/rev/45713818fd81
changeset:   105359:45713818fd81
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Thu Nov 24 23:31:59 2016 +0100
summary:
  Issue #28792: Remove aliases from _bisect

Remove aliases from the C module. Always implement bisect() and insort()
aliases in bisect.py

Remove also the "# backward compatibility" command, there is no plan to
deprecate nor remove these aliases. When keys are equal, it makes sense to use
bisect.bisect() and bisect.insort().

files:
  Lib/bisect.py           |  8 ++++----
  Modules/_bisectmodule.c |  7 -------
  2 files changed, 4 insertions(+), 11 deletions(-)


diff --git a/Lib/bisect.py b/Lib/bisect.py
--- a/Lib/bisect.py
+++ b/Lib/bisect.py
@@ -19,8 +19,6 @@
         else: lo = mid+1
     a.insert(lo, x)
 
-insort = insort_right   # backward compatibility
-
 def bisect_right(a, x, lo=0, hi=None):
     """Return the index where to insert item x in list a, assuming a is sorted.
 
@@ -42,8 +40,6 @@
         else: lo = mid+1
     return lo
 
-bisect = bisect_right   # backward compatibility
-
 def insort_left(a, x, lo=0, hi=None):
     """Insert item x in list a, and keep it sorted assuming a is sorted.
 
@@ -90,3 +86,7 @@
     from _bisect import *
 except ImportError:
     pass
+
+# Create aliases
+bisect = bisect_right
+insort = insort_right
diff --git a/Modules/_bisectmodule.c b/Modules/_bisectmodule.c
--- a/Modules/_bisectmodule.c
+++ b/Modules/_bisectmodule.c
@@ -216,18 +216,11 @@
 Optional args lo (default 0) and hi (default len(a)) bound the\n\
 slice of a to be searched.\n");
 
-PyDoc_STRVAR(bisect_doc, "Alias for bisect_right().\n");
-PyDoc_STRVAR(insort_doc, "Alias for insort_right().\n");
-
 static PyMethodDef bisect_methods[] = {
     {"bisect_right", (PyCFunction)bisect_right,
         METH_VARARGS|METH_KEYWORDS, bisect_right_doc},
-    {"bisect", (PyCFunction)bisect_right,
-        METH_VARARGS|METH_KEYWORDS, bisect_doc},
     {"insort_right", (PyCFunction)insort_right,
         METH_VARARGS|METH_KEYWORDS, insort_right_doc},
-    {"insort", (PyCFunction)insort_right,
-        METH_VARARGS|METH_KEYWORDS, insort_doc},
     {"bisect_left", (PyCFunction)bisect_left,
         METH_VARARGS|METH_KEYWORDS, bisect_left_doc},
     {"insort_left", (PyCFunction)insort_left,

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Fri Nov 25 04:07:09 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Fri, 25 Nov 2016 09:07:09 +0000
Subject: [Python-checkins] Daily reference leaks (45713818fd81): sum=6
Message-ID: <20161125090709.115973.47364.B1DA4562@psf.io>

results for 45713818fd81 on branch "default"
--------------------------------------------

test_functools leaked [0, 3, 1] memory blocks, sum=4
test_multiprocessing_fork leaked [2, 0, 0] memory blocks, sum=2


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog89w1yJ', '--timeout', '7200']

From python-checkins at python.org  Fri Nov 25 06:00:26 2016
From: python-checkins at python.org (victor.stinner)
Date: Fri, 25 Nov 2016 11:00:26 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_a_ResourceWarning_in_g?=
 =?utf-8?q?enerate=5Fopcode=5Fh=2Epy?=
Message-ID: <20161125110026.21389.17470.105323D4@psf.io>

https://hg.python.org/cpython/rev/46e2755b022c
changeset:   105360:46e2755b022c
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Fri Nov 25 11:59:52 2016 +0100
summary:
  Fix a ResourceWarning in generate_opcode_h.py

Use a context manager to close the Python file. Replace also open() with
tokenize.open() to handle coding cookie if any in Lib/opcode.py.

files:
  Tools/scripts/generate_opcode_h.py |  6 +++++-
  1 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Tools/scripts/generate_opcode_h.py b/Tools/scripts/generate_opcode_h.py
--- a/Tools/scripts/generate_opcode_h.py
+++ b/Tools/scripts/generate_opcode_h.py
@@ -32,10 +32,14 @@
 #endif /* !Py_OPCODE_H */
 """
 
+import tokenize
+
 
 def main(opcode_py, outfile='Include/opcode.h'):
     opcode = {}
-    exec(open(opcode_py).read(), opcode)
+    with tokenize.open(opcode_py) as fp:
+        code = fp.read()
+    exec(code, opcode)
     opmap = opcode['opmap']
     with open(outfile, 'w') as fobj:
         fobj.write(header)

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Fri Nov 25 08:07:59 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Fri, 25 Nov 2016 13:07:59 +0000
Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-11-25
Message-ID: <f6482f8f-576a-49e0-9931-d25c70124526@irsmsx102.ger.corp.intel.com>

No new revisions. Here are the previous results:

Results for project Python 2.7, build date 2016-11-25 03:45:38 +0000
commit:		6eefdf0fade6
previous commit:	4de956751cf1
revision date:	2016-11-21 09:37:18 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.17%         -1.09%          3.37%             8.32%
:-)             pybench      0.13%         -0.05%          5.71%             3.52%
:-(            regex_v8      0.57%          0.32%         -2.32%            11.28%
:-)               nbody      0.86%         -0.57%          7.89%             5.91%
:-)        json_dump_v2      0.30%          0.72%          3.79%             7.84%
:-|      normal_startup      1.00%         -0.84%         -0.31%             2.71%
:-)             ssbench      0.23%          0.11%          2.63%             1.30%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-11-25/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Fri Nov 25 08:09:26 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Fri, 25 Nov 2016 13:09:26 +0000
Subject: [Python-checkins] BAD Benchmark Results for Python Default
 2016-11-25
Message-ID: <0efaf501-82b2-4cee-8a1b-1e37b46d1357@irsmsx102.ger.corp.intel.com>

Results for project Python default, build date 2016-11-25 03:01:57 +0000
commit:		45713818fd81
previous commit:	831f73efe3a6
revision date:	2016-11-24 22:31:59 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.26%         -0.28%          6.66%            14.36%
:-)             pybench      0.19%         -0.14%          5.38%             4.93%
:-(            regex_v8      3.77%         -0.75%         -3.05%             3.01%
:-(               nbody      0.17%         -3.45%         -0.56%             7.10%
:-(        json_dump_v2      0.32%         -0.54%         -4.02%            10.97%
:-|      normal_startup      0.75%          0.47%          0.12%             6.63%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-default-2016-11-25/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Fri Nov 25 09:31:32 2016
From: python-checkins at python.org (berker.peksag)
Date: Fri, 25 Nov 2016 14:31:32 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4Nzkz?=
 =?utf-8?q?=3A_Fix_c/p_error_in_AsyncGenerator_documentation?=
Message-ID: <20161125143130.116992.58657.C35BA77D@psf.io>

https://hg.python.org/cpython/rev/078c037b1571
changeset:   105361:078c037b1571
branch:      3.6
parent:      105357:303cedfb9e7a
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Fri Nov 25 17:31:27 2016 +0300
summary:
  Issue #28793: Fix c/p error in AsyncGenerator documentation

Patch by Julien Palard.

files:
  Doc/library/collections.abc.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/collections.abc.rst b/Doc/library/collections.abc.rst
--- a/Doc/library/collections.abc.rst
+++ b/Doc/library/collections.abc.rst
@@ -223,7 +223,7 @@
 
    .. versionadded:: 3.5
 
-.. class:: Generator
+.. class:: AsyncGenerator
 
    ABC for asynchronous generator classes that implement the protocol
    defined in :pep:`525` and :pep:`492`.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 25 09:31:32 2016
From: python-checkins at python.org (berker.peksag)
Date: Fri, 25 Nov 2016 14:31:32 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328793=3A_Merge_from_3=2E6?=
Message-ID: <20161125143130.108353.46405.A0FA58F6@psf.io>

https://hg.python.org/cpython/rev/0bf7c2d99bf1
changeset:   105362:0bf7c2d99bf1
parent:      105360:46e2755b022c
parent:      105361:078c037b1571
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Fri Nov 25 17:31:57 2016 +0300
summary:
  Issue #28793: Merge from 3.6

files:
  Doc/library/collections.abc.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/collections.abc.rst b/Doc/library/collections.abc.rst
--- a/Doc/library/collections.abc.rst
+++ b/Doc/library/collections.abc.rst
@@ -223,7 +223,7 @@
 
    .. versionadded:: 3.5
 
-.. class:: Generator
+.. class:: AsyncGenerator
 
    ABC for asynchronous generator classes that implement the protocol
    defined in :pep:`525` and :pep:`492`.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 25 09:37:47 2016
From: python-checkins at python.org (berker.peksag)
Date: Fri, 25 Nov 2016 14:37:47 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328796=3A_Silence_?=
 =?utf-8?q?Sphinx_warnings?=
Message-ID: <20161125143747.108019.52457.A4A735CB@psf.io>

https://hg.python.org/cpython/rev/f16f0500b49b
changeset:   105363:f16f0500b49b
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Fri Nov 25 17:38:13 2016 +0300
summary:
  Issue #28796: Silence Sphinx warnings

WARNING: Inline emphasis start-string without end-string.

Patch by Julien Palard.

files:
  Misc/NEWS |  8 ++++----
  1 files changed, 4 insertions(+), 4 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -443,10 +443,10 @@
 
 - Issue #28761: The fields name and doc of structures PyMemberDef, PyGetSetDef,
   PyStructSequence_Field, PyStructSequence_Desc, and wrapperbase are now of
-  type "const char *" rather of "char *".
-
-- Issue #28748: Private variable _Py_PackageContext is now of type "const char *"
-  rather of "char *".
+  type ``const char *`` rather of ``char *``.
+
+- Issue #28748: Private variable _Py_PackageContext is now of type ``const char *``
+  rather of ``char *``.
 
 - Issue #19569: Compiler warnings are now emitted if use most of deprecated
   functions.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 25 11:47:41 2016
From: python-checkins at python.org (berker.peksag)
Date: Fri, 25 Nov 2016 16:47:41 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NzM4?=
 =?utf-8?q?=3A_Document_SIGBREAK_as_an_acceptable_value_on_Windows?=
Message-ID: <20161125164741.29321.11560.1C371F55@psf.io>

https://hg.python.org/cpython/rev/dddce0f539dd
changeset:   105364:dddce0f539dd
branch:      3.5
parent:      105353:5d83b6a9b568
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Fri Nov 25 19:46:57 2016 +0300
summary:
  Issue #28738: Document SIGBREAK as an acceptable value on Windows

Patch by Wojtek Ruszczewski.

files:
  Doc/library/signal.rst |  5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst
--- a/Doc/library/signal.rst
+++ b/Doc/library/signal.rst
@@ -350,8 +350,9 @@
    attribute descriptions in the :mod:`inspect` module).
 
    On Windows, :func:`signal` can only be called with :const:`SIGABRT`,
-   :const:`SIGFPE`, :const:`SIGILL`, :const:`SIGINT`, :const:`SIGSEGV`, or
-   :const:`SIGTERM`. A :exc:`ValueError` will be raised in any other case.
+   :const:`SIGFPE`, :const:`SIGILL`, :const:`SIGINT`, :const:`SIGSEGV`,
+   :const:`SIGTERM`, or :const:`SIGBREAK`.
+   A :exc:`ValueError` will be raised in any other case.
    Note that not all systems define the same set of signal names; an
    :exc:`AttributeError` will be raised if a signal name is not defined as
    ``SIG*`` module level constant.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 25 11:47:42 2016
From: python-checkins at python.org (berker.peksag)
Date: Fri, 25 Nov 2016 16:47:42 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328738=3A_Merge_from_3=2E6?=
Message-ID: <20161125164741.116185.67185.38C5410C@psf.io>

https://hg.python.org/cpython/rev/04eba322be92
changeset:   105366:04eba322be92
parent:      105363:f16f0500b49b
parent:      105365:43856eb83dc1
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Fri Nov 25 19:48:05 2016 +0300
summary:
  Issue #28738: Merge from 3.6

files:
  Doc/library/signal.rst |  5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst
--- a/Doc/library/signal.rst
+++ b/Doc/library/signal.rst
@@ -350,8 +350,9 @@
    attribute descriptions in the :mod:`inspect` module).
 
    On Windows, :func:`signal` can only be called with :const:`SIGABRT`,
-   :const:`SIGFPE`, :const:`SIGILL`, :const:`SIGINT`, :const:`SIGSEGV`, or
-   :const:`SIGTERM`. A :exc:`ValueError` will be raised in any other case.
+   :const:`SIGFPE`, :const:`SIGILL`, :const:`SIGINT`, :const:`SIGSEGV`,
+   :const:`SIGTERM`, or :const:`SIGBREAK`.
+   A :exc:`ValueError` will be raised in any other case.
    Note that not all systems define the same set of signal names; an
    :exc:`AttributeError` will be raised if a signal name is not defined as
    ``SIG*`` module level constant.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 25 11:47:42 2016
From: python-checkins at python.org (berker.peksag)
Date: Fri, 25 Nov 2016 16:47:42 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328738=3A_Merge_from_3=2E6?=
Message-ID: <20161125164741.90403.53777.22A43352@psf.io>

https://hg.python.org/cpython/rev/43856eb83dc1
changeset:   105365:43856eb83dc1
branch:      3.6
parent:      105361:078c037b1571
parent:      105364:dddce0f539dd
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Fri Nov 25 19:47:43 2016 +0300
summary:
  Issue #28738: Merge from 3.6

files:
  Doc/library/signal.rst |  5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst
--- a/Doc/library/signal.rst
+++ b/Doc/library/signal.rst
@@ -350,8 +350,9 @@
    attribute descriptions in the :mod:`inspect` module).
 
    On Windows, :func:`signal` can only be called with :const:`SIGABRT`,
-   :const:`SIGFPE`, :const:`SIGILL`, :const:`SIGINT`, :const:`SIGSEGV`, or
-   :const:`SIGTERM`. A :exc:`ValueError` will be raised in any other case.
+   :const:`SIGFPE`, :const:`SIGILL`, :const:`SIGINT`, :const:`SIGSEGV`,
+   :const:`SIGTERM`, or :const:`SIGBREAK`.
+   A :exc:`ValueError` will be raised in any other case.
    Note that not all systems define the same set of signal names; an
    :exc:`AttributeError` will be raised if a signal name is not defined as
    ``SIG*`` module level constant.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 25 12:10:26 2016
From: python-checkins at python.org (berker.peksag)
Date: Fri, 25 Nov 2016 17:10:26 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_from_3=2E6?=
Message-ID: <20161125171026.32529.32673.2405DC85@psf.io>

https://hg.python.org/cpython/rev/07dfab805cf3
changeset:   105369:07dfab805cf3
parent:      105366:04eba322be92
parent:      105368:17fe6e23a8bc
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Fri Nov 25 20:10:51 2016 +0300
summary:
  Merge from 3.6

files:
  Doc/library/typing.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst
--- a/Doc/library/typing.rst
+++ b/Doc/library/typing.rst
@@ -754,7 +754,7 @@
    runtime we intentionally don't check anything (we want this
    to be as fast as possible).
 
-.. function:: get_type_hints(obj[, globals[, locals])
+.. function:: get_type_hints(obj[, globals[, locals]])
 
    Return a dictionary containing type hints for a function, method, module
    or class object.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 25 12:10:27 2016
From: python-checkins at python.org (berker.peksag)
Date: Fri, 25 Nov 2016 17:10:27 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Add_missing_sq?=
 =?utf-8?q?uare_bracket_in_typing=2Eget=5Ftype=5Fhints=28=29?=
Message-ID: <20161125171026.108181.48171.861F6DEC@psf.io>

https://hg.python.org/cpython/rev/18394c3c9640
changeset:   105367:18394c3c9640
branch:      3.5
parent:      105364:dddce0f539dd
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Fri Nov 25 20:10:07 2016 +0300
summary:
  Add missing square bracket in typing.get_type_hints()

files:
  Doc/library/typing.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst
--- a/Doc/library/typing.rst
+++ b/Doc/library/typing.rst
@@ -733,7 +733,7 @@
    runtime we intentionally don't check anything (we want this
    to be as fast as possible).
 
-.. function:: get_type_hints(obj[, globals[, locals])
+.. function:: get_type_hints(obj[, globals[, locals]])
 
    Return a dictionary containing type hints for a function, method, module
    or class object.

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Fri Nov 25 12:10:27 2016
From: python-checkins at python.org (berker.peksag)
Date: Fri, 25 Nov 2016 17:10:27 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_from_3=2E5?=
Message-ID: <20161125171026.116064.33036.9095FF0D@psf.io>

https://hg.python.org/cpython/rev/17fe6e23a8bc
changeset:   105368:17fe6e23a8bc
branch:      3.6
parent:      105365:43856eb83dc1
parent:      105367:18394c3c9640
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Fri Nov 25 20:10:30 2016 +0300
summary:
  Merge from 3.5

files:
  Doc/library/typing.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst
--- a/Doc/library/typing.rst
+++ b/Doc/library/typing.rst
@@ -754,7 +754,7 @@
    runtime we intentionally don't check anything (we want this
    to be as fast as possible).
 
-.. function:: get_type_hints(obj[, globals[, locals])
+.. function:: get_type_hints(obj[, globals[, locals]])
 
    Return a dictionary containing type hints for a function, method, module
    or class object.

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Sat Nov 26 04:05:46 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sat, 26 Nov 2016 09:05:46 +0000
Subject: [Python-checkins] Daily reference leaks (07dfab805cf3): sum=5
Message-ID: <20161126090545.22917.40576.04CA3131@psf.io>

results for 07dfab805cf3 on branch "default"
--------------------------------------------

test_collections leaked [7, -7, 1] memory blocks, sum=1
test_functools leaked [0, 3, 1] memory blocks, sum=4


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogBnpiDg', '--timeout', '7200']

From python-checkins at python.org  Sat Nov 26 06:50:43 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sat, 26 Nov 2016 11:50:43 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328763=3A_Use_double_hyphens_=28rendered_as_en-dashes?=
 =?utf-8?q?=29_in_numerical_ranges?=
Message-ID: <20161126115043.82101.95271.798C2FF8@psf.io>

https://hg.python.org/cpython/rev/3434d84efdd4
changeset:   105372:3434d84efdd4
branch:      3.6
parent:      105368:17fe6e23a8bc
parent:      105370:77307437ae15
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sat Nov 26 13:49:59 2016 +0200
summary:
  Issue #28763: Use double hyphens (rendered as en-dashes) in numerical ranges
in the documentation.

files:
  Doc/faq/design.rst           |  4 ++--
  Doc/faq/general.rst          |  2 +-
  Doc/howto/urllib2.rst        |  4 ++--
  Doc/library/calendar.rst     |  2 +-
  Doc/library/cmath.rst        |  2 +-
  Doc/library/codecs.rst       |  6 +++---
  Doc/library/curses.ascii.rst |  2 +-
  Doc/library/curses.rst       |  2 +-
  Doc/library/functions.rst    |  2 +-
  Doc/library/ipaddress.rst    |  2 +-
  Doc/library/json.rst         |  2 +-
  Doc/library/random.rst       |  2 +-
  Doc/library/statistics.rst   |  6 +++---
  Doc/whatsnew/2.1.rst         |  2 +-
  Doc/whatsnew/2.7.rst         |  4 ++--
  Doc/whatsnew/3.5.rst         |  4 ++--
  16 files changed, 24 insertions(+), 24 deletions(-)


diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst
--- a/Doc/faq/design.rst
+++ b/Doc/faq/design.rst
@@ -31,7 +31,7 @@
 Many coding styles place begin/end brackets on a line by themselves.  This makes
 programs considerably longer and wastes valuable screen space, making it harder
 to get a good overview of a program.  Ideally, a function should fit on one
-screen (say, 20-30 lines).  20 lines of Python can do a lot more work than 20
+screen (say, 20--30 lines).  20 lines of Python can do a lot more work than 20
 lines of C.  This is not solely due to the lack of begin/end brackets -- the
 lack of declarations and the high-level data types are also responsible -- but
 the indentation-based syntax certainly helps.
@@ -77,7 +77,7 @@
 
     1.1999999999999999555910790149937383830547332763671875 (decimal)
 
-The typical precision of 53 bits provides Python floats with 15-16
+The typical precision of 53 bits provides Python floats with 15--16
 decimal digits of accuracy.
 
 For a fuller explanation, please see the :ref:`floating point arithmetic
diff --git a/Doc/faq/general.rst b/Doc/faq/general.rst
--- a/Doc/faq/general.rst
+++ b/Doc/faq/general.rst
@@ -252,7 +252,7 @@
 
     Guido van Rossum and Jelke de Boer, "Interactively Testing Remote Servers
     Using the Python Programming Language", CWI Quarterly, Volume 4, Issue 4
-    (December 1991), Amsterdam, pp 283-303.
+    (December 1991), Amsterdam, pp 283--303.
 
 
 Are there any books on Python?
diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst
--- a/Doc/howto/urllib2.rst
+++ b/Doc/howto/urllib2.rst
@@ -240,8 +240,8 @@
 ~~~~~~~~~~~
 
 Because the default handlers handle redirects (codes in the 300 range), and
-codes in the 100-299 range indicate success, you will usually only see error
-codes in the 400-599 range.
+codes in the 100--299 range indicate success, you will usually only see error
+codes in the 400--599 range.
 
 :attr:`http.server.BaseHTTPRequestHandler.responses` is a useful dictionary of
 response codes in that shows all the response codes used by RFC 2616. The
diff --git a/Doc/library/calendar.rst b/Doc/library/calendar.rst
--- a/Doc/library/calendar.rst
+++ b/Doc/library/calendar.rst
@@ -47,7 +47,7 @@
 
    .. method:: itermonthdates(year, month)
 
-      Return an iterator for the month *month* (1-12) in the year *year*. This
+      Return an iterator for the month *month* (1--12) in the year *year*. This
       iterator will return all days (as :class:`datetime.date` objects) for the
       month and all days before the start of the month or after the end of the
       month that are required to get a complete week.
diff --git a/Doc/library/cmath.rst b/Doc/library/cmath.rst
--- a/Doc/library/cmath.rst
+++ b/Doc/library/cmath.rst
@@ -309,4 +309,4 @@
 
    Kahan, W:  Branch cuts for complex elementary functions; or, Much ado about
    nothing's sign bit.  In Iserles, A., and Powell, M. (eds.), The state of the art
-   in numerical analysis. Clarendon Press (1987) pp165-211.
+   in numerical analysis. Clarendon Press (1987) pp165--211.
diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst
--- a/Doc/library/codecs.rst
+++ b/Doc/library/codecs.rst
@@ -858,7 +858,7 @@
 ---------------------
 
 Strings are stored internally as sequences of code points in
-range ``0x0``-``0x10FFFF``.  (See :pep:`393` for
+range ``0x0``--``0x10FFFF``.  (See :pep:`393` for
 more details about the implementation.)
 Once a string object is used outside of CPU and memory, endianness
 and how these arrays are stored as bytes become an issue.  As with other
@@ -868,7 +868,7 @@
 collectivity referred to as :term:`text encodings <text encoding>`.
 
 The simplest text encoding (called ``'latin-1'`` or ``'iso-8859-1'``) maps
-the code points 0-255 to the bytes ``0x0``-``0xff``, which means that a string
+the code points 0--255 to the bytes ``0x0``--``0xff``, which means that a string
 object that contains code points above ``U+00FF`` can't be encoded with this
 codec. Doing so will raise a :exc:`UnicodeEncodeError` that looks
 like the following (although the details of the error message may differ):
@@ -877,7 +877,7 @@
 
 There's another group of encodings (the so called charmap encodings) that choose
 a different subset of all Unicode code points and how these code points are
-mapped to the bytes ``0x0``-``0xff``. To see how this is done simply open
+mapped to the bytes ``0x0``--``0xff``. To see how this is done simply open
 e.g. :file:`encodings/cp1252.py` (which is an encoding that is used primarily on
 Windows). There's a string constant with 256 characters that shows you which
 character is mapped to which byte value.
diff --git a/Doc/library/curses.ascii.rst b/Doc/library/curses.ascii.rst
--- a/Doc/library/curses.ascii.rst
+++ b/Doc/library/curses.ascii.rst
@@ -213,7 +213,7 @@
 
    Return a string representation of the ASCII character *c*.  If *c* is printable,
    this string is the character itself.  If the character is a control character
-   (0x00-0x1f) the string consists of a caret (``'^'``) followed by the
+   (0x00--0x1f) the string consists of a caret (``'^'``) followed by the
    corresponding uppercase letter. If the character is an ASCII delete (0x7f) the
    string is ``'^?'``.  If the character has its meta bit (0x80) set, the meta bit
    is stripped, the preceding rules applied, and ``'!'`` prepended to the result.
diff --git a/Doc/library/curses.rst b/Doc/library/curses.rst
--- a/Doc/library/curses.rst
+++ b/Doc/library/curses.rst
@@ -316,7 +316,7 @@
    Return the name of the key numbered *k*.  The name of a key generating printable
    ASCII character is the key's character.  The name of a control-key combination
    is a two-character string consisting of a caret followed by the corresponding
-   printable ASCII character.  The name of an alt-key combination (128-255) is a
+   printable ASCII character.  The name of an alt-key combination (128--255) is a
    string consisting of the prefix 'M-' followed by the name of the corresponding
    ASCII character.
 
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -692,7 +692,7 @@
    preceded by ``+`` or ``-`` (with no space in between) and surrounded by
    whitespace.  A base-n literal consists of the digits 0 to n-1, with ``a``
    to ``z`` (or ``A`` to ``Z``) having
-   values 10 to 35.  The default *base* is 10. The allowed values are 0 and 2-36.
+   values 10 to 35.  The default *base* is 10. The allowed values are 0 and 2--36.
    Base-2, -8, and -16 literals can be optionally prefixed with ``0b``/``0B``,
    ``0o``/``0O``, or ``0x``/``0X``, as with integer literals in code.  Base 0
    means to interpret exactly as a code literal, so that the actual base is 2,
diff --git a/Doc/library/ipaddress.rst b/Doc/library/ipaddress.rst
--- a/Doc/library/ipaddress.rst
+++ b/Doc/library/ipaddress.rst
@@ -99,7 +99,7 @@
    The following constitutes a valid IPv4 address:
 
    1. A string in decimal-dot notation, consisting of four decimal integers in
-      the inclusive range 0-255, separated by dots (e.g. ``192.168.0.1``). Each
+      the inclusive range 0--255, separated by dots (e.g. ``192.168.0.1``). Each
       integer represents an octet (byte) in the address. Leading zeroes are
       tolerated only for values less than 8 (as there is no ambiguity
       between the decimal and octal interpretations of such strings).
diff --git a/Doc/library/json.rst b/Doc/library/json.rst
--- a/Doc/library/json.rst
+++ b/Doc/library/json.rst
@@ -345,7 +345,7 @@
 
    If *strict* is false (``True`` is the default), then control characters
    will be allowed inside strings.  Control characters in this context are
-   those with character codes in the 0-31 range, including ``'\t'`` (tab),
+   those with character codes in the 0--31 range, including ``'\t'`` (tab),
    ``'\n'``, ``'\r'`` and ``'\0'``.
 
    If the data being deserialized is not a valid JSON document, a
diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -53,7 +53,7 @@
 
    M. Matsumoto and T. Nishimura, "Mersenne Twister: A 623-dimensionally
    equidistributed uniform pseudorandom number generator", ACM Transactions on
-   Modeling and Computer Simulation Vol. 8, No. 1, January pp.3-30 1998.
+   Modeling and Computer Simulation Vol. 8, No. 1, January pp.3--30 1998.
 
 
    `Complementary-Multiply-with-Carry recipe
diff --git a/Doc/library/statistics.rst b/Doc/library/statistics.rst
--- a/Doc/library/statistics.rst
+++ b/Doc/library/statistics.rst
@@ -224,9 +224,9 @@
       52.5
 
    In the following example, the data are rounded, so that each value represents
-   the midpoint of data classes, e.g. 1 is the midpoint of the class 0.5-1.5, 2
-   is the midpoint of 1.5-2.5, 3 is the midpoint of 2.5-3.5, etc.  With the data
-   given, the middle value falls somewhere in the class 3.5-4.5, and
+   the midpoint of data classes, e.g. 1 is the midpoint of the class 0.5--1.5, 2
+   is the midpoint of 1.5--2.5, 3 is the midpoint of 2.5--3.5, etc.  With the data
+   given, the middle value falls somewhere in the class 3.5--4.5, and
    interpolation is used to estimate it:
 
    .. doctest::
diff --git a/Doc/whatsnew/2.1.rst b/Doc/whatsnew/2.1.rst
--- a/Doc/whatsnew/2.1.rst
+++ b/Doc/whatsnew/2.1.rst
@@ -731,7 +731,7 @@
          ...
 
   For a fuller discussion of the line I/O changes, see the python-dev summary for
-  January 1-15, 2001 at https://mail.python.org/pipermail/python-dev/2001-January/.
+  January 1--15, 2001 at https://mail.python.org/pipermail/python-dev/2001-January/.
 
 * A new method, :meth:`popitem`, was added to dictionaries to enable
   destructively iterating through the contents of a dictionary; this can be faster
diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst
--- a/Doc/whatsnew/2.7.rst
+++ b/Doc/whatsnew/2.7.rst
@@ -104,7 +104,7 @@
   when compared to earlier 2.x versions. Python 2.7 is currently expected to
   remain supported by the core development team (receiving security updates
   and other bug fixes) until at least 2020 (10 years after its initial
-  release, compared to the more typical support period of 18-24 months).
+  release, compared to the more typical support period of 18--24 months).
 
 * As the Python 2.7 standard library ages, making effective use of the
   Python Package Index (either directly or via a redistributor) becomes
@@ -989,7 +989,7 @@
   Gregory Smith; :issue:`1087418`).
 
 * The implementation of ``%`` checks for the left-side operand being
-  a Python string and special-cases it; this results in a 1-3%
+  a Python string and special-cases it; this results in a 1--3%
   performance increase for applications that frequently use ``%``
   with strings, such as templating libraries.
   (Implemented by Collin Winter; :issue:`5176`.)
diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst
--- a/Doc/whatsnew/3.5.rst
+++ b/Doc/whatsnew/3.5.rst
@@ -2130,8 +2130,8 @@
 (Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in
 :issue:`22003`.)
 
-The :func:`marshal.dumps` function is now faster: 65-85% with versions 3
-and 4, 20-25% with versions 0 to 2 on typical data, and up to 5 times in
+The :func:`marshal.dumps` function is now faster: 65--85% with versions 3
+and 4, 20--25% with versions 0 to 2 on typical data, and up to 5 times in
 best cases.
 (Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 26 06:50:43 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sat, 26 Nov 2016 11:50:43 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4NzYz?=
 =?utf-8?q?=3A_Use_double_hyphens_=28rendered_as_en-dashes=29_in_numerical?=
 =?utf-8?q?_ranges?=
Message-ID: <20161126115043.29496.9529.9EE3FF5D@psf.io>

https://hg.python.org/cpython/rev/59bd48afa1bc
changeset:   105371:59bd48afa1bc
branch:      2.7
parent:      105267:6eefdf0fade6
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sat Nov 26 13:43:39 2016 +0200
summary:
  Issue #28763: Use double hyphens (rendered as en-dashes) in numerical ranges
in the documentation.

files:
  Doc/faq/design.rst           |   2 +-
  Doc/faq/general.rst          |   2 +-
  Doc/howto/unicode.rst        |  10 +++++-----
  Doc/howto/urllib2.rst        |   4 ++--
  Doc/library/calendar.rst     |   2 +-
  Doc/library/cmath.rst        |   2 +-
  Doc/library/codecs.rst       |   6 +++---
  Doc/library/curses.ascii.rst |   2 +-
  Doc/library/curses.rst       |   2 +-
  Doc/library/functions.rst    |   2 +-
  Doc/library/json.rst         |   2 +-
  Doc/library/random.rst       |   2 +-
  Doc/library/sgmllib.rst      |   2 +-
  Doc/library/sys.rst          |   2 +-
  Doc/reference/datamodel.rst  |   2 +-
  Doc/whatsnew/2.1.rst         |   2 +-
  Doc/whatsnew/2.7.rst         |   4 ++--
  17 files changed, 25 insertions(+), 25 deletions(-)


diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst
--- a/Doc/faq/design.rst
+++ b/Doc/faq/design.rst
@@ -31,7 +31,7 @@
 Many coding styles place begin/end brackets on a line by themselves.  This makes
 programs considerably longer and wastes valuable screen space, making it harder
 to get a good overview of a program.  Ideally, a function should fit on one
-screen (say, 20-30 lines).  20 lines of Python can do a lot more work than 20
+screen (say, 20--30 lines).  20 lines of Python can do a lot more work than 20
 lines of C.  This is not solely due to the lack of begin/end brackets -- the
 lack of declarations and the high-level data types are also responsible -- but
 the indentation-based syntax certainly helps.
diff --git a/Doc/faq/general.rst b/Doc/faq/general.rst
--- a/Doc/faq/general.rst
+++ b/Doc/faq/general.rst
@@ -252,7 +252,7 @@
 
     Guido van Rossum and Jelke de Boer, "Interactively Testing Remote Servers
     Using the Python Programming Language", CWI Quarterly, Volume 4, Issue 4
-    (December 1991), Amsterdam, pp 283-303.
+    (December 1991), Amsterdam, pp 283--303.
 
 
 Are there any books on Python?
diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst
--- a/Doc/howto/unicode.rst
+++ b/Doc/howto/unicode.rst
@@ -42,14 +42,14 @@
 hold values ranging from 0 to 255.  ASCII codes only went up to 127, so some
 machines assigned values between 128 and 255 to accented characters.  Different
 machines had different codes, however, which led to problems exchanging files.
-Eventually various commonly used sets of values for the 128-255 range emerged.
+Eventually various commonly used sets of values for the 128--255 range emerged.
 Some were true standards, defined by the International Standards Organization,
 and some were **de facto** conventions that were invented by one company or
 another and managed to catch on.
 
 255 characters aren't very many.  For example, you can't fit both the accented
 characters used in Western Europe and the Cyrillic alphabet used for Russian
-into the 128-255 range because there are more than 128 such characters.
+into the 128--255 range because there are more than 128 such characters.
 
 You could write files using different codes (all your Russian files in a coding
 system called KOI8, all your French files in a different coding system called
@@ -62,7 +62,7 @@
 to represent many different characters from many different alphabets; an initial
 goal was to have Unicode contain the alphabets for every single human language.
 It turns out that even 16 bits isn't enough to meet that goal, and the modern
-Unicode specification uses a wider range of codes, 0-1,114,111 (0x10ffff in
+Unicode specification uses a wider range of codes, 0--1,114,111 (0x10ffff in
 base-16).
 
 There's a related ISO standard, ISO 10646.  Unicode and ISO 10646 were
@@ -116,7 +116,7 @@
 
 To summarize the previous section: a Unicode string is a sequence of code
 points, which are numbers from 0 to 0x10ffff.  This sequence needs to be
-represented as a set of bytes (meaning, values from 0-255) in memory.  The rules
+represented as a set of bytes (meaning, values from 0--255) in memory.  The rules
 for translating a Unicode string into a sequence of bytes are called an
 **encoding**.
 
@@ -163,7 +163,7 @@
    case.)
 
 Latin-1, also known as ISO-8859-1, is a similar encoding.  Unicode code points
-0-255 are identical to the Latin-1 values, so converting to this encoding simply
+0--255 are identical to the Latin-1 values, so converting to this encoding simply
 requires converting code points to byte values; if a code point larger than 255
 is encountered, the string can't be encoded into Latin-1.
 
diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst
--- a/Doc/howto/urllib2.rst
+++ b/Doc/howto/urllib2.rst
@@ -229,8 +229,8 @@
 ~~~~~~~~~~~
 
 Because the default handlers handle redirects (codes in the 300 range), and
-codes in the 100-299 range indicate success, you will usually only see error
-codes in the 400-599 range.
+codes in the 100--299 range indicate success, you will usually only see error
+codes in the 400--599 range.
 
 ``BaseHTTPServer.BaseHTTPRequestHandler.responses`` is a useful dictionary of
 response codes in that shows all the response codes used by RFC 2616. The
diff --git a/Doc/library/calendar.rst b/Doc/library/calendar.rst
--- a/Doc/library/calendar.rst
+++ b/Doc/library/calendar.rst
@@ -48,7 +48,7 @@
 
    .. method:: itermonthdates(year, month)
 
-      Return an iterator for the month *month* (1-12) in the year *year*. This
+      Return an iterator for the month *month* (1--12) in the year *year*. This
       iterator will return all days (as :class:`datetime.date` objects) for the
       month and all days before the start of the month or after the end of the
       month that are required to get a complete week.
diff --git a/Doc/library/cmath.rst b/Doc/library/cmath.rst
--- a/Doc/library/cmath.rst
+++ b/Doc/library/cmath.rst
@@ -255,6 +255,6 @@
 
    Kahan, W:  Branch cuts for complex elementary functions; or, Much ado about
    nothing's sign bit.  In Iserles, A., and Powell, M. (eds.), The state of the art
-   in numerical analysis. Clarendon Press (1987) pp165-211.
+   in numerical analysis. Clarendon Press (1987) pp165--211.
 
 
diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst
--- a/Doc/library/codecs.rst
+++ b/Doc/library/codecs.rst
@@ -796,8 +796,8 @@
 unicode object into a sequence of bytes is called encoding and recreating the
 unicode object from the sequence of bytes is known as decoding.  There are many
 different methods for how this transformation can be done (these methods are
-also called encodings). The simplest method is to map the code points 0-255 to
-the bytes ``0x0``-``0xff``. This means that a unicode object that contains
+also called encodings). The simplest method is to map the code points 0--255 to
+the bytes ``0x0``--``0xff``. This means that a unicode object that contains
 code points above ``U+00FF`` can't be encoded with this method (which is called
 ``'latin-1'`` or ``'iso-8859-1'``). :func:`unicode.encode` will raise a
 :exc:`UnicodeEncodeError` that looks like this: ``UnicodeEncodeError: 'latin-1'
@@ -806,7 +806,7 @@
 
 There's another group of encodings (the so called charmap encodings) that choose
 a different subset of all unicode code points and how these code points are
-mapped to the bytes ``0x0``-``0xff``. To see how this is done simply open
+mapped to the bytes ``0x0``--``0xff``. To see how this is done simply open
 e.g. :file:`encodings/cp1252.py` (which is an encoding that is used primarily on
 Windows). There's a string constant with 256 characters that shows you which
 character is mapped to which byte value.
diff --git a/Doc/library/curses.ascii.rst b/Doc/library/curses.ascii.rst
--- a/Doc/library/curses.ascii.rst
+++ b/Doc/library/curses.ascii.rst
@@ -214,7 +214,7 @@
 
    Return a string representation of the ASCII character *c*.  If *c* is printable,
    this string is the character itself.  If the character is a control character
-   (0x00-0x1f) the string consists of a caret (``'^'``) followed by the
+   (0x00--0x1f) the string consists of a caret (``'^'``) followed by the
    corresponding uppercase letter. If the character is an ASCII delete (0x7f) the
    string is ``'^?'``.  If the character has its meta bit (0x80) set, the meta bit
    is stripped, the preceding rules applied, and ``'!'`` prepended to the result.
diff --git a/Doc/library/curses.rst b/Doc/library/curses.rst
--- a/Doc/library/curses.rst
+++ b/Doc/library/curses.rst
@@ -316,7 +316,7 @@
    Return the name of the key numbered *k*.  The name of a key generating printable
    ASCII character is the key's character.  The name of a control-key combination
    is a two-character string consisting of a caret followed by the corresponding
-   printable ASCII character.  The name of an alt-key combination (128-255) is a
+   printable ASCII character.  The name of an alt-key combination (128--255) is a
    string consisting of the prefix 'M-' followed by the name of the corresponding
    ASCII character.
 
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -670,7 +670,7 @@
    preceded by ``+`` or ``-`` (with no space in between) and surrounded by
    whitespace.  A base-n literal consists of the digits 0 to n-1, with ``a``
    to ``z`` (or ``A`` to ``Z``) having
-   values 10 to 35.  The default *base* is 10. The allowed values are 0 and 2-36.
+   values 10 to 35.  The default *base* is 10. The allowed values are 0 and 2--36.
    Base-2, -8, and -16 literals can be optionally prefixed with ``0b``/``0B``,
    ``0o``/``0O``/``0``, or ``0x``/``0X``, as with integer literals in code.
    Base 0 means to interpret the string exactly as an integer literal, so that
diff --git a/Doc/library/json.rst b/Doc/library/json.rst
--- a/Doc/library/json.rst
+++ b/Doc/library/json.rst
@@ -350,7 +350,7 @@
 
    If *strict* is false (``True`` is the default), then control characters
    will be allowed inside strings.  Control characters in this context are
-   those with character codes in the 0-31 range, including ``'\t'`` (tab),
+   those with character codes in the 0--31 range, including ``'\t'`` (tab),
    ``'\n'``, ``'\r'`` and ``'\0'``.
 
    If the data being deserialized is not a valid JSON document, a
diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -343,7 +343,7 @@
 
    M. Matsumoto and T. Nishimura, "Mersenne Twister: A 623-dimensionally
    equidistributed uniform pseudorandom number generator", ACM Transactions on
-   Modeling and Computer Simulation Vol. 8, No. 1, January pp.3-30 1998.
+   Modeling and Computer Simulation Vol. 8, No. 1, January pp.3--30 1998.
 
    Wichmann, B. A. & Hill, I. D., "Algorithm AS 183: An efficient and portable
    pseudo-random number generator", Applied Statistics 31 (1982) 188-190.
diff --git a/Doc/library/sgmllib.rst b/Doc/library/sgmllib.rst
--- a/Doc/library/sgmllib.rst
+++ b/Doc/library/sgmllib.rst
@@ -143,7 +143,7 @@
 
    Convert a character reference to a string, or ``None``.  *ref* is the reference
    passed in as a string.  In the base implementation, *ref* must be a decimal
-   number in the range 0-255.  It converts the code point found using the
+   number in the range 0--255.  It converts the code point found using the
    :meth:`convert_codepoint` method. If *ref* is invalid or out of range, this
    method returns ``None``.  This method is called by the default
    :meth:`handle_charref` implementation and by the attribute value parser.
diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst
--- a/Doc/library/sys.rst
+++ b/Doc/library/sys.rst
@@ -231,7 +231,7 @@
    (defaulting to zero), or another type of object.  If it is an integer, zero
    is considered "successful termination" and any nonzero value is considered
    "abnormal termination" by shells and the like.  Most systems require it to be
-   in the range 0-127, and produce undefined results otherwise.  Some systems
+   in the range 0--127, and produce undefined results otherwise.  Some systems
    have a convention for assigning specific meanings to specific exit codes, but
    these are generally underdeveloped; Unix programs generally use 2 for command
    line syntax errors and 1 for all other kind of errors.  If another type of
diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst
--- a/Doc/reference/datamodel.rst
+++ b/Doc/reference/datamodel.rst
@@ -301,7 +301,7 @@
          character is represented by a string of one item. Characters represent (at
          least) 8-bit bytes.  The built-in functions :func:`chr` and :func:`ord` convert
          between characters and nonnegative integers representing the byte values.  Bytes
-         with the values 0-127 usually represent the corresponding ASCII values, but the
+         with the values 0--127 usually represent the corresponding ASCII values, but the
          interpretation of values is up to the program.  The string data type is also
          used to represent arrays of bytes, e.g., to hold data read from a file.
 
diff --git a/Doc/whatsnew/2.1.rst b/Doc/whatsnew/2.1.rst
--- a/Doc/whatsnew/2.1.rst
+++ b/Doc/whatsnew/2.1.rst
@@ -731,7 +731,7 @@
          ...
 
   For a fuller discussion of the line I/O changes, see the python-dev summary for
-  January 1-15, 2001 at https://mail.python.org/pipermail/python-dev/2001-January/.
+  January 1--15, 2001 at https://mail.python.org/pipermail/python-dev/2001-January/.
 
 * A new method, :meth:`popitem`, was added to dictionaries to enable
   destructively iterating through the contents of a dictionary; this can be faster
diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst
--- a/Doc/whatsnew/2.7.rst
+++ b/Doc/whatsnew/2.7.rst
@@ -104,7 +104,7 @@
   when compared to earlier 2.x versions. Python 2.7 is currently expected to
   remain supported by the core development team (receiving security updates
   and other bug fixes) until at least 2020 (10 years after its initial
-  release, compared to the more typical support period of 18-24 months).
+  release, compared to the more typical support period of 18--24 months).
 
 * As the Python 2.7 standard library ages, making effective use of the
   Python Package Index (either directly or via a redistributor) becomes
@@ -974,7 +974,7 @@
   Gregory Smith; :issue:`1087418`).
 
 * The implementation of ``%`` checks for the left-side operand being
-  a Python string and special-cases it; this results in a 1-3%
+  a Python string and special-cases it; this results in a 1--3%
   performance increase for applications that frequently use ``%``
   with strings, such as templating libraries.
   (Implemented by Collin Winter; :issue:`5176`.)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 26 06:50:43 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sat, 26 Nov 2016 11:50:43 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4NzYz?=
 =?utf-8?q?=3A_Use_double_hyphens_=28rendered_as_en-dashes=29_in_numerical?=
 =?utf-8?q?_ranges?=
Message-ID: <20161126115043.30108.23973.564648DD@psf.io>

https://hg.python.org/cpython/rev/77307437ae15
changeset:   105370:77307437ae15
branch:      3.5
parent:      105367:18394c3c9640
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sat Nov 26 13:43:28 2016 +0200
summary:
  Issue #28763: Use double hyphens (rendered as en-dashes) in numerical ranges
in the documentation.

files:
  Doc/faq/design.rst           |  4 ++--
  Doc/faq/general.rst          |  2 +-
  Doc/howto/urllib2.rst        |  4 ++--
  Doc/library/calendar.rst     |  2 +-
  Doc/library/cmath.rst        |  2 +-
  Doc/library/codecs.rst       |  6 +++---
  Doc/library/curses.ascii.rst |  2 +-
  Doc/library/curses.rst       |  2 +-
  Doc/library/functions.rst    |  2 +-
  Doc/library/ipaddress.rst    |  2 +-
  Doc/library/json.rst         |  2 +-
  Doc/library/random.rst       |  2 +-
  Doc/library/statistics.rst   |  6 +++---
  Doc/library/sys.rst          |  2 +-
  Doc/whatsnew/2.1.rst         |  2 +-
  Doc/whatsnew/2.7.rst         |  4 ++--
  Doc/whatsnew/3.5.rst         |  4 ++--
  17 files changed, 25 insertions(+), 25 deletions(-)


diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst
--- a/Doc/faq/design.rst
+++ b/Doc/faq/design.rst
@@ -31,7 +31,7 @@
 Many coding styles place begin/end brackets on a line by themselves.  This makes
 programs considerably longer and wastes valuable screen space, making it harder
 to get a good overview of a program.  Ideally, a function should fit on one
-screen (say, 20-30 lines).  20 lines of Python can do a lot more work than 20
+screen (say, 20--30 lines).  20 lines of Python can do a lot more work than 20
 lines of C.  This is not solely due to the lack of begin/end brackets -- the
 lack of declarations and the high-level data types are also responsible -- but
 the indentation-based syntax certainly helps.
@@ -77,7 +77,7 @@
 
     1.1999999999999999555910790149937383830547332763671875 (decimal)
 
-The typical precision of 53 bits provides Python floats with 15-16
+The typical precision of 53 bits provides Python floats with 15--16
 decimal digits of accuracy.
 
 For a fuller explanation, please see the :ref:`floating point arithmetic
diff --git a/Doc/faq/general.rst b/Doc/faq/general.rst
--- a/Doc/faq/general.rst
+++ b/Doc/faq/general.rst
@@ -252,7 +252,7 @@
 
     Guido van Rossum and Jelke de Boer, "Interactively Testing Remote Servers
     Using the Python Programming Language", CWI Quarterly, Volume 4, Issue 4
-    (December 1991), Amsterdam, pp 283-303.
+    (December 1991), Amsterdam, pp 283--303.
 
 
 Are there any books on Python?
diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst
--- a/Doc/howto/urllib2.rst
+++ b/Doc/howto/urllib2.rst
@@ -240,8 +240,8 @@
 ~~~~~~~~~~~
 
 Because the default handlers handle redirects (codes in the 300 range), and
-codes in the 100-299 range indicate success, you will usually only see error
-codes in the 400-599 range.
+codes in the 100--299 range indicate success, you will usually only see error
+codes in the 400--599 range.
 
 :attr:`http.server.BaseHTTPRequestHandler.responses` is a useful dictionary of
 response codes in that shows all the response codes used by RFC 2616. The
diff --git a/Doc/library/calendar.rst b/Doc/library/calendar.rst
--- a/Doc/library/calendar.rst
+++ b/Doc/library/calendar.rst
@@ -47,7 +47,7 @@
 
    .. method:: itermonthdates(year, month)
 
-      Return an iterator for the month *month* (1-12) in the year *year*. This
+      Return an iterator for the month *month* (1--12) in the year *year*. This
       iterator will return all days (as :class:`datetime.date` objects) for the
       month and all days before the start of the month or after the end of the
       month that are required to get a complete week.
diff --git a/Doc/library/cmath.rst b/Doc/library/cmath.rst
--- a/Doc/library/cmath.rst
+++ b/Doc/library/cmath.rst
@@ -275,6 +275,6 @@
 
    Kahan, W:  Branch cuts for complex elementary functions; or, Much ado about
    nothing's sign bit.  In Iserles, A., and Powell, M. (eds.), The state of the art
-   in numerical analysis. Clarendon Press (1987) pp165-211.
+   in numerical analysis. Clarendon Press (1987) pp165--211.
 
 
diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst
--- a/Doc/library/codecs.rst
+++ b/Doc/library/codecs.rst
@@ -858,7 +858,7 @@
 ---------------------
 
 Strings are stored internally as sequences of code points in
-range ``0x0``-``0x10FFFF``.  (See :pep:`393` for
+range ``0x0``--``0x10FFFF``.  (See :pep:`393` for
 more details about the implementation.)
 Once a string object is used outside of CPU and memory, endianness
 and how these arrays are stored as bytes become an issue.  As with other
@@ -868,7 +868,7 @@
 collectivity referred to as :term:`text encodings <text encoding>`.
 
 The simplest text encoding (called ``'latin-1'`` or ``'iso-8859-1'``) maps
-the code points 0-255 to the bytes ``0x0``-``0xff``, which means that a string
+the code points 0--255 to the bytes ``0x0``--``0xff``, which means that a string
 object that contains code points above ``U+00FF`` can't be encoded with this
 codec. Doing so will raise a :exc:`UnicodeEncodeError` that looks
 like the following (although the details of the error message may differ):
@@ -877,7 +877,7 @@
 
 There's another group of encodings (the so called charmap encodings) that choose
 a different subset of all Unicode code points and how these code points are
-mapped to the bytes ``0x0``-``0xff``. To see how this is done simply open
+mapped to the bytes ``0x0``--``0xff``. To see how this is done simply open
 e.g. :file:`encodings/cp1252.py` (which is an encoding that is used primarily on
 Windows). There's a string constant with 256 characters that shows you which
 character is mapped to which byte value.
diff --git a/Doc/library/curses.ascii.rst b/Doc/library/curses.ascii.rst
--- a/Doc/library/curses.ascii.rst
+++ b/Doc/library/curses.ascii.rst
@@ -213,7 +213,7 @@
 
    Return a string representation of the ASCII character *c*.  If *c* is printable,
    this string is the character itself.  If the character is a control character
-   (0x00-0x1f) the string consists of a caret (``'^'``) followed by the
+   (0x00--0x1f) the string consists of a caret (``'^'``) followed by the
    corresponding uppercase letter. If the character is an ASCII delete (0x7f) the
    string is ``'^?'``.  If the character has its meta bit (0x80) set, the meta bit
    is stripped, the preceding rules applied, and ``'!'`` prepended to the result.
diff --git a/Doc/library/curses.rst b/Doc/library/curses.rst
--- a/Doc/library/curses.rst
+++ b/Doc/library/curses.rst
@@ -316,7 +316,7 @@
    Return the name of the key numbered *k*.  The name of a key generating printable
    ASCII character is the key's character.  The name of a control-key combination
    is a two-character string consisting of a caret followed by the corresponding
-   printable ASCII character.  The name of an alt-key combination (128-255) is a
+   printable ASCII character.  The name of an alt-key combination (128--255) is a
    string consisting of the prefix 'M-' followed by the name of the corresponding
    ASCII character.
 
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -686,7 +686,7 @@
    preceded by ``+`` or ``-`` (with no space in between) and surrounded by
    whitespace.  A base-n literal consists of the digits 0 to n-1, with ``a``
    to ``z`` (or ``A`` to ``Z``) having
-   values 10 to 35.  The default *base* is 10. The allowed values are 0 and 2-36.
+   values 10 to 35.  The default *base* is 10. The allowed values are 0 and 2--36.
    Base-2, -8, and -16 literals can be optionally prefixed with ``0b``/``0B``,
    ``0o``/``0O``, or ``0x``/``0X``, as with integer literals in code.  Base 0
    means to interpret exactly as a code literal, so that the actual base is 2,
diff --git a/Doc/library/ipaddress.rst b/Doc/library/ipaddress.rst
--- a/Doc/library/ipaddress.rst
+++ b/Doc/library/ipaddress.rst
@@ -99,7 +99,7 @@
    The following constitutes a valid IPv4 address:
 
    1. A string in decimal-dot notation, consisting of four decimal integers in
-      the inclusive range 0-255, separated by dots (e.g. ``192.168.0.1``). Each
+      the inclusive range 0--255, separated by dots (e.g. ``192.168.0.1``). Each
       integer represents an octet (byte) in the address. Leading zeroes are
       tolerated only for values less than 8 (as there is no ambiguity
       between the decimal and octal interpretations of such strings).
diff --git a/Doc/library/json.rst b/Doc/library/json.rst
--- a/Doc/library/json.rst
+++ b/Doc/library/json.rst
@@ -338,7 +338,7 @@
 
    If *strict* is false (``True`` is the default), then control characters
    will be allowed inside strings.  Control characters in this context are
-   those with character codes in the 0-31 range, including ``'\t'`` (tab),
+   those with character codes in the 0--31 range, including ``'\t'`` (tab),
    ``'\n'``, ``'\r'`` and ``'\0'``.
 
    If the data being deserialized is not a valid JSON document, a
diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -266,7 +266,7 @@
 
    M. Matsumoto and T. Nishimura, "Mersenne Twister: A 623-dimensionally
    equidistributed uniform pseudorandom number generator", ACM Transactions on
-   Modeling and Computer Simulation Vol. 8, No. 1, January pp.3-30 1998.
+   Modeling and Computer Simulation Vol. 8, No. 1, January pp.3--30 1998.
 
 
    `Complementary-Multiply-with-Carry recipe
diff --git a/Doc/library/statistics.rst b/Doc/library/statistics.rst
--- a/Doc/library/statistics.rst
+++ b/Doc/library/statistics.rst
@@ -191,9 +191,9 @@
       52.5
 
    In the following example, the data are rounded, so that each value represents
-   the midpoint of data classes, e.g. 1 is the midpoint of the class 0.5-1.5, 2
-   is the midpoint of 1.5-2.5, 3 is the midpoint of 2.5-3.5, etc.  With the data
-   given, the middle value falls somewhere in the class 3.5-4.5, and
+   the midpoint of data classes, e.g. 1 is the midpoint of the class 0.5--1.5, 2
+   is the midpoint of 1.5--2.5, 3 is the midpoint of 2.5--3.5, etc.  With the data
+   given, the middle value falls somewhere in the class 3.5--4.5, and
    interpolation is used to estimate it:
 
    .. doctest::
diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst
--- a/Doc/library/sys.rst
+++ b/Doc/library/sys.rst
@@ -256,7 +256,7 @@
    (defaulting to zero), or another type of object.  If it is an integer, zero
    is considered "successful termination" and any nonzero value is considered
    "abnormal termination" by shells and the like.  Most systems require it to be
-   in the range 0-127, and produce undefined results otherwise.  Some systems
+   in the range 0--127, and produce undefined results otherwise.  Some systems
    have a convention for assigning specific meanings to specific exit codes, but
    these are generally underdeveloped; Unix programs generally use 2 for command
    line syntax errors and 1 for all other kind of errors.  If another type of
diff --git a/Doc/whatsnew/2.1.rst b/Doc/whatsnew/2.1.rst
--- a/Doc/whatsnew/2.1.rst
+++ b/Doc/whatsnew/2.1.rst
@@ -731,7 +731,7 @@
          ...
 
   For a fuller discussion of the line I/O changes, see the python-dev summary for
-  January 1-15, 2001 at https://mail.python.org/pipermail/python-dev/2001-January/.
+  January 1--15, 2001 at https://mail.python.org/pipermail/python-dev/2001-January/.
 
 * A new method, :meth:`popitem`, was added to dictionaries to enable
   destructively iterating through the contents of a dictionary; this can be faster
diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst
--- a/Doc/whatsnew/2.7.rst
+++ b/Doc/whatsnew/2.7.rst
@@ -104,7 +104,7 @@
   when compared to earlier 2.x versions. Python 2.7 is currently expected to
   remain supported by the core development team (receiving security updates
   and other bug fixes) until at least 2020 (10 years after its initial
-  release, compared to the more typical support period of 18-24 months).
+  release, compared to the more typical support period of 18--24 months).
 
 * As the Python 2.7 standard library ages, making effective use of the
   Python Package Index (either directly or via a redistributor) becomes
@@ -989,7 +989,7 @@
   Gregory Smith; :issue:`1087418`).
 
 * The implementation of ``%`` checks for the left-side operand being
-  a Python string and special-cases it; this results in a 1-3%
+  a Python string and special-cases it; this results in a 1--3%
   performance increase for applications that frequently use ``%``
   with strings, such as templating libraries.
   (Implemented by Collin Winter; :issue:`5176`.)
diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst
--- a/Doc/whatsnew/3.5.rst
+++ b/Doc/whatsnew/3.5.rst
@@ -2129,8 +2129,8 @@
 (Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in
 :issue:`22003`.)
 
-The :func:`marshal.dumps` function is now faster: 65-85% with versions 3
-and 4, 20-25% with versions 0 to 2 on typical data, and up to 5 times in
+The :func:`marshal.dumps` function is now faster: 65--85% with versions 3
+and 4, 20--25% with versions 0 to 2 on typical data, and up to 5 times in
 best cases.
 (Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 26 06:50:44 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Sat, 26 Nov 2016 11:50:44 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328763=3A_Use_double_hyphens_=28rendered_as_en-d?=
 =?utf-8?q?ashes=29_in_numerical_ranges?=
Message-ID: <20161126115043.82176.36218.B94576DD@psf.io>

https://hg.python.org/cpython/rev/dc407f50e823
changeset:   105373:dc407f50e823
parent:      105369:07dfab805cf3
parent:      105372:3434d84efdd4
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sat Nov 26 13:50:21 2016 +0200
summary:
  Issue #28763: Use double hyphens (rendered as en-dashes) in numerical ranges
in the documentation.

files:
  Doc/faq/design.rst           |  4 ++--
  Doc/faq/general.rst          |  2 +-
  Doc/howto/urllib2.rst        |  4 ++--
  Doc/library/calendar.rst     |  2 +-
  Doc/library/cmath.rst        |  2 +-
  Doc/library/codecs.rst       |  6 +++---
  Doc/library/curses.ascii.rst |  2 +-
  Doc/library/curses.rst       |  2 +-
  Doc/library/functions.rst    |  2 +-
  Doc/library/ipaddress.rst    |  2 +-
  Doc/library/json.rst         |  2 +-
  Doc/library/random.rst       |  2 +-
  Doc/library/statistics.rst   |  6 +++---
  Doc/whatsnew/2.1.rst         |  2 +-
  Doc/whatsnew/2.7.rst         |  4 ++--
  Doc/whatsnew/3.5.rst         |  4 ++--
  16 files changed, 24 insertions(+), 24 deletions(-)


diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst
--- a/Doc/faq/design.rst
+++ b/Doc/faq/design.rst
@@ -31,7 +31,7 @@
 Many coding styles place begin/end brackets on a line by themselves.  This makes
 programs considerably longer and wastes valuable screen space, making it harder
 to get a good overview of a program.  Ideally, a function should fit on one
-screen (say, 20-30 lines).  20 lines of Python can do a lot more work than 20
+screen (say, 20--30 lines).  20 lines of Python can do a lot more work than 20
 lines of C.  This is not solely due to the lack of begin/end brackets -- the
 lack of declarations and the high-level data types are also responsible -- but
 the indentation-based syntax certainly helps.
@@ -77,7 +77,7 @@
 
     1.1999999999999999555910790149937383830547332763671875 (decimal)
 
-The typical precision of 53 bits provides Python floats with 15-16
+The typical precision of 53 bits provides Python floats with 15--16
 decimal digits of accuracy.
 
 For a fuller explanation, please see the :ref:`floating point arithmetic
diff --git a/Doc/faq/general.rst b/Doc/faq/general.rst
--- a/Doc/faq/general.rst
+++ b/Doc/faq/general.rst
@@ -252,7 +252,7 @@
 
     Guido van Rossum and Jelke de Boer, "Interactively Testing Remote Servers
     Using the Python Programming Language", CWI Quarterly, Volume 4, Issue 4
-    (December 1991), Amsterdam, pp 283-303.
+    (December 1991), Amsterdam, pp 283--303.
 
 
 Are there any books on Python?
diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst
--- a/Doc/howto/urllib2.rst
+++ b/Doc/howto/urllib2.rst
@@ -240,8 +240,8 @@
 ~~~~~~~~~~~
 
 Because the default handlers handle redirects (codes in the 300 range), and
-codes in the 100-299 range indicate success, you will usually only see error
-codes in the 400-599 range.
+codes in the 100--299 range indicate success, you will usually only see error
+codes in the 400--599 range.
 
 :attr:`http.server.BaseHTTPRequestHandler.responses` is a useful dictionary of
 response codes in that shows all the response codes used by RFC 2616. The
diff --git a/Doc/library/calendar.rst b/Doc/library/calendar.rst
--- a/Doc/library/calendar.rst
+++ b/Doc/library/calendar.rst
@@ -47,7 +47,7 @@
 
    .. method:: itermonthdates(year, month)
 
-      Return an iterator for the month *month* (1-12) in the year *year*. This
+      Return an iterator for the month *month* (1--12) in the year *year*. This
       iterator will return all days (as :class:`datetime.date` objects) for the
       month and all days before the start of the month or after the end of the
       month that are required to get a complete week.
diff --git a/Doc/library/cmath.rst b/Doc/library/cmath.rst
--- a/Doc/library/cmath.rst
+++ b/Doc/library/cmath.rst
@@ -309,4 +309,4 @@
 
    Kahan, W:  Branch cuts for complex elementary functions; or, Much ado about
    nothing's sign bit.  In Iserles, A., and Powell, M. (eds.), The state of the art
-   in numerical analysis. Clarendon Press (1987) pp165-211.
+   in numerical analysis. Clarendon Press (1987) pp165--211.
diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst
--- a/Doc/library/codecs.rst
+++ b/Doc/library/codecs.rst
@@ -858,7 +858,7 @@
 ---------------------
 
 Strings are stored internally as sequences of code points in
-range ``0x0``-``0x10FFFF``.  (See :pep:`393` for
+range ``0x0``--``0x10FFFF``.  (See :pep:`393` for
 more details about the implementation.)
 Once a string object is used outside of CPU and memory, endianness
 and how these arrays are stored as bytes become an issue.  As with other
@@ -868,7 +868,7 @@
 collectivity referred to as :term:`text encodings <text encoding>`.
 
 The simplest text encoding (called ``'latin-1'`` or ``'iso-8859-1'``) maps
-the code points 0-255 to the bytes ``0x0``-``0xff``, which means that a string
+the code points 0--255 to the bytes ``0x0``--``0xff``, which means that a string
 object that contains code points above ``U+00FF`` can't be encoded with this
 codec. Doing so will raise a :exc:`UnicodeEncodeError` that looks
 like the following (although the details of the error message may differ):
@@ -877,7 +877,7 @@
 
 There's another group of encodings (the so called charmap encodings) that choose
 a different subset of all Unicode code points and how these code points are
-mapped to the bytes ``0x0``-``0xff``. To see how this is done simply open
+mapped to the bytes ``0x0``--``0xff``. To see how this is done simply open
 e.g. :file:`encodings/cp1252.py` (which is an encoding that is used primarily on
 Windows). There's a string constant with 256 characters that shows you which
 character is mapped to which byte value.
diff --git a/Doc/library/curses.ascii.rst b/Doc/library/curses.ascii.rst
--- a/Doc/library/curses.ascii.rst
+++ b/Doc/library/curses.ascii.rst
@@ -213,7 +213,7 @@
 
    Return a string representation of the ASCII character *c*.  If *c* is printable,
    this string is the character itself.  If the character is a control character
-   (0x00-0x1f) the string consists of a caret (``'^'``) followed by the
+   (0x00--0x1f) the string consists of a caret (``'^'``) followed by the
    corresponding uppercase letter. If the character is an ASCII delete (0x7f) the
    string is ``'^?'``.  If the character has its meta bit (0x80) set, the meta bit
    is stripped, the preceding rules applied, and ``'!'`` prepended to the result.
diff --git a/Doc/library/curses.rst b/Doc/library/curses.rst
--- a/Doc/library/curses.rst
+++ b/Doc/library/curses.rst
@@ -316,7 +316,7 @@
    Return the name of the key numbered *k*.  The name of a key generating printable
    ASCII character is the key's character.  The name of a control-key combination
    is a two-character string consisting of a caret followed by the corresponding
-   printable ASCII character.  The name of an alt-key combination (128-255) is a
+   printable ASCII character.  The name of an alt-key combination (128--255) is a
    string consisting of the prefix 'M-' followed by the name of the corresponding
    ASCII character.
 
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -692,7 +692,7 @@
    preceded by ``+`` or ``-`` (with no space in between) and surrounded by
    whitespace.  A base-n literal consists of the digits 0 to n-1, with ``a``
    to ``z`` (or ``A`` to ``Z``) having
-   values 10 to 35.  The default *base* is 10. The allowed values are 0 and 2-36.
+   values 10 to 35.  The default *base* is 10. The allowed values are 0 and 2--36.
    Base-2, -8, and -16 literals can be optionally prefixed with ``0b``/``0B``,
    ``0o``/``0O``, or ``0x``/``0X``, as with integer literals in code.  Base 0
    means to interpret exactly as a code literal, so that the actual base is 2,
diff --git a/Doc/library/ipaddress.rst b/Doc/library/ipaddress.rst
--- a/Doc/library/ipaddress.rst
+++ b/Doc/library/ipaddress.rst
@@ -99,7 +99,7 @@
    The following constitutes a valid IPv4 address:
 
    1. A string in decimal-dot notation, consisting of four decimal integers in
-      the inclusive range 0-255, separated by dots (e.g. ``192.168.0.1``). Each
+      the inclusive range 0--255, separated by dots (e.g. ``192.168.0.1``). Each
       integer represents an octet (byte) in the address. Leading zeroes are
       tolerated only for values less than 8 (as there is no ambiguity
       between the decimal and octal interpretations of such strings).
diff --git a/Doc/library/json.rst b/Doc/library/json.rst
--- a/Doc/library/json.rst
+++ b/Doc/library/json.rst
@@ -345,7 +345,7 @@
 
    If *strict* is false (``True`` is the default), then control characters
    will be allowed inside strings.  Control characters in this context are
-   those with character codes in the 0-31 range, including ``'\t'`` (tab),
+   those with character codes in the 0--31 range, including ``'\t'`` (tab),
    ``'\n'``, ``'\r'`` and ``'\0'``.
 
    If the data being deserialized is not a valid JSON document, a
diff --git a/Doc/library/random.rst b/Doc/library/random.rst
--- a/Doc/library/random.rst
+++ b/Doc/library/random.rst
@@ -53,7 +53,7 @@
 
    M. Matsumoto and T. Nishimura, "Mersenne Twister: A 623-dimensionally
    equidistributed uniform pseudorandom number generator", ACM Transactions on
-   Modeling and Computer Simulation Vol. 8, No. 1, January pp.3-30 1998.
+   Modeling and Computer Simulation Vol. 8, No. 1, January pp.3--30 1998.
 
 
    `Complementary-Multiply-with-Carry recipe
diff --git a/Doc/library/statistics.rst b/Doc/library/statistics.rst
--- a/Doc/library/statistics.rst
+++ b/Doc/library/statistics.rst
@@ -224,9 +224,9 @@
       52.5
 
    In the following example, the data are rounded, so that each value represents
-   the midpoint of data classes, e.g. 1 is the midpoint of the class 0.5-1.5, 2
-   is the midpoint of 1.5-2.5, 3 is the midpoint of 2.5-3.5, etc.  With the data
-   given, the middle value falls somewhere in the class 3.5-4.5, and
+   the midpoint of data classes, e.g. 1 is the midpoint of the class 0.5--1.5, 2
+   is the midpoint of 1.5--2.5, 3 is the midpoint of 2.5--3.5, etc.  With the data
+   given, the middle value falls somewhere in the class 3.5--4.5, and
    interpolation is used to estimate it:
 
    .. doctest::
diff --git a/Doc/whatsnew/2.1.rst b/Doc/whatsnew/2.1.rst
--- a/Doc/whatsnew/2.1.rst
+++ b/Doc/whatsnew/2.1.rst
@@ -731,7 +731,7 @@
          ...
 
   For a fuller discussion of the line I/O changes, see the python-dev summary for
-  January 1-15, 2001 at https://mail.python.org/pipermail/python-dev/2001-January/.
+  January 1--15, 2001 at https://mail.python.org/pipermail/python-dev/2001-January/.
 
 * A new method, :meth:`popitem`, was added to dictionaries to enable
   destructively iterating through the contents of a dictionary; this can be faster
diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst
--- a/Doc/whatsnew/2.7.rst
+++ b/Doc/whatsnew/2.7.rst
@@ -104,7 +104,7 @@
   when compared to earlier 2.x versions. Python 2.7 is currently expected to
   remain supported by the core development team (receiving security updates
   and other bug fixes) until at least 2020 (10 years after its initial
-  release, compared to the more typical support period of 18-24 months).
+  release, compared to the more typical support period of 18--24 months).
 
 * As the Python 2.7 standard library ages, making effective use of the
   Python Package Index (either directly or via a redistributor) becomes
@@ -989,7 +989,7 @@
   Gregory Smith; :issue:`1087418`).
 
 * The implementation of ``%`` checks for the left-side operand being
-  a Python string and special-cases it; this results in a 1-3%
+  a Python string and special-cases it; this results in a 1--3%
   performance increase for applications that frequently use ``%``
   with strings, such as templating libraries.
   (Implemented by Collin Winter; :issue:`5176`.)
diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst
--- a/Doc/whatsnew/3.5.rst
+++ b/Doc/whatsnew/3.5.rst
@@ -2130,8 +2130,8 @@
 (Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in
 :issue:`22003`.)
 
-The :func:`marshal.dumps` function is now faster: 65-85% with versions 3
-and 4, 20-25% with versions 0 to 2 on typical data, and up to 5 times in
+The :func:`marshal.dumps` function is now faster: 65--85% with versions 3
+and 4, 20--25% with versions 0 to 2 on typical data, and up to 5 times in
 best cases.
 (Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 26 17:05:44 2016
From: python-checkins at python.org (lukasz.langa)
Date: Sat, 26 Nov 2016 22:05:44 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogRml4ZXMgIzI0MTQy?=
 =?utf-8?q?=3A_=5Bconfigparser=5D_always_join_multiline_values_to_not_leav?=
 =?utf-8?q?e_the?=
Message-ID: <20161126220544.23331.2567.6FED5D58@psf.io>

https://hg.python.org/cpython/rev/40567b8e3478
changeset:   105374:40567b8e3478
branch:      3.5
parent:      105370:77307437ae15
user:        ?ukasz Langa <lukasz at langa.pl>
date:        Sat Nov 26 14:00:39 2016 -0800
summary:
  Fixes #24142: [configparser] always join multiline values to not leave the parser in an invalid state

files:
  Lib/configparser.py           |   2 +-
  Lib/test/test_configparser.py |  24 +++++++++++++++++++++++
  Misc/ACKS                     |   1 +
  Misc/NEWS                     |   3 ++
  4 files changed, 29 insertions(+), 1 deletions(-)


diff --git a/Lib/configparser.py b/Lib/configparser.py
--- a/Lib/configparser.py
+++ b/Lib/configparser.py
@@ -1102,10 +1102,10 @@
                         # raised at the end of the file and will contain a
                         # list of all bogus lines
                         e = self._handle_error(e, fpname, lineno, line)
+        self._join_multiline_values()
         # if any parsing errors occurred, raise an exception
         if e:
             raise e
-        self._join_multiline_values()
 
     def _join_multiline_values(self):
         defaults = self.default_section, self._defaults
diff --git a/Lib/test/test_configparser.py b/Lib/test/test_configparser.py
--- a/Lib/test/test_configparser.py
+++ b/Lib/test/test_configparser.py
@@ -9,6 +9,7 @@
 
 from test import support
 
+
 class SortedDict(collections.UserDict):
 
     def items(self):
@@ -64,6 +65,7 @@
         cf.read_string(string)
         return cf
 
+
 class BasicTestCase(CfgParserTestCaseClass):
 
     def basic_test(self, cf):
@@ -828,6 +830,21 @@
         self.assertEqual(set(cf['section3'].keys()), set())
         self.assertEqual(cf.sections(), ['section1', 'section2', 'section3'])
 
+    def test_invalid_multiline_value(self):
+        if self.allow_no_value:
+            self.skipTest('if no_value is allowed, ParsingError is not raised')
+
+        invalid = textwrap.dedent("""\
+            [DEFAULT]
+            test {0} test
+            invalid""".format(self.delimiters[0])
+        )
+        cf = self.newconfig()
+        with self.assertRaises(configparser.ParsingError):
+            cf.read_string(invalid)
+        self.assertEqual(cf.get('DEFAULT', 'test'), 'test')
+        self.assertEqual(cf['DEFAULT']['test'], 'test')
+
 
 class StrictTestCase(BasicTestCase, unittest.TestCase):
     config_class = configparser.RawConfigParser
@@ -981,14 +998,17 @@
         cf.set("sect", "option2", "foo%%bar")
         self.assertEqual(cf.get("sect", "option2"), "foo%%bar")
 
+
 class ConfigParserTestCaseNonStandardDelimiters(ConfigParserTestCase):
     delimiters = (':=', '$')
     comment_prefixes = ('//', '"')
     inline_comment_prefixes = ('//', '"')
 
+
 class ConfigParserTestCaseNonStandardDefaultSection(ConfigParserTestCase):
     default_section = 'general'
 
+
 class MultilineValuesTestCase(BasicTestCase, unittest.TestCase):
     config_class = configparser.ConfigParser
     wonderful_spam = ("I'm having spam spam spam spam "
@@ -1017,6 +1037,7 @@
         self.assertEqual(cf_from_file.get('section8', 'lovely_spam4'),
                          self.wonderful_spam.replace('\t\n', '\n'))
 
+
 class RawConfigParserTestCase(BasicTestCase, unittest.TestCase):
     config_class = configparser.RawConfigParser
 
@@ -1059,11 +1080,13 @@
             cf.set('non-string', 1, 1)
             self.assertEqual(cf.get('non-string', 1), 1)
 
+
 class RawConfigParserTestCaseNonStandardDelimiters(RawConfigParserTestCase):
     delimiters = (':=', '$')
     comment_prefixes = ('//', '"')
     inline_comment_prefixes = ('//', '"')
 
+
 class RawConfigParserTestSambaConf(CfgParserTestCaseClass, unittest.TestCase):
     config_class = configparser.RawConfigParser
     comment_prefixes = ('#', ';', '----')
@@ -1258,6 +1281,7 @@
 class ConfigParserTestCaseNoValue(ConfigParserTestCase):
     allow_no_value = True
 
+
 class ConfigParserTestCaseTrickyFile(CfgParserTestCaseClass, unittest.TestCase):
     config_class = configparser.ConfigParser
     delimiters = {'='}
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -655,6 +655,7 @@
 Ludwig H?hne
 Gerhard H?ring
 Fredrik H??rd
+Florian H?ch
 Catalin Iacob
 Mihai Ibanescu
 Ali Ikinci
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -472,6 +472,9 @@
 
 - Issue #28703: Fix asyncio.iscoroutinefunction to handle Mock objects.
 
+- Issue #24142: Reading a corrupt config file left the parser in an
+  invalid state.  Original patch by Florian H?ch.
+
 IDLE
 ----
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 26 17:05:44 2016
From: python-checkins at python.org (lukasz.langa)
Date: Sat, 26 Nov 2016 22:05:44 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Merge_3=2E5=2C_fix_for_=2324142?=
Message-ID: <20161126220544.23293.44926.D9A0ED30@psf.io>

https://hg.python.org/cpython/rev/306cfb866399
changeset:   105375:306cfb866399
branch:      3.6
parent:      105372:3434d84efdd4
parent:      105374:40567b8e3478
user:        ?ukasz Langa <lukasz at langa.pl>
date:        Sat Nov 26 14:02:48 2016 -0800
summary:
  Merge 3.5, fix for #24142

files:
  Lib/configparser.py           |   2 +-
  Lib/test/test_configparser.py |  24 +++++++++++++++++++++++
  Misc/ACKS                     |   1 +
  Misc/NEWS                     |   3 ++
  4 files changed, 29 insertions(+), 1 deletions(-)


diff --git a/Lib/configparser.py b/Lib/configparser.py
--- a/Lib/configparser.py
+++ b/Lib/configparser.py
@@ -1102,10 +1102,10 @@
                         # raised at the end of the file and will contain a
                         # list of all bogus lines
                         e = self._handle_error(e, fpname, lineno, line)
+        self._join_multiline_values()
         # if any parsing errors occurred, raise an exception
         if e:
             raise e
-        self._join_multiline_values()
 
     def _join_multiline_values(self):
         defaults = self.default_section, self._defaults
diff --git a/Lib/test/test_configparser.py b/Lib/test/test_configparser.py
--- a/Lib/test/test_configparser.py
+++ b/Lib/test/test_configparser.py
@@ -9,6 +9,7 @@
 
 from test import support
 
+
 class SortedDict(collections.UserDict):
 
     def items(self):
@@ -64,6 +65,7 @@
         cf.read_string(string)
         return cf
 
+
 class BasicTestCase(CfgParserTestCaseClass):
 
     def basic_test(self, cf):
@@ -828,6 +830,21 @@
         self.assertEqual(set(cf['section3'].keys()), set())
         self.assertEqual(cf.sections(), ['section1', 'section2', 'section3'])
 
+    def test_invalid_multiline_value(self):
+        if self.allow_no_value:
+            self.skipTest('if no_value is allowed, ParsingError is not raised')
+
+        invalid = textwrap.dedent("""\
+            [DEFAULT]
+            test {0} test
+            invalid""".format(self.delimiters[0])
+        )
+        cf = self.newconfig()
+        with self.assertRaises(configparser.ParsingError):
+            cf.read_string(invalid)
+        self.assertEqual(cf.get('DEFAULT', 'test'), 'test')
+        self.assertEqual(cf['DEFAULT']['test'], 'test')
+
 
 class StrictTestCase(BasicTestCase, unittest.TestCase):
     config_class = configparser.RawConfigParser
@@ -981,14 +998,17 @@
         cf.set("sect", "option2", "foo%%bar")
         self.assertEqual(cf.get("sect", "option2"), "foo%%bar")
 
+
 class ConfigParserTestCaseNonStandardDelimiters(ConfigParserTestCase):
     delimiters = (':=', '$')
     comment_prefixes = ('//', '"')
     inline_comment_prefixes = ('//', '"')
 
+
 class ConfigParserTestCaseNonStandardDefaultSection(ConfigParserTestCase):
     default_section = 'general'
 
+
 class MultilineValuesTestCase(BasicTestCase, unittest.TestCase):
     config_class = configparser.ConfigParser
     wonderful_spam = ("I'm having spam spam spam spam "
@@ -1017,6 +1037,7 @@
         self.assertEqual(cf_from_file.get('section8', 'lovely_spam4'),
                          self.wonderful_spam.replace('\t\n', '\n'))
 
+
 class RawConfigParserTestCase(BasicTestCase, unittest.TestCase):
     config_class = configparser.RawConfigParser
 
@@ -1059,11 +1080,13 @@
             cf.set('non-string', 1, 1)
             self.assertEqual(cf.get('non-string', 1), 1)
 
+
 class RawConfigParserTestCaseNonStandardDelimiters(RawConfigParserTestCase):
     delimiters = (':=', '$')
     comment_prefixes = ('//', '"')
     inline_comment_prefixes = ('//', '"')
 
+
 class RawConfigParserTestSambaConf(CfgParserTestCaseClass, unittest.TestCase):
     config_class = configparser.RawConfigParser
     comment_prefixes = ('#', ';', '----')
@@ -1258,6 +1281,7 @@
 class ConfigParserTestCaseNoValue(ConfigParserTestCase):
     allow_no_value = True
 
+
 class ConfigParserTestCaseTrickyFile(CfgParserTestCaseClass, unittest.TestCase):
     config_class = configparser.ConfigParser
     delimiters = {'='}
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -663,6 +663,7 @@
 Ludwig H?hne
 Gerhard H?ring
 Fredrik H??rd
+Florian H?ch
 Catalin Iacob
 Mihai Ibanescu
 Ali Ikinci
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -17,6 +17,9 @@
 Library
 -------
 
+- Issue #24142: Reading a corrupt config file left configparser in an
+  invalid state.  Original patch by Florian H?ch.
+
 Tools/Demos
 -----------
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Sat Nov 26 17:05:44 2016
From: python-checkins at python.org (lukasz.langa)
Date: Sat, 26 Nov 2016 22:05:44 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_3=2E6=2C_fix_for_=2324142?=
Message-ID: <20161126220544.77814.53491.9152411A@psf.io>

https://hg.python.org/cpython/rev/876bee0bd0ba
changeset:   105376:876bee0bd0ba
parent:      105373:dc407f50e823
parent:      105375:306cfb866399
user:        ?ukasz Langa <lukasz at langa.pl>
date:        Sat Nov 26 14:04:40 2016 -0800
summary:
  Merge 3.6, fix for #24142

files:
  Lib/configparser.py           |   2 +-
  Lib/test/test_configparser.py |  24 +++++++++++++++++++++++
  Misc/ACKS                     |   1 +
  Misc/NEWS                     |   3 ++
  4 files changed, 29 insertions(+), 1 deletions(-)


diff --git a/Lib/configparser.py b/Lib/configparser.py
--- a/Lib/configparser.py
+++ b/Lib/configparser.py
@@ -1102,10 +1102,10 @@
                         # raised at the end of the file and will contain a
                         # list of all bogus lines
                         e = self._handle_error(e, fpname, lineno, line)
+        self._join_multiline_values()
         # if any parsing errors occurred, raise an exception
         if e:
             raise e
-        self._join_multiline_values()
 
     def _join_multiline_values(self):
         defaults = self.default_section, self._defaults
diff --git a/Lib/test/test_configparser.py b/Lib/test/test_configparser.py
--- a/Lib/test/test_configparser.py
+++ b/Lib/test/test_configparser.py
@@ -9,6 +9,7 @@
 
 from test import support
 
+
 class SortedDict(collections.UserDict):
 
     def items(self):
@@ -64,6 +65,7 @@
         cf.read_string(string)
         return cf
 
+
 class BasicTestCase(CfgParserTestCaseClass):
 
     def basic_test(self, cf):
@@ -828,6 +830,21 @@
         self.assertEqual(set(cf['section3'].keys()), set())
         self.assertEqual(cf.sections(), ['section1', 'section2', 'section3'])
 
+    def test_invalid_multiline_value(self):
+        if self.allow_no_value:
+            self.skipTest('if no_value is allowed, ParsingError is not raised')
+
+        invalid = textwrap.dedent("""\
+            [DEFAULT]
+            test {0} test
+            invalid""".format(self.delimiters[0])
+        )
+        cf = self.newconfig()
+        with self.assertRaises(configparser.ParsingError):
+            cf.read_string(invalid)
+        self.assertEqual(cf.get('DEFAULT', 'test'), 'test')
+        self.assertEqual(cf['DEFAULT']['test'], 'test')
+
 
 class StrictTestCase(BasicTestCase, unittest.TestCase):
     config_class = configparser.RawConfigParser
@@ -981,14 +998,17 @@
         cf.set("sect", "option2", "foo%%bar")
         self.assertEqual(cf.get("sect", "option2"), "foo%%bar")
 
+
 class ConfigParserTestCaseNonStandardDelimiters(ConfigParserTestCase):
     delimiters = (':=', '$')
     comment_prefixes = ('//', '"')
     inline_comment_prefixes = ('//', '"')
 
+
 class ConfigParserTestCaseNonStandardDefaultSection(ConfigParserTestCase):
     default_section = 'general'
 
+
 class MultilineValuesTestCase(BasicTestCase, unittest.TestCase):
     config_class = configparser.ConfigParser
     wonderful_spam = ("I'm having spam spam spam spam "
@@ -1017,6 +1037,7 @@
         self.assertEqual(cf_from_file.get('section8', 'lovely_spam4'),
                          self.wonderful_spam.replace('\t\n', '\n'))
 
+
 class RawConfigParserTestCase(BasicTestCase, unittest.TestCase):
     config_class = configparser.RawConfigParser
 
@@ -1059,11 +1080,13 @@
             cf.set('non-string', 1, 1)
             self.assertEqual(cf.get('non-string', 1), 1)
 
+
 class RawConfigParserTestCaseNonStandardDelimiters(RawConfigParserTestCase):
     delimiters = (':=', '$')
     comment_prefixes = ('//', '"')
     inline_comment_prefixes = ('//', '"')
 
+
 class RawConfigParserTestSambaConf(CfgParserTestCaseClass, unittest.TestCase):
     config_class = configparser.RawConfigParser
     comment_prefixes = ('#', ';', '----')
@@ -1258,6 +1281,7 @@
 class ConfigParserTestCaseNoValue(ConfigParserTestCase):
     allow_no_value = True
 
+
 class ConfigParserTestCaseTrickyFile(CfgParserTestCaseClass, unittest.TestCase):
     config_class = configparser.ConfigParser
     delimiters = {'='}
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -664,6 +664,7 @@
 Ludwig H?hne
 Gerhard H?ring
 Fredrik H??rd
+Florian H?ch
 Catalin Iacob
 Mihai Ibanescu
 Ali Ikinci
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -409,6 +409,9 @@
 
 - Issue #27972: Prohibit Tasks to await on themselves.
 
+- Issue #24142: Reading a corrupt config file left configparser in an
+  invalid state.  Original patch by Florian H?ch.
+
 Windows
 -------
 

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Sun Nov 27 04:05:45 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sun, 27 Nov 2016 09:05:45 +0000
Subject: [Python-checkins] Daily reference leaks (876bee0bd0ba): sum=4
Message-ID: <20161127090545.13735.68858.13D04BD3@psf.io>

results for 876bee0bd0ba on branch "default"
--------------------------------------------

test_functools leaked [0, 3, 1] memory blocks, sum=4


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogEKRAoc', '--timeout', '7200']

From python-checkins at python.org  Mon Nov 28 01:19:47 2016
From: python-checkins at python.org (zach.ware)
Date: Mon, 28 Nov 2016 06:19:47 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E6=29=3A_Fix_grammar_in?=
 =?utf-8?q?_whatsnew?=
Message-ID: <20161128061947.30014.88872.F63494C7@psf.io>

https://hg.python.org/cpython/rev/20f3636d396f
changeset:   105377:20f3636d396f
branch:      3.6
parent:      105375:306cfb866399
user:        Zachary Ware <zachary.ware at gmail.com>
date:        Mon Nov 28 00:19:07 2016 -0600
summary:
  Fix grammar in whatsnew

files:
  Doc/whatsnew/3.6.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -163,7 +163,7 @@
   remains unchanged - "python" refers to Python 2 in that case.
 
 * ``python.exe`` and ``pythonw.exe`` have been marked as long-path aware,
-  which means that when the 260 character path limit may no longer apply.
+  which means that the 260 character path limit may no longer apply.
   See :ref:`removing the MAX_PATH limitation <max-path>` for details.
 
 * A ``._pth`` file can be added to force isolated mode and fully specify

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 28 01:19:47 2016
From: python-checkins at python.org (zach.ware)
Date: Mon, 28 Nov 2016 06:19:47 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_with_3=2E6?=
Message-ID: <20161128061947.13668.72527.D7E0AFC5@psf.io>

https://hg.python.org/cpython/rev/21e095907c93
changeset:   105378:21e095907c93
parent:      105376:876bee0bd0ba
parent:      105377:20f3636d396f
user:        Zachary Ware <zachary.ware at gmail.com>
date:        Mon Nov 28 00:19:23 2016 -0600
summary:
  Merge with 3.6

files:
  Doc/whatsnew/3.6.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -163,7 +163,7 @@
   remains unchanged - "python" refers to Python 2 in that case.
 
 * ``python.exe`` and ``pythonw.exe`` have been marked as long-path aware,
-  which means that when the 260 character path limit may no longer apply.
+  which means that the 260 character path limit may no longer apply.
   See :ref:`removing the MAX_PATH limitation <max-path>` for details.
 
 * A ``._pth`` file can be added to force isolated mode and fully specify

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 28 03:52:18 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 28 Nov 2016 08:52:18 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2312844=3A_More_tha?=
 =?utf-8?q?n_255_arguments_can_now_be_passed_to_a_function=2E?=
Message-ID: <20161128085218.22664.80231.1901B0A9@psf.io>

https://hg.python.org/cpython/rev/5c1bb72c0f5d
changeset:   105379:5c1bb72c0f5d
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 28 10:52:05 2016 +0200
summary:
  Issue #12844: More than 255 arguments can now be passed to a function.

files:
  Doc/whatsnew/3.7.rst    |    2 +
  Lib/test/test_syntax.py |  128 ++++++++++++++-------------
  Misc/NEWS               |    2 +
  Python/ast.c            |    5 -
  4 files changed, 70 insertions(+), 67 deletions(-)


diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst
--- a/Doc/whatsnew/3.7.rst
+++ b/Doc/whatsnew/3.7.rst
@@ -75,6 +75,8 @@
 Other Language Changes
 ======================
 
+* More than 255 arguments can now be passed to a function.
+  (Contributed by Serhiy Storchaka in :issue:`12844`.)
 
 
 New Modules
diff --git a/Lib/test/test_syntax.py b/Lib/test/test_syntax.py
--- a/Lib/test/test_syntax.py
+++ b/Lib/test/test_syntax.py
@@ -139,69 +139,73 @@
 >>> f((x for x in L), 1)
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
 
->>> f(i0,  i1,  i2,  i3,  i4,  i5,  i6,  i7,  i8,  i9,  i10,  i11,
-...   i12,  i13,  i14,  i15,  i16,  i17,  i18,  i19,  i20,  i21,  i22,
-...   i23,  i24,  i25,  i26,  i27,  i28,  i29,  i30,  i31,  i32,  i33,
-...   i34,  i35,  i36,  i37,  i38,  i39,  i40,  i41,  i42,  i43,  i44,
-...   i45,  i46,  i47,  i48,  i49,  i50,  i51,  i52,  i53,  i54,  i55,
-...   i56,  i57,  i58,  i59,  i60,  i61,  i62,  i63,  i64,  i65,  i66,
-...   i67,  i68,  i69,  i70,  i71,  i72,  i73,  i74,  i75,  i76,  i77,
-...   i78,  i79,  i80,  i81,  i82,  i83,  i84,  i85,  i86,  i87,  i88,
-...   i89,  i90,  i91,  i92,  i93,  i94,  i95,  i96,  i97,  i98,  i99,
-...   i100,  i101,  i102,  i103,  i104,  i105,  i106,  i107,  i108,
-...   i109,  i110,  i111,  i112,  i113,  i114,  i115,  i116,  i117,
-...   i118,  i119,  i120,  i121,  i122,  i123,  i124,  i125,  i126,
-...   i127,  i128,  i129,  i130,  i131,  i132,  i133,  i134,  i135,
-...   i136,  i137,  i138,  i139,  i140,  i141,  i142,  i143,  i144,
-...   i145,  i146,  i147,  i148,  i149,  i150,  i151,  i152,  i153,
-...   i154,  i155,  i156,  i157,  i158,  i159,  i160,  i161,  i162,
-...   i163,  i164,  i165,  i166,  i167,  i168,  i169,  i170,  i171,
-...   i172,  i173,  i174,  i175,  i176,  i177,  i178,  i179,  i180,
-...   i181,  i182,  i183,  i184,  i185,  i186,  i187,  i188,  i189,
-...   i190,  i191,  i192,  i193,  i194,  i195,  i196,  i197,  i198,
-...   i199,  i200,  i201,  i202,  i203,  i204,  i205,  i206,  i207,
-...   i208,  i209,  i210,  i211,  i212,  i213,  i214,  i215,  i216,
-...   i217,  i218,  i219,  i220,  i221,  i222,  i223,  i224,  i225,
-...   i226,  i227,  i228,  i229,  i230,  i231,  i232,  i233,  i234,
-...   i235,  i236,  i237,  i238,  i239,  i240,  i241,  i242,  i243,
-...   i244,  i245,  i246,  i247,  i248,  i249,  i250,  i251,  i252,
-...   i253,  i254,  i255)
-Traceback (most recent call last):
-SyntaxError: more than 255 arguments
+>>> def g(*args, **kwargs):
+...     print(args, sorted(kwargs.items()))
+>>> g(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19,
+...   20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37,
+...   38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55,
+...   56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73,
+...   74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91,
+...   92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107,
+...   108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121,
+...   122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135,
+...   136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149,
+...   150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163,
+...   164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177,
+...   178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191,
+...   192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205,
+...   206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219,
+...   220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233,
+...   234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247,
+...   248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261,
+...   262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275,
+...   276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289,
+...   290, 291, 292, 293, 294, 295, 296, 297, 298, 299)  # doctest: +ELLIPSIS
+(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, ..., 297, 298, 299) []
 
-The actual error cases counts positional arguments, keyword arguments,
-and generator expression arguments separately.  This test combines the
-three.
-
->>> f(i0,  i1,  i2,  i3,  i4,  i5,  i6,  i7,  i8,  i9,  i10,  i11,
-...   i12,  i13,  i14,  i15,  i16,  i17,  i18,  i19,  i20,  i21,  i22,
-...   i23,  i24,  i25,  i26,  i27,  i28,  i29,  i30,  i31,  i32,  i33,
-...   i34,  i35,  i36,  i37,  i38,  i39,  i40,  i41,  i42,  i43,  i44,
-...   i45,  i46,  i47,  i48,  i49,  i50,  i51,  i52,  i53,  i54,  i55,
-...   i56,  i57,  i58,  i59,  i60,  i61,  i62,  i63,  i64,  i65,  i66,
-...   i67,  i68,  i69,  i70,  i71,  i72,  i73,  i74,  i75,  i76,  i77,
-...   i78,  i79,  i80,  i81,  i82,  i83,  i84,  i85,  i86,  i87,  i88,
-...   i89,  i90,  i91,  i92,  i93,  i94,  i95,  i96,  i97,  i98,  i99,
-...   i100,  i101,  i102,  i103,  i104,  i105,  i106,  i107,  i108,
-...   i109,  i110,  i111,  i112,  i113,  i114,  i115,  i116,  i117,
-...   i118,  i119,  i120,  i121,  i122,  i123,  i124,  i125,  i126,
-...   i127,  i128,  i129,  i130,  i131,  i132,  i133,  i134,  i135,
-...   i136,  i137,  i138,  i139,  i140,  i141,  i142,  i143,  i144,
-...   i145,  i146,  i147,  i148,  i149,  i150,  i151,  i152,  i153,
-...   i154,  i155,  i156,  i157,  i158,  i159,  i160,  i161,  i162,
-...   i163,  i164,  i165,  i166,  i167,  i168,  i169,  i170,  i171,
-...   i172,  i173,  i174,  i175,  i176,  i177,  i178,  i179,  i180,
-...   i181,  i182,  i183,  i184,  i185,  i186,  i187,  i188,  i189,
-...   i190,  i191,  i192,  i193,  i194,  i195,  i196,  i197,  i198,
-...   i199,  i200,  i201,  i202,  i203,  i204,  i205,  i206,  i207,
-...   i208,  i209,  i210,  i211,  i212,  i213,  i214,  i215,  i216,
-...   i217,  i218,  i219,  i220,  i221,  i222,  i223,  i224,  i225,
-...   i226,  i227,  i228,  i229,  i230,  i231,  i232,  i233,  i234,
-...   i235, i236,  i237,  i238,  i239,  i240,  i241,  i242,  i243,
-...   (x for x in i244),  i245,  i246,  i247,  i248,  i249,  i250,  i251,
-...    i252=1, i253=1,  i254=1,  i255=1)
-Traceback (most recent call last):
-SyntaxError: more than 255 arguments
+>>> g(a000=0, a001=1, a002=2, a003=3, a004=4, a005=5, a006=6, a007=7, a008=8,
+...   a009=9, a010=10, a011=11, a012=12, a013=13, a014=14, a015=15, a016=16,
+...   a017=17, a018=18, a019=19, a020=20, a021=21, a022=22, a023=23, a024=24,
+...   a025=25, a026=26, a027=27, a028=28, a029=29, a030=30, a031=31, a032=32,
+...   a033=33, a034=34, a035=35, a036=36, a037=37, a038=38, a039=39, a040=40,
+...   a041=41, a042=42, a043=43, a044=44, a045=45, a046=46, a047=47, a048=48,
+...   a049=49, a050=50, a051=51, a052=52, a053=53, a054=54, a055=55, a056=56,
+...   a057=57, a058=58, a059=59, a060=60, a061=61, a062=62, a063=63, a064=64,
+...   a065=65, a066=66, a067=67, a068=68, a069=69, a070=70, a071=71, a072=72,
+...   a073=73, a074=74, a075=75, a076=76, a077=77, a078=78, a079=79, a080=80,
+...   a081=81, a082=82, a083=83, a084=84, a085=85, a086=86, a087=87, a088=88,
+...   a089=89, a090=90, a091=91, a092=92, a093=93, a094=94, a095=95, a096=96,
+...   a097=97, a098=98, a099=99, a100=100, a101=101, a102=102, a103=103,
+...   a104=104, a105=105, a106=106, a107=107, a108=108, a109=109, a110=110,
+...   a111=111, a112=112, a113=113, a114=114, a115=115, a116=116, a117=117,
+...   a118=118, a119=119, a120=120, a121=121, a122=122, a123=123, a124=124,
+...   a125=125, a126=126, a127=127, a128=128, a129=129, a130=130, a131=131,
+...   a132=132, a133=133, a134=134, a135=135, a136=136, a137=137, a138=138,
+...   a139=139, a140=140, a141=141, a142=142, a143=143, a144=144, a145=145,
+...   a146=146, a147=147, a148=148, a149=149, a150=150, a151=151, a152=152,
+...   a153=153, a154=154, a155=155, a156=156, a157=157, a158=158, a159=159,
+...   a160=160, a161=161, a162=162, a163=163, a164=164, a165=165, a166=166,
+...   a167=167, a168=168, a169=169, a170=170, a171=171, a172=172, a173=173,
+...   a174=174, a175=175, a176=176, a177=177, a178=178, a179=179, a180=180,
+...   a181=181, a182=182, a183=183, a184=184, a185=185, a186=186, a187=187,
+...   a188=188, a189=189, a190=190, a191=191, a192=192, a193=193, a194=194,
+...   a195=195, a196=196, a197=197, a198=198, a199=199, a200=200, a201=201,
+...   a202=202, a203=203, a204=204, a205=205, a206=206, a207=207, a208=208,
+...   a209=209, a210=210, a211=211, a212=212, a213=213, a214=214, a215=215,
+...   a216=216, a217=217, a218=218, a219=219, a220=220, a221=221, a222=222,
+...   a223=223, a224=224, a225=225, a226=226, a227=227, a228=228, a229=229,
+...   a230=230, a231=231, a232=232, a233=233, a234=234, a235=235, a236=236,
+...   a237=237, a238=238, a239=239, a240=240, a241=241, a242=242, a243=243,
+...   a244=244, a245=245, a246=246, a247=247, a248=248, a249=249, a250=250,
+...   a251=251, a252=252, a253=253, a254=254, a255=255, a256=256, a257=257,
+...   a258=258, a259=259, a260=260, a261=261, a262=262, a263=263, a264=264,
+...   a265=265, a266=266, a267=267, a268=268, a269=269, a270=270, a271=271,
+...   a272=272, a273=273, a274=274, a275=275, a276=276, a277=277, a278=278,
+...   a279=279, a280=280, a281=281, a282=282, a283=283, a284=284, a285=285,
+...   a286=286, a287=287, a288=288, a289=289, a290=290, a291=291, a292=292,
+...   a293=293, a294=294, a295=295, a296=296, a297=297, a298=298, a299=299)
+...  # doctest: +ELLIPSIS
+() [('a000', 0), ('a001', 1), ('a002', 2), ..., ('a298', 298), ('a299', 299)]
 
 >>> f(lambda x: x[0] = 3)
 Traceback (most recent call last):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,8 @@
 Core and Builtins
 -----------------
 
+- Issue #12844: More than 255 arguments can now be passed to a function.
+
 - Issue #28782: Fix a bug in the implementation ``yield from`` when checking
   if the next instruction is YIELD_FROM. Regression introduced by WORDCODE
   (issue #26647).
diff --git a/Python/ast.c b/Python/ast.c
--- a/Python/ast.c
+++ b/Python/ast.c
@@ -2738,11 +2738,6 @@
         return NULL;
     }
 
-    if (nargs + nkeywords + ngens > 255) {
-        ast_error(c, n, "more than 255 arguments");
-        return NULL;
-    }
-
     args = _Py_asdl_seq_new(nargs + ngens, c->c_arena);
     if (!args)
         return NULL;

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Mon Nov 28 04:06:24 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Mon, 28 Nov 2016 09:06:24 +0000
Subject: [Python-checkins] Daily reference leaks (876bee0bd0ba): sum=11
Message-ID: <20161128090624.81547.42052.9141950B@psf.io>

results for 876bee0bd0ba on branch "default"
--------------------------------------------

test_collections leaked [0, 0, 7] memory blocks, sum=7
test_functools leaked [0, 3, 1] memory blocks, sum=4
test_multiprocessing_fork leaked [0, -2, 2] memory blocks, sum=0
test_multiprocessing_forkserver leaked [-1, -1, 2] memory blocks, sum=0


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogx9LuNm', '--timeout', '7200']

From python-checkins at python.org  Mon Nov 28 06:05:23 2016
From: python-checkins at python.org (victor.stinner)
Date: Mon, 28 Nov 2016 11:05:23 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_CALL=5FPROFILE_spec?=
 =?utf-8?q?ial_build?=
Message-ID: <20161128110513.31341.52199.9099FD85@psf.io>

https://hg.python.org/cpython/rev/5aa2171ee43f
changeset:   105380:5aa2171ee43f
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Nov 28 11:59:04 2016 +0100
summary:
  Remove CALL_PROFILE special build

Issue #28799:

* Remove the PyEval_GetCallStats() function.
* Deprecate the untested and undocumented sys.callstats() function.
* Remove the CALL_PROFILE special build

Use the sys.setprofile() function, cProfile or profile module to profile
function calls.

files:
  Doc/c-api/init.rst |  40 --------------
  Include/ceval.h    |   1 -
  Misc/NEWS          |   5 +
  Python/ceval.c     |  91 ----------------------------------
  Python/sysmodule.c |  15 +++++-
  5 files changed, 19 insertions(+), 133 deletions(-)


diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst
--- a/Doc/c-api/init.rst
+++ b/Doc/c-api/init.rst
@@ -1147,46 +1147,6 @@
    :c:func:`PyEval_SetProfile`, except the tracing function does receive line-number
    events.
 
-.. c:function:: PyObject* PyEval_GetCallStats(PyObject *self)
-
-   Return a tuple of function call counts.  There are constants defined for the
-   positions within the tuple:
-
-   +-------------------------------+-------+
-   | Name                          | Value |
-   +===============================+=======+
-   | :const:`PCALL_ALL`            | 0     |
-   +-------------------------------+-------+
-   | :const:`PCALL_FUNCTION`       | 1     |
-   +-------------------------------+-------+
-   | :const:`PCALL_FAST_FUNCTION`  | 2     |
-   +-------------------------------+-------+
-   | :const:`PCALL_FASTER_FUNCTION`| 3     |
-   +-------------------------------+-------+
-   | :const:`PCALL_METHOD`         | 4     |
-   +-------------------------------+-------+
-   | :const:`PCALL_BOUND_METHOD`   | 5     |
-   +-------------------------------+-------+
-   | :const:`PCALL_CFUNCTION`      | 6     |
-   +-------------------------------+-------+
-   | :const:`PCALL_TYPE`           | 7     |
-   +-------------------------------+-------+
-   | :const:`PCALL_GENERATOR`      | 8     |
-   +-------------------------------+-------+
-   | :const:`PCALL_OTHER`          | 9     |
-   +-------------------------------+-------+
-   | :const:`PCALL_POP`            | 10    |
-   +-------------------------------+-------+
-
-   :const:`PCALL_FAST_FUNCTION` means no argument tuple needs to be created.
-   :const:`PCALL_FASTER_FUNCTION` means that the fast-path frame setup code is used.
-
-   If there is a method call where the call can be optimized by changing
-   the argument tuple and calling the function directly, it gets recorded
-   twice.
-
-   This function is only present if Python is compiled with :const:`CALL_PROFILE`
-   defined.
 
 .. _advanced-debugging:
 
diff --git a/Include/ceval.h b/Include/ceval.h
--- a/Include/ceval.h
+++ b/Include/ceval.h
@@ -120,7 +120,6 @@
 PyAPI_FUNC(const char *) PyEval_GetFuncName(PyObject *);
 PyAPI_FUNC(const char *) PyEval_GetFuncDesc(PyObject *);
 
-PyAPI_FUNC(PyObject *) PyEval_GetCallStats(PyObject *);
 PyAPI_FUNC(PyObject *) PyEval_EvalFrame(struct _frame *);
 PyAPI_FUNC(PyObject *) PyEval_EvalFrameEx(struct _frame *f, int exc);
 #ifndef Py_LIMITED_API
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,11 @@
 Core and Builtins
 -----------------
 
+- Issue #28799: Remove the ``PyEval_GetCallStats()`` function and deprecate
+  the untested and undocumented ``sys.callstats()`` function. Remove the
+  ``CALL_PROFILE`` special build: use the :func:`sys.setprofile` function,
+  :mod:`cProfile` or :mod:`profile` to profile function calls.
+
 - Issue #12844: More than 255 arguments can now be passed to a function.
 
 - Issue #28782: Fix a bug in the implementation ``yield from`` when checking
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -83,63 +83,6 @@
 #endif
 #endif
 
-/* Function call profile */
-#ifdef CALL_PROFILE
-#define PCALL_NUM 11
-static int pcall[PCALL_NUM];
-
-#define PCALL_ALL 0
-#define PCALL_FUNCTION 1
-#define PCALL_FAST_FUNCTION 2
-#define PCALL_FASTER_FUNCTION 3
-#define PCALL_METHOD 4
-#define PCALL_BOUND_METHOD 5
-#define PCALL_CFUNCTION 6
-#define PCALL_TYPE 7
-#define PCALL_GENERATOR 8
-#define PCALL_OTHER 9
-#define PCALL_POP 10
-
-/* Notes about the statistics
-
-   PCALL_FAST stats
-
-   FAST_FUNCTION means no argument tuple needs to be created.
-   FASTER_FUNCTION means that the fast-path frame setup code is used.
-
-   If there is a method call where the call can be optimized by changing
-   the argument tuple and calling the function directly, it gets recorded
-   twice.
-
-   As a result, the relationship among the statistics appears to be
-   PCALL_ALL == PCALL_FUNCTION + PCALL_METHOD - PCALL_BOUND_METHOD +
-                PCALL_CFUNCTION + PCALL_TYPE + PCALL_GENERATOR + PCALL_OTHER
-   PCALL_FUNCTION > PCALL_FAST_FUNCTION > PCALL_FASTER_FUNCTION
-   PCALL_METHOD > PCALL_BOUND_METHOD
-*/
-
-#define PCALL(POS) pcall[POS]++
-
-PyObject *
-PyEval_GetCallStats(PyObject *self)
-{
-    return Py_BuildValue("iiiiiiiiiii",
-                         pcall[0], pcall[1], pcall[2], pcall[3],
-                         pcall[4], pcall[5], pcall[6], pcall[7],
-                         pcall[8], pcall[9], pcall[10]);
-}
-#else
-#define PCALL(O)
-
-PyObject *
-PyEval_GetCallStats(PyObject *self)
-{
-    Py_INCREF(Py_None);
-    return Py_None;
-}
-#endif
-
-
 #ifdef WITH_THREAD
 #define GIL_REQUEST _Py_atomic_load_relaxed(&gil_drop_request)
 #else
@@ -3278,7 +3221,6 @@
         PREDICTED(CALL_FUNCTION);
         TARGET(CALL_FUNCTION) {
             PyObject **sp, *res;
-            PCALL(PCALL_ALL);
             sp = stack_pointer;
             res = call_function(&sp, oparg, NULL);
             stack_pointer = sp;
@@ -3294,7 +3236,6 @@
 
             names = POP();
             assert(PyTuple_CheckExact(names) && PyTuple_GET_SIZE(names) <= oparg);
-            PCALL(PCALL_ALL);
             sp = stack_pointer;
             res = call_function(&sp, oparg, names);
             stack_pointer = sp;
@@ -3309,7 +3250,6 @@
 
         TARGET(CALL_FUNCTION_EX) {
             PyObject *func, *callargs, *kwargs = NULL, *result;
-            PCALL(PCALL_ALL);
             if (oparg & 0x01) {
                 kwargs = POP();
                 if (!PyDict_CheckExact(kwargs)) {
@@ -4099,8 +4039,6 @@
          * when the generator is resumed. */
         Py_CLEAR(f->f_back);
 
-        PCALL(PCALL_GENERATOR);
-
         /* Create a new generator that owns the ready to run frame
          * and return that as the value. */
         if (is_coro) {
@@ -4793,8 +4731,6 @@
     if (PyCFunction_Check(func)) {
         PyThreadState *tstate = PyThreadState_GET();
 
-        PCALL(PCALL_CFUNCTION);
-
         stack = (*pp_stack) - nargs - nkwargs;
         C_TRACE(x, _PyCFunction_FastCallKeywords(func, stack, nargs, kwnames));
     }
@@ -4802,8 +4738,6 @@
         if (PyMethod_Check(func) && PyMethod_GET_SELF(func) != NULL) {
             /* optimize access to bound methods */
             PyObject *self = PyMethod_GET_SELF(func);
-            PCALL(PCALL_METHOD);
-            PCALL(PCALL_BOUND_METHOD);
             Py_INCREF(self);
             func = PyMethod_GET_FUNCTION(func);
             Py_INCREF(func);
@@ -4835,7 +4769,6 @@
     while ((*pp_stack) > pfunc) {
         w = EXT_POP(*pp_stack);
         Py_DECREF(w);
-        PCALL(PCALL_POP);
     }
 
     return x;
@@ -4860,7 +4793,6 @@
     Py_ssize_t i;
     PyObject *result;
 
-    PCALL(PCALL_FASTER_FUNCTION);
     assert(globals != NULL);
     /* XXX Perhaps we should create a specialized
        PyFrame_New() that doesn't take locals, but does
@@ -4906,9 +4838,6 @@
     /* kwnames must only contains str strings, no subclass, and all keys must
        be unique */
 
-    PCALL(PCALL_FUNCTION);
-    PCALL(PCALL_FAST_FUNCTION);
-
     if (co->co_kwonlyargcount == 0 && nkwargs == 0 &&
         co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE))
     {
@@ -4971,9 +4900,6 @@
     assert(nargs == 0 || args != NULL);
     assert(kwargs == NULL || PyDict_Check(kwargs));
 
-    PCALL(PCALL_FUNCTION);
-    PCALL(PCALL_FAST_FUNCTION);
-
     if (co->co_kwonlyargcount == 0 &&
         (kwargs == NULL || PyDict_Size(kwargs) == 0) &&
         co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE))
@@ -5041,23 +4967,6 @@
 static PyObject *
 do_call_core(PyObject *func, PyObject *callargs, PyObject *kwdict)
 {
-#ifdef CALL_PROFILE
-    /* At this point, we have to look at the type of func to
-       update the call stats properly.  Do it here so as to avoid
-       exposing the call stats machinery outside ceval.c
-    */
-    if (PyFunction_Check(func))
-        PCALL(PCALL_FUNCTION);
-    else if (PyMethod_Check(func))
-        PCALL(PCALL_METHOD);
-    else if (PyType_Check(func))
-        PCALL(PCALL_TYPE);
-    else if (PyCFunction_Check(func))
-        PCALL(PCALL_CFUNCTION);
-    else
-        PCALL(PCALL_OTHER);
-#endif
-
     if (PyCFunction_Check(func)) {
         PyObject *result;
         PyThreadState *tstate = PyThreadState_GET();
diff --git a/Python/sysmodule.c b/Python/sysmodule.c
--- a/Python/sysmodule.c
+++ b/Python/sysmodule.c
@@ -1287,6 +1287,19 @@
 10. Number of stack pops performed by call_function()"
 );
 
+static PyObject *
+sys_callstats(PyObject *self)
+{
+    if (PyErr_WarnEx(PyExc_DeprecationWarning,
+                      "sys.callstats() has been deprecated in Python 3.7 "
+                      "and will be removed in the future", 1) < 0) {
+        return NULL;
+    }
+
+    Py_RETURN_NONE;
+}
+
+
 #ifdef __cplusplus
 extern "C" {
 #endif
@@ -1352,7 +1365,7 @@
 
 static PyMethodDef sys_methods[] = {
     /* Might as well keep this in alphabetic order */
-    {"callstats", (PyCFunction)PyEval_GetCallStats, METH_NOARGS,
+    {"callstats", (PyCFunction)sys_callstats, METH_NOARGS,
      callstats_doc},
     {"_clear_type_cache",       sys_clear_type_cache,     METH_NOARGS,
      sys_clear_type_cache__doc__},

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 28 06:06:19 2016
From: python-checkins at python.org (victor.stinner)
Date: Mon, 28 Nov 2016 11:06:19 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328799=3A_Update_M?=
 =?utf-8?q?isc/SpecialBuilds=2Etxt?=
Message-ID: <20161128110618.22982.35175.BEA42564@psf.io>

https://hg.python.org/cpython/rev/07d8272d61e7
changeset:   105381:07d8272d61e7
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Nov 28 12:06:13 2016 +0100
summary:
  Issue #28799: Update Misc/SpecialBuilds.txt

Remove CALL_PROFILE.

files:
  Misc/SpecialBuilds.txt |  11 -----------
  1 files changed, 0 insertions(+), 11 deletions(-)


diff --git a/Misc/SpecialBuilds.txt b/Misc/SpecialBuilds.txt
--- a/Misc/SpecialBuilds.txt
+++ b/Misc/SpecialBuilds.txt
@@ -223,14 +223,3 @@
 argument and values pushed onto and popped off the value stack.
 
 Not useful very often, but very useful when needed.
-
-
-CALL_PROFILE
-------------
-
-Count the number of function calls executed.
-
-When this symbol is defined, the ceval mainloop and helper functions count the
-number of function calls made.  It keeps detailed statistics about what kind of
-object was called and whether the call hit any of the special fast paths in the
-code.

-- 
Repository URL: https://hg.python.org/cpython

From lp_benchmark_robot at intel.com  Mon Nov 28 07:05:05 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Mon, 28 Nov 2016 12:05:05 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python 2.7 2016-11-28
Message-ID: <73ff5e58-ba4a-4ee7-aaf6-4c1fcaa3bfa0@irsmsx104.ger.corp.intel.com>

No new revisions. Here are the previous results:

Results for project Python 2.7, build date 2016-11-28 10:55:02 +0000
commit:		59bd48afa1bc
previous commit:	6eefdf0fade6
revision date:	2016-11-26 11:43:39 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.14%          1.76%          5.07%             8.38%
:-)             pybench      0.24%         -0.01%          5.70%             4.55%
:-(            regex_v8      0.57%          0.06%         -2.26%            10.27%
:-)               nbody      0.85%          0.01%          7.90%             2.67%
:-)        json_dump_v2      0.34%         -0.26%          3.54%            10.10%
:-|      normal_startup      1.12%          0.06%         -0.24%             2.36%
:-)             ssbench      0.20%         -0.56%          2.06%             1.80%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-2-7-2016-11-28/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Mon Nov 28 07:06:26 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Mon, 28 Nov 2016 12:06:26 +0000
Subject: [Python-checkins] NEUTRAL Benchmark Results for Python Default
 2016-11-28
Message-ID: <8cfad661-fd57-44fe-b6bb-f65b59742b5b@irsmsx104.ger.corp.intel.com>

Results for project Python default, build date 2016-11-28 10:11:23 +0000
commit:		5c1bb72c0f5d
previous commit:	45713818fd81
revision date:	2016-11-28 08:52:05 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.25%         -0.78%          5.93%            14.94%
:-)             pybench      0.16%          0.10%          5.47%             5.75%
:-(            regex_v8      3.81%          0.06%         -2.99%             3.38%
:-|               nbody      0.24%         -0.06%         -0.62%             9.22%
:-(        json_dump_v2      0.40%         -0.32%         -4.34%            12.56%
:-|      normal_startup      0.84%          0.21%          0.84%             6.47%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-default-2016-11-28/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Mon Nov 28 11:46:02 2016
From: python-checkins at python.org (yury.selivanov)
Date: Mon, 28 Nov 2016 16:46:02 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogSXNzdWUgIzI4NjM1?=
 =?utf-8?q?=3A_Document_Python_3=2E6_opcode_changes?=
Message-ID: <20161128164602.13693.88639.8F362337@psf.io>

https://hg.python.org/cpython/rev/52038705827d
changeset:   105382:52038705827d
branch:      3.6
parent:      105377:20f3636d396f
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 28 11:45:36 2016 -0500
summary:
  Issue #28635: Document Python 3.6 opcode changes

Thanks to Serhiy Storchaka for pointing out the missing notes.

Patch by Elvis Pranskevichus.

files:
  Doc/whatsnew/3.6.rst |  34 ++++++++++++++++++++++++++++++++
  1 files changed, 34 insertions(+), 0 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -2240,3 +2240,37 @@
 
 * :c:func:`Py_Exit` (and the main interpreter) now override the exit status
   with 120 if flushing buffered data failed.  See :issue:`5319`.
+
+
+CPython bytecode changes
+------------------------
+
+There have been several major changes to the :term:`bytecode` in Python 3.6.
+
+* The Python interpreter now uses a 16-bit wordcode instead of bytecode.
+  (Contributed by Demur Rumed with input and reviews from
+  Serhiy Storchaka and Victor Stinner in :issue:`26647` and :issue:`28050`.)
+
+* The new :opcode:`FORMAT_VALUE` and :opcode:`BUILD_STRING` opcodes as part
+  of the  :ref:`formatted string literal <whatsnew36-pep498>` implementation.
+  (Contributed by Eric Smith in :issue:`25483` and
+  Serhiy Storchaka in :issue:`27078`.)
+
+* The new :opcode:`BUILD_CONST_KEY_MAP` opcode to optimize the creation
+  of dictionaries with constant keys.
+  (Contributed by Serhiy Storchaka in :issue:`27140`.)
+
+* The function call opcodes have been heavily reworked for better performance
+  and simpler implementation.
+  The :opcode:`MAKE_FUNCTION`, :opcode:`CALL_FUNCTION`,
+  :opcode:`CALL_FUNCTION_KW` and :opcode:`BUILD_MAP_UNPACK_WITH_CALL` opcodes
+  have been modified, the new :opcode:`CALL_FUNCTION_EX` and
+  :opcode:`BUILD_TUPLE_UNPACK_WITH_CALL` have been added, and
+  ``CALL_FUNCTION_VAR``, ``CALL_FUNCTION_VAR_KW`` and ``MAKE_CLOSURE`` opcodes
+  have been removed.
+  (Contributed by Demur Rumed in :issue:`27095`, and Serhiy Storchaka in
+  :issue:`27213`, :issue:`28257`.)
+
+* The new :opcode:`SETUP_ANNOTATIONS` and :opcode:`STORE_ANNOTATION` opcodes
+  have been added to support the new :term:`variable annotation` syntax.
+  (Contributed by Ivan Levkivskyi in :issue:`27985`.)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 28 11:46:04 2016
From: python-checkins at python.org (yury.selivanov)
Date: Mon, 28 Nov 2016 16:46:04 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?b?KTogTWVyZ2UgMy42IChpc3N1ZSAjMjg2MzUp?=
Message-ID: <20161128164602.77814.75470.A3A5C454@psf.io>

https://hg.python.org/cpython/rev/48fb6a4cb5f8
changeset:   105383:48fb6a4cb5f8
parent:      105381:07d8272d61e7
parent:      105382:52038705827d
user:        Yury Selivanov <yury at magic.io>
date:        Mon Nov 28 11:45:58 2016 -0500
summary:
  Merge 3.6 (issue #28635)

files:
  Doc/whatsnew/3.6.rst |  34 ++++++++++++++++++++++++++++++++
  1 files changed, 34 insertions(+), 0 deletions(-)


diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -2240,3 +2240,37 @@
 
 * :c:func:`Py_Exit` (and the main interpreter) now override the exit status
   with 120 if flushing buffered data failed.  See :issue:`5319`.
+
+
+CPython bytecode changes
+------------------------
+
+There have been several major changes to the :term:`bytecode` in Python 3.6.
+
+* The Python interpreter now uses a 16-bit wordcode instead of bytecode.
+  (Contributed by Demur Rumed with input and reviews from
+  Serhiy Storchaka and Victor Stinner in :issue:`26647` and :issue:`28050`.)
+
+* The new :opcode:`FORMAT_VALUE` and :opcode:`BUILD_STRING` opcodes as part
+  of the  :ref:`formatted string literal <whatsnew36-pep498>` implementation.
+  (Contributed by Eric Smith in :issue:`25483` and
+  Serhiy Storchaka in :issue:`27078`.)
+
+* The new :opcode:`BUILD_CONST_KEY_MAP` opcode to optimize the creation
+  of dictionaries with constant keys.
+  (Contributed by Serhiy Storchaka in :issue:`27140`.)
+
+* The function call opcodes have been heavily reworked for better performance
+  and simpler implementation.
+  The :opcode:`MAKE_FUNCTION`, :opcode:`CALL_FUNCTION`,
+  :opcode:`CALL_FUNCTION_KW` and :opcode:`BUILD_MAP_UNPACK_WITH_CALL` opcodes
+  have been modified, the new :opcode:`CALL_FUNCTION_EX` and
+  :opcode:`BUILD_TUPLE_UNPACK_WITH_CALL` have been added, and
+  ``CALL_FUNCTION_VAR``, ``CALL_FUNCTION_VAR_KW`` and ``MAKE_CLOSURE`` opcodes
+  have been removed.
+  (Contributed by Demur Rumed in :issue:`27095`, and Serhiy Storchaka in
+  :issue:`27213`, :issue:`28257`.)
+
+* The new :opcode:`SETUP_ANNOTATIONS` and :opcode:`STORE_ANNOTATION` opcodes
+  have been added to support the new :term:`variable annotation` syntax.
+  (Contributed by Ivan Levkivskyi in :issue:`27985`.)

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 28 12:15:59 2016
From: python-checkins at python.org (victor.stinner)
Date: Mon, 28 Nov 2016 17:15:59 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Reintroduce_Python2_suppor?=
 =?utf-8?q?t_in_generate=5Fopcode=5Fh=2Epy?=
Message-ID: <20161128171557.31950.24644.F579E429@psf.io>

https://hg.python.org/cpython/rev/a6ea34c8b413
changeset:   105384:a6ea34c8b413
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Nov 28 18:13:52 2016 +0100
summary:
  Reintroduce Python2 support in generate_opcode_h.py

Issue #28821.

Add also a message to show that the command did something :-)

files:
  Tools/scripts/generate_opcode_h.py |  12 +++++++++---
  1 files changed, 9 insertions(+), 3 deletions(-)


diff --git a/Tools/scripts/generate_opcode_h.py b/Tools/scripts/generate_opcode_h.py
--- a/Tools/scripts/generate_opcode_h.py
+++ b/Tools/scripts/generate_opcode_h.py
@@ -1,6 +1,8 @@
 # This script generates the opcode.h header file.
 
 import sys
+import tokenize
+
 header = """/* Auto-generated by Tools/scripts/generate_opcode_h.py */
 #ifndef Py_OPCODE_H
 #define Py_OPCODE_H
@@ -32,12 +34,14 @@
 #endif /* !Py_OPCODE_H */
 """
 
-import tokenize
-
 
 def main(opcode_py, outfile='Include/opcode.h'):
     opcode = {}
-    with tokenize.open(opcode_py) as fp:
+    if hasattr(tokenize, 'open'):
+        fp = tokenize.open(opcode_py)   # Python 3.2+
+    else:
+        fp = open(opcode_py)            # Python 2.7
+    with fp:
         code = fp.read()
     exec(code, opcode)
     opmap = opcode['opmap']
@@ -51,6 +55,8 @@
                             ('HAVE_ARGUMENT', opcode['HAVE_ARGUMENT']))
         fobj.write(footer)
 
+    print("%s regenerated from %s" % (outfile, opcode_py))
+
 
 if __name__ == '__main__':
     main(sys.argv[1], sys.argv[2])

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 28 12:33:30 2016
From: python-checkins at python.org (victor.stinner)
Date: Mon, 28 Nov 2016 17:33:30 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbjogY2FsbF9mdW5jdGlvbigpOiBk?=
 =?utf-8?q?ocument_PyMethod_optimization?=
Message-ID: <20161128173329.78498.5695.6CC1D071@psf.io>

https://hg.python.org/cpython/rev/c085f2e0e1af
changeset:   105385:c085f2e0e1af
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Nov 28 18:32:31 2016 +0100
summary:
  call_function(): document PyMethod optimization

files:
  Python/ceval.c |  6 +++++-
  1 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -4736,7 +4736,11 @@
     }
     else {
         if (PyMethod_Check(func) && PyMethod_GET_SELF(func) != NULL) {
-            /* optimize access to bound methods */
+            /* Optimize access to bound methods. Reuse the Python stack
+               to pass 'self' as the first argument, replace 'func'
+               with 'self'. It avoids the creation of a new temporary tuple
+               for arguments (to replace func with self) when the method uses
+               FASTCALL. */
             PyObject *self = PyMethod_GET_SELF(func);
             Py_INCREF(self);
             func = PyMethod_GET_FUNCTION(func);

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Mon Nov 28 13:56:51 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 28 Nov 2016 18:56:51 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328823=3A_Simplifi?=
 =?utf-8?q?ed_compiling_with_opcode_BUILD=5FMAP=5FUNPACK=2E?=
Message-ID: <20161128185651.82176.89178.684AAA40@psf.io>

https://hg.python.org/cpython/rev/9ded2433dc2c
changeset:   105386:9ded2433dc2c
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Nov 28 20:56:37 2016 +0200
summary:
  Issue #28823: Simplified compiling with opcode BUILD_MAP_UNPACK.

files:
  Python/compile.c |  7 ++-----
  1 files changed, 2 insertions(+), 5 deletions(-)


diff --git a/Python/compile.c b/Python/compile.c
--- a/Python/compile.c
+++ b/Python/compile.c
@@ -3344,11 +3344,8 @@
     /* If there is more than one dict, they need to be merged into a new
      * dict.  If there is one dict and it's an unpacking, then it needs
      * to be copied into a new dict." */
-    while (containers > 1 || is_unpacking) {
-        int oparg = containers < 255 ? containers : 255;
-        ADDOP_I(c, BUILD_MAP_UNPACK, oparg);
-        containers -= (oparg - 1);
-        is_unpacking = 0;
+    if (containers > 1 || is_unpacking) {
+        ADDOP_I(c, BUILD_MAP_UNPACK, containers);
     }
     return 1;
 }

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 29 02:56:23 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 29 Nov 2016 07:56:23 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogIElzc3VlICMyODc5?=
 =?utf-8?q?7=3A_Modifying_the_class_=5F=5Fdict=5F=5F_inside_the_=5F=5Fset?=
 =?utf-8?q?=5Fname=5F=5F_method_of?=
Message-ID: <20161129075622.30977.80431.A1B94C8A@psf.io>

https://hg.python.org/cpython/rev/6b8f7d1e2ba4
changeset:   105387:6b8f7d1e2ba4
branch:      3.6
parent:      105382:52038705827d
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 29 09:54:17 2016 +0200
summary:
   Issue #28797: Modifying the class __dict__ inside the __set_name__ method of
 a descriptor that is used inside that class no longer prevents calling the
 __set_name__ method of other descriptors.

files:
  Lib/test/test_subclassinit.py |  16 ++++++++++++++++
  Misc/NEWS                     |   4 ++++
  Objects/typeobject.c          |  18 +++++++++++++-----
  3 files changed, 33 insertions(+), 5 deletions(-)


diff --git a/Lib/test/test_subclassinit.py b/Lib/test/test_subclassinit.py
--- a/Lib/test/test_subclassinit.py
+++ b/Lib/test/test_subclassinit.py
@@ -198,6 +198,22 @@
         self.assertIs(B.meta_owner, B)
         self.assertEqual(B.name, 'd')
 
+    def test_set_name_modifying_dict(self):
+        notified = []
+        class Descriptor:
+            def __set_name__(self, owner, name):
+                setattr(owner, name + 'x', None)
+                notified.append(name)
+
+        class A:
+            a = Descriptor()
+            b = Descriptor()
+            c = Descriptor()
+            d = Descriptor()
+            e = Descriptor()
+
+        self.assertCountEqual(notified, ['a', 'b', 'c', 'd', 'e'])
+
     def test_errors(self):
         class MyMeta(type):
             pass
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,10 @@
 Core and Builtins
 -----------------
 
+- Issue #28797: Modifying the class __dict__ inside the __set_name__ method of
+  a descriptor that is used inside that class no longer prevents calling the
+  __set_name__ method of other descriptors.
+
 - Issue #28782: Fix a bug in the implementation ``yield from`` when checking
   if the next instruction is YIELD_FROM. Regression introduced by WORDCODE
   (issue #26647).
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -7004,10 +7004,14 @@
 static int
 set_names(PyTypeObject *type)
 {
-    PyObject *key, *value, *set_name, *tmp;
+    PyObject *names_to_set, *key, *value, *set_name, *tmp;
     Py_ssize_t i = 0;
 
-    while (PyDict_Next(type->tp_dict, &i, &key, &value)) {
+    names_to_set = PyDict_Copy(type->tp_dict);
+    if (names_to_set == NULL)
+        return -1;
+
+    while (PyDict_Next(names_to_set, &i, &key, &value)) {
         set_name = lookup_maybe(value, &PyId___set_name__);
         if (set_name != NULL) {
             tmp = PyObject_CallFunctionObjArgs(set_name, type, key, NULL);
@@ -7017,15 +7021,19 @@
                     "Error calling __set_name__ on '%.100s' instance %R "
                     "in '%.100s'",
                     value->ob_type->tp_name, key, type->tp_name);
+                Py_DECREF(names_to_set);
                 return -1;
             }
             else
                 Py_DECREF(tmp);
         }
-        else if (PyErr_Occurred())
+        else if (PyErr_Occurred()) {
+            Py_DECREF(names_to_set);
             return -1;
-    }
-
+        }
+    }
+
+    Py_DECREF(names_to_set);
     return 0;
 }
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 29 02:56:23 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 29 Nov 2016 07:56:23 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328797=3A_Modifying_the_class_=5F=5Fdict=5F=5F_i?=
 =?utf-8?q?nside_the_=5F=5Fset=5Fname=5F=5F_method_of?=
Message-ID: <20161129075623.108179.9541.EF77EE10@psf.io>

https://hg.python.org/cpython/rev/18ed518d2eef
changeset:   105388:18ed518d2eef
parent:      105386:9ded2433dc2c
parent:      105387:6b8f7d1e2ba4
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 29 09:56:07 2016 +0200
summary:
  Issue #28797: Modifying the class __dict__ inside the __set_name__ method of
 a descriptor that is used inside that class no longer prevents calling the
 __set_name__ method of other descriptors.

files:
  Lib/test/test_subclassinit.py |  16 ++++++++++++++++
  Misc/NEWS                     |   4 ++++
  Objects/typeobject.c          |  18 +++++++++++++-----
  3 files changed, 33 insertions(+), 5 deletions(-)


diff --git a/Lib/test/test_subclassinit.py b/Lib/test/test_subclassinit.py
--- a/Lib/test/test_subclassinit.py
+++ b/Lib/test/test_subclassinit.py
@@ -198,6 +198,22 @@
         self.assertIs(B.meta_owner, B)
         self.assertEqual(B.name, 'd')
 
+    def test_set_name_modifying_dict(self):
+        notified = []
+        class Descriptor:
+            def __set_name__(self, owner, name):
+                setattr(owner, name + 'x', None)
+                notified.append(name)
+
+        class A:
+            a = Descriptor()
+            b = Descriptor()
+            c = Descriptor()
+            d = Descriptor()
+            e = Descriptor()
+
+        self.assertCountEqual(notified, ['a', 'b', 'c', 'd', 'e'])
+
     def test_errors(self):
         class MyMeta(type):
             pass
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,10 @@
 Core and Builtins
 -----------------
 
+- Issue #28797: Modifying the class __dict__ inside the __set_name__ method of
+  a descriptor that is used inside that class no longer prevents calling the
+  __set_name__ method of other descriptors.
+
 - Issue #28799: Remove the ``PyEval_GetCallStats()`` function and deprecate
   the untested and undocumented ``sys.callstats()`` function. Remove the
   ``CALL_PROFILE`` special build: use the :func:`sys.setprofile` function,
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -7004,10 +7004,14 @@
 static int
 set_names(PyTypeObject *type)
 {
-    PyObject *key, *value, *set_name, *tmp;
+    PyObject *names_to_set, *key, *value, *set_name, *tmp;
     Py_ssize_t i = 0;
 
-    while (PyDict_Next(type->tp_dict, &i, &key, &value)) {
+    names_to_set = PyDict_Copy(type->tp_dict);
+    if (names_to_set == NULL)
+        return -1;
+
+    while (PyDict_Next(names_to_set, &i, &key, &value)) {
         set_name = lookup_maybe(value, &PyId___set_name__);
         if (set_name != NULL) {
             tmp = PyObject_CallFunctionObjArgs(set_name, type, key, NULL);
@@ -7017,15 +7021,19 @@
                     "Error calling __set_name__ on '%.100s' instance %R "
                     "in '%.100s'",
                     value->ob_type->tp_name, key, type->tp_name);
+                Py_DECREF(names_to_set);
                 return -1;
             }
             else
                 Py_DECREF(tmp);
         }
-        else if (PyErr_Occurred())
+        else if (PyErr_Occurred()) {
+            Py_DECREF(names_to_set);
             return -1;
-    }
-
+        }
+    }
+
+    Py_DECREF(names_to_set);
     return 0;
 }
 

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Tue Nov 29 04:05:20 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Tue, 29 Nov 2016 09:05:20 +0000
Subject: [Python-checkins] Daily reference leaks (9ded2433dc2c): sum=12
Message-ID: <20161129090519.18493.70326.82D93699@psf.io>

results for 9ded2433dc2c on branch "default"
--------------------------------------------

test_collections leaked [7, -7, 8] memory blocks, sum=8
test_functools leaked [0, 3, 1] memory blocks, sum=4


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogbAVG8X', '--timeout', '7200']

From lp_benchmark_robot at intel.com  Tue Nov 29 09:59:43 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Tue, 29 Nov 2016 14:59:43 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python 2.7 2016-11-29
Message-ID: <c2cf794b-6fc7-4395-8567-55956b10bc52@irsmsx105.ger.corp.intel.com>

No new revisions. Here are the previous results:

Results for project Python 2.7, build date 2016-11-29 07:56:58 +0000
commit:		59bd48afa1bc
previous commit:	6eefdf0fade6
revision date:	2016-11-26 11:43:39 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc
from 2015-05-23 16:02:14+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.14%          1.76%          5.07%             8.38%
:-)             pybench      0.24%         -0.01%          5.70%             4.55%
:-(            regex_v8      0.57%          0.06%         -2.26%            10.27%
:-)               nbody      0.85%          0.01%          7.90%             2.67%
:-)        json_dump_v2      0.34%         -0.26%          3.54%            10.10%
:-|      normal_startup      1.12%          0.06%         -0.24%             2.36%
:-)             ssbench      0.20%         -0.56%          2.06%             1.80%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-2-7-2016-11-29/

Note: Benchmark results for ssbench are measured in requests/second while all
other are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From lp_benchmark_robot at intel.com  Tue Nov 29 10:00:35 2016
From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com)
Date: Tue, 29 Nov 2016 15:00:35 +0000
Subject: [Python-checkins] GOOD Benchmark Results for Python Default
 2016-11-29
Message-ID: <76ca4aab-b8ba-4771-bcc7-80a87377e435@irsmsx105.ger.corp.intel.com>

Results for project Python default, build date 2016-11-29 03:02:18 +0000
commit:		9ded2433dc2c
previous commit:	5c1bb72c0f5d
revision date:	2016-11-28 18:56:37 +0000
environment:	Haswell-EP
	cpu:		Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
	mem:		128 GB
	os:		CentOS 7.1
	kernel:	Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release v3.4.3, with hash b4cbecbc0781
from 2015-02-25 12:15:33+00:00

----------------------------------------------------------------------------------
              benchmark   relative   change since   change since   current rev run
                          std_dev*       last run       baseline          with PGO
----------------------------------------------------------------------------------
:-)           django_v2      0.20%         -0.82%          5.16%            13.76%
:-)             pybench      0.26%          0.14%          5.61%             5.33%
:-(            regex_v8      3.77%         -0.13%         -3.12%             3.26%
:-)               nbody      0.10%          2.07%          1.46%             4.56%
:-(        json_dump_v2      0.29%          1.46%         -2.82%             8.44%
:-|      normal_startup      0.54%         -0.20%          0.81%             6.86%
----------------------------------------------------------------------------------

* Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-11-29/

Note: Benchmark results are measured in seconds.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload


Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

From python-checkins at python.org  Tue Nov 29 10:55:27 2016
From: python-checkins at python.org (victor.stinner)
Date: Tue, 29 Nov 2016 15:55:27 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_TCP=5FCONGESTION_and_T?=
 =?utf-8?q?CP=5FUSER=5FTIMEOUT?=
Message-ID: <20161129155527.18337.57724.C1CDE226@psf.io>

https://hg.python.org/cpython/rev/674fb9644eaa
changeset:   105389:674fb9644eaa
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 29 16:55:04 2016 +0100
summary:
  Add TCP_CONGESTION and TCP_USER_TIMEOUT

Issue #26273: Add new socket.TCP_CONGESTION (Linux 2.6.13) and
socket.TCP_USER_TIMEOUT (Linux 2.6.37) constants.

Patch written by Omar Sandoval.

files:
  Misc/NEWS              |  4 ++++
  Modules/socketmodule.c |  6 ++++++
  2 files changed, 10 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -160,6 +160,10 @@
 Library
 -------
 
+- Issue #26273: Add new :data:`socket.TCP_CONGESTION` (Linux 2.6.13) and
+  :data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37) constants. Patch written by
+  Omar Sandoval.
+
 - Issue #28752: Restored the __reduce__() methods of datetime objects.
 
 - Issue #28727: Regular expression patterns, _sre.SRE_Pattern objects created
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -7540,6 +7540,12 @@
 #ifdef  TCP_FASTOPEN
     PyModule_AddIntMacro(m, TCP_FASTOPEN);
 #endif
+#ifdef  TCP_CONGESTION
+    PyModule_AddIntMacro(m, TCP_CONGESTION);
+#endif
+#ifdef  TCP_USER_TIMEOUT
+    PyModule_AddIntMacro(m, TCP_USER_TIMEOUT);
+#endif
 
     /* IPX options */
 #ifdef  IPX_TYPE

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 29 12:22:51 2016
From: python-checkins at python.org (victor.stinner)
Date: Tue, 29 Nov 2016 17:22:51 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Null_merge_3=2E6?=
Message-ID: <20161129172248.39792.70608.7C01C789@psf.io>

https://hg.python.org/cpython/rev/c9a348edf8d2
changeset:   105391:c9a348edf8d2
parent:      105389:674fb9644eaa
parent:      105390:6d69da76be6a
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 29 18:22:02 2016 +0100
summary:
  Null merge 3.6

files:



-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 29 12:22:51 2016
From: python-checkins at python.org (victor.stinner)
Date: Tue, 29 Nov 2016 17:22:51 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy42KTogQWRkIFRDUF9DT05H?=
 =?utf-8?q?ESTION_and_TCP=5FUSER=5FTIMEOUT?=
Message-ID: <20161129172247.108528.36529.D194EC01@psf.io>

https://hg.python.org/cpython/rev/6d69da76be6a
changeset:   105390:6d69da76be6a
branch:      3.6
parent:      105387:6b8f7d1e2ba4
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 29 16:55:04 2016 +0100
summary:
  Add TCP_CONGESTION and TCP_USER_TIMEOUT

Issue #26273: Add new socket.TCP_CONGESTION (Linux 2.6.13) and
socket.TCP_USER_TIMEOUT (Linux 2.6.37) constants.

Patch written by Omar Sandoval.

files:
  Misc/NEWS              |  4 ++++
  Modules/socketmodule.c |  6 ++++++
  2 files changed, 10 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -74,6 +74,10 @@
 Library
 -------
 
+- Issue #26273: Add new :data:`socket.TCP_CONGESTION` (Linux 2.6.13) and
+  :data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37) constants. Patch written by
+  Omar Sandoval.
+
 - Issue #28752: Restored the __reduce__() methods of datetime objects.
 
 - Issue #28727: Regular expression patterns, _sre.SRE_Pattern objects created
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -7512,6 +7512,12 @@
 #ifdef  TCP_FASTOPEN
     PyModule_AddIntMacro(m, TCP_FASTOPEN);
 #endif
+#ifdef  TCP_CONGESTION
+    PyModule_AddIntMacro(m, TCP_CONGESTION);
+#endif
+#ifdef  TCP_USER_TIMEOUT
+    PyModule_AddIntMacro(m, TCP_USER_TIMEOUT);
+#endif
 
     /* IPX options */
 #ifdef  IPX_TYPE

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 29 12:47:00 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Tue, 29 Nov 2016 17:47:00 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4Nzkw?=
 =?utf-8?q?=3A_Fix_error_when_using_Generic_and_=5F=5Fslots=5F=5F_=28Ivan_?=
 =?utf-8?q?L=29?=
Message-ID: <20161129174658.30574.24127.8F6FA9D0@psf.io>

https://hg.python.org/cpython/rev/0bbd29405c9d
changeset:   105392:0bbd29405c9d
branch:      3.5
parent:      105374:40567b8e3478
user:        Guido van Rossum <guido at python.org>
date:        Tue Nov 29 09:46:21 2016 -0800
summary:
  Issue #28790: Fix error when using Generic and __slots__ (Ivan L)

files:
  Lib/test/test_typing.py |  38 +++++++++++++++++++++++++++++
  Lib/typing.py           |  18 +++++++++++--
  2 files changed, 53 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -896,6 +896,44 @@
             self.assertEqual(t, copy(t))
             self.assertEqual(t, deepcopy(t))
 
+    def test_parameterized_slots(self):
+        T = TypeVar('T')
+        class C(Generic[T]):
+            __slots__ = ('potato',)
+
+        c = C()
+        c_int = C[int]()
+        self.assertEqual(C.__slots__, C[str].__slots__)
+
+        c.potato = 0
+        c_int.potato = 0
+        with self.assertRaises(AttributeError):
+            c.tomato = 0
+        with self.assertRaises(AttributeError):
+            c_int.tomato = 0
+
+        def foo(x: C['C']): ...
+        self.assertEqual(get_type_hints(foo, globals(), locals())['x'], C[C])
+        self.assertEqual(get_type_hints(foo, globals(), locals())['x'].__slots__,
+                         C.__slots__)
+        self.assertEqual(copy(C[int]), deepcopy(C[int]))
+
+    def test_parameterized_slots_dict(self):
+        T = TypeVar('T')
+        class D(Generic[T]):
+            __slots__ = {'banana': 42}
+
+        d = D()
+        d_int = D[int]()
+        self.assertEqual(D.__slots__, D[str].__slots__)
+
+        d.banana = 'yes'
+        d_int.banana = 'yes'
+        with self.assertRaises(AttributeError):
+            d.foobar = 'no'
+        with self.assertRaises(AttributeError):
+            d_int.foobar = 'no'
+
     def test_errors(self):
         with self.assertRaises(TypeError):
             B = SimpleMapping[XK, Any]
diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -870,6 +870,17 @@
     return __extrahook__
 
 
+def _no_slots_copy(dct):
+    """Internal helper: copy class __dict__ and clean slots class variables.
+    (They will be re-created if necessary by normal class machinery.)
+    """
+    dict_copy = dict(dct)
+    if '__slots__' in dict_copy:
+        for slot in dict_copy['__slots__']:
+            dict_copy.pop(slot, None)
+    return dict_copy
+
+
 class GenericMeta(TypingMeta, abc.ABCMeta):
     """Metaclass for generic types."""
 
@@ -967,7 +978,7 @@
             return self
         return self.__class__(self.__name__,
                               self.__bases__,
-                              dict(self.__dict__),
+                              _no_slots_copy(self.__dict__),
                               tvars=_type_vars(ev_args) if ev_args else None,
                               args=ev_args,
                               origin=ev_origin,
@@ -1043,7 +1054,7 @@
             args = params
         return self.__class__(self.__name__,
                               self.__bases__,
-                              dict(self.__dict__),
+                              _no_slots_copy(self.__dict__),
                               tvars=tvars,
                               args=args,
                               origin=self,
@@ -1059,7 +1070,8 @@
         return issubclass(instance.__class__, self)
 
     def __copy__(self):
-        return self.__class__(self.__name__, self.__bases__, dict(self.__dict__),
+        return self.__class__(self.__name__, self.__bases__,
+                              _no_slots_copy(self.__dict__),
                               self.__parameters__, self.__args__, self.__origin__,
                               self.__extra__, self.__orig_bases__)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 29 12:47:00 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Tue, 29 Nov 2016 17:47:00 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNik6?=
 =?utf-8?q?_Issue_=2328790=3A_Fix_error_when_using_Generic_and_=5F=5Fslots?=
 =?utf-8?b?X18gKEl2YW4gTCkgKDMuNS0+My42KQ==?=
Message-ID: <20161129174658.39469.70547.BBB9CD9A@psf.io>

https://hg.python.org/cpython/rev/2dd08b5b5ee6
changeset:   105393:2dd08b5b5ee6
branch:      3.6
parent:      105390:6d69da76be6a
parent:      105392:0bbd29405c9d
user:        Guido van Rossum <guido at python.org>
date:        Tue Nov 29 09:46:26 2016 -0800
summary:
  Issue #28790: Fix error when using Generic and __slots__ (Ivan L) (3.5->3.6)

files:
  Lib/test/test_typing.py |  38 +++++++++++++++++++++++++++++
  Lib/typing.py           |  18 +++++++++++--
  2 files changed, 53 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -896,6 +896,44 @@
             self.assertEqual(t, copy(t))
             self.assertEqual(t, deepcopy(t))
 
+    def test_parameterized_slots(self):
+        T = TypeVar('T')
+        class C(Generic[T]):
+            __slots__ = ('potato',)
+
+        c = C()
+        c_int = C[int]()
+        self.assertEqual(C.__slots__, C[str].__slots__)
+
+        c.potato = 0
+        c_int.potato = 0
+        with self.assertRaises(AttributeError):
+            c.tomato = 0
+        with self.assertRaises(AttributeError):
+            c_int.tomato = 0
+
+        def foo(x: C['C']): ...
+        self.assertEqual(get_type_hints(foo, globals(), locals())['x'], C[C])
+        self.assertEqual(get_type_hints(foo, globals(), locals())['x'].__slots__,
+                         C.__slots__)
+        self.assertEqual(copy(C[int]), deepcopy(C[int]))
+
+    def test_parameterized_slots_dict(self):
+        T = TypeVar('T')
+        class D(Generic[T]):
+            __slots__ = {'banana': 42}
+
+        d = D()
+        d_int = D[int]()
+        self.assertEqual(D.__slots__, D[str].__slots__)
+
+        d.banana = 'yes'
+        d_int.banana = 'yes'
+        with self.assertRaises(AttributeError):
+            d.foobar = 'no'
+        with self.assertRaises(AttributeError):
+            d_int.foobar = 'no'
+
     def test_errors(self):
         with self.assertRaises(TypeError):
             B = SimpleMapping[XK, Any]
diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -870,6 +870,17 @@
     return __extrahook__
 
 
+def _no_slots_copy(dct):
+    """Internal helper: copy class __dict__ and clean slots class variables.
+    (They will be re-created if necessary by normal class machinery.)
+    """
+    dict_copy = dict(dct)
+    if '__slots__' in dict_copy:
+        for slot in dict_copy['__slots__']:
+            dict_copy.pop(slot, None)
+    return dict_copy
+
+
 class GenericMeta(TypingMeta, abc.ABCMeta):
     """Metaclass for generic types."""
 
@@ -967,7 +978,7 @@
             return self
         return self.__class__(self.__name__,
                               self.__bases__,
-                              dict(self.__dict__),
+                              _no_slots_copy(self.__dict__),
                               tvars=_type_vars(ev_args) if ev_args else None,
                               args=ev_args,
                               origin=ev_origin,
@@ -1043,7 +1054,7 @@
             args = params
         return self.__class__(self.__name__,
                               self.__bases__,
-                              dict(self.__dict__),
+                              _no_slots_copy(self.__dict__),
                               tvars=tvars,
                               args=args,
                               origin=self,
@@ -1059,7 +1070,8 @@
         return issubclass(instance.__class__, self)
 
     def __copy__(self):
-        return self.__class__(self.__name__, self.__bases__, dict(self.__dict__),
+        return self.__class__(self.__name__, self.__bases__,
+                              _no_slots_copy(self.__dict__),
                               self.__parameters__, self.__args__, self.__origin__,
                               self.__extra__, self.__orig_bases__)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 29 12:47:06 2016
From: python-checkins at python.org (guido.van.rossum)
Date: Tue, 29 Nov 2016 17:47:06 +0000
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E6_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2328790=3A_Fix_error_when_using_Generic_and_=5F?=
 =?utf-8?b?X3Nsb3RzX18gKEl2YW4gTCkgKDMuNi0+My43KQ==?=
Message-ID: <20161129174659.9540.50135.58BB104A@psf.io>

https://hg.python.org/cpython/rev/b9915ca4b3da
changeset:   105394:b9915ca4b3da
parent:      105391:c9a348edf8d2
parent:      105393:2dd08b5b5ee6
user:        Guido van Rossum <guido at python.org>
date:        Tue Nov 29 09:46:29 2016 -0800
summary:
  Issue #28790: Fix error when using Generic and __slots__ (Ivan L) (3.6->3.7)

files:
  Lib/test/test_typing.py |  38 +++++++++++++++++++++++++++++
  Lib/typing.py           |  18 +++++++++++--
  2 files changed, 53 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py
--- a/Lib/test/test_typing.py
+++ b/Lib/test/test_typing.py
@@ -896,6 +896,44 @@
             self.assertEqual(t, copy(t))
             self.assertEqual(t, deepcopy(t))
 
+    def test_parameterized_slots(self):
+        T = TypeVar('T')
+        class C(Generic[T]):
+            __slots__ = ('potato',)
+
+        c = C()
+        c_int = C[int]()
+        self.assertEqual(C.__slots__, C[str].__slots__)
+
+        c.potato = 0
+        c_int.potato = 0
+        with self.assertRaises(AttributeError):
+            c.tomato = 0
+        with self.assertRaises(AttributeError):
+            c_int.tomato = 0
+
+        def foo(x: C['C']): ...
+        self.assertEqual(get_type_hints(foo, globals(), locals())['x'], C[C])
+        self.assertEqual(get_type_hints(foo, globals(), locals())['x'].__slots__,
+                         C.__slots__)
+        self.assertEqual(copy(C[int]), deepcopy(C[int]))
+
+    def test_parameterized_slots_dict(self):
+        T = TypeVar('T')
+        class D(Generic[T]):
+            __slots__ = {'banana': 42}
+
+        d = D()
+        d_int = D[int]()
+        self.assertEqual(D.__slots__, D[str].__slots__)
+
+        d.banana = 'yes'
+        d_int.banana = 'yes'
+        with self.assertRaises(AttributeError):
+            d.foobar = 'no'
+        with self.assertRaises(AttributeError):
+            d_int.foobar = 'no'
+
     def test_errors(self):
         with self.assertRaises(TypeError):
             B = SimpleMapping[XK, Any]
diff --git a/Lib/typing.py b/Lib/typing.py
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -870,6 +870,17 @@
     return __extrahook__
 
 
+def _no_slots_copy(dct):
+    """Internal helper: copy class __dict__ and clean slots class variables.
+    (They will be re-created if necessary by normal class machinery.)
+    """
+    dict_copy = dict(dct)
+    if '__slots__' in dict_copy:
+        for slot in dict_copy['__slots__']:
+            dict_copy.pop(slot, None)
+    return dict_copy
+
+
 class GenericMeta(TypingMeta, abc.ABCMeta):
     """Metaclass for generic types."""
 
@@ -967,7 +978,7 @@
             return self
         return self.__class__(self.__name__,
                               self.__bases__,
-                              dict(self.__dict__),
+                              _no_slots_copy(self.__dict__),
                               tvars=_type_vars(ev_args) if ev_args else None,
                               args=ev_args,
                               origin=ev_origin,
@@ -1043,7 +1054,7 @@
             args = params
         return self.__class__(self.__name__,
                               self.__bases__,
-                              dict(self.__dict__),
+                              _no_slots_copy(self.__dict__),
                               tvars=tvars,
                               args=args,
                               origin=self,
@@ -1059,7 +1070,8 @@
         return issubclass(instance.__class__, self)
 
     def __copy__(self):
-        return self.__class__(self.__name__, self.__bases__, dict(self.__dict__),
+        return self.__class__(self.__name__, self.__bases__,
+                              _no_slots_copy(self.__dict__),
                               self.__parameters__, self.__args__, self.__origin__,
                               self.__extra__, self.__orig_bases__)
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 29 12:58:39 2016
From: python-checkins at python.org (victor.stinner)
Date: Tue, 29 Nov 2016 17:58:39 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Uniformize_argument_names_?=
 =?utf-8?q?of_=22call=22_functions?=
Message-ID: <20161129175839.27752.62358.C471A8E6@psf.io>

https://hg.python.org/cpython/rev/7efddbf1aa70
changeset:   105395:7efddbf1aa70
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Nov 29 18:47:56 2016 +0100
summary:
  Uniformize argument names of "call" functions

* Callable object: callable, o, callable_object => func
* Object for method calls: o => obj
* Method name: name or nameid => method

Cleanup also the C code:

* Don't initialize variables to NULL if they are not used before their first
  assignement
* Add braces for readability

files:
  Include/abstract.h   |   40 ++--
  Include/ceval.h      |    4 +-
  Objects/abstract.c   |  236 +++++++++++++++---------------
  Objects/typeobject.c |   21 +-
  4 files changed, 152 insertions(+), 149 deletions(-)


diff --git a/Include/abstract.h b/Include/abstract.h
--- a/Include/abstract.h
+++ b/Include/abstract.h
@@ -257,15 +257,15 @@
 
        /* Declared elsewhere
 
-     PyAPI_FUNC(int) PyCallable_Check(PyObject *o);
+     PyAPI_FUNC(int) PyCallable_Check(PyObject *obj);
 
-     Determine if the object, o, is callable.  Return 1 if the
+     Determine if the object, obj, is callable.  Return 1 if the
      object is callable and 0 otherwise.
 
      This function always succeeds.
        */
 
-     PyAPI_FUNC(PyObject *) PyObject_Call(PyObject *callable_object,
+     PyAPI_FUNC(PyObject *) PyObject_Call(PyObject *func,
                                           PyObject *args, PyObject *kwargs);
 
 #ifndef Py_LIMITED_API
@@ -344,7 +344,7 @@
     _PyObject_FastCall((func), &(arg), 1)
 
     PyAPI_FUNC(PyObject *) _PyObject_Call_Prepend(PyObject *func,
-                                                  PyObject *obj, PyObject *args,
+                                                  PyObject *arg0, PyObject *args,
                                                   PyObject *kwargs);
 
      PyAPI_FUNC(PyObject *) _Py_CheckFunctionResult(PyObject *func,
@@ -353,27 +353,27 @@
 #endif   /* Py_LIMITED_API */
 
        /*
-     Call a callable Python object, callable_object, with
+     Call a callable Python object, func, with
      arguments and keywords arguments.  The 'args' argument can not be
      NULL.
        */
 
-     PyAPI_FUNC(PyObject *) PyObject_CallObject(PyObject *callable_object,
+     PyAPI_FUNC(PyObject *) PyObject_CallObject(PyObject *func,
                                                 PyObject *args);
 
        /*
-     Call a callable Python object, callable_object, with
+     Call a callable Python object, func, with
      arguments given by the tuple, args.  If no arguments are
      needed, then args may be NULL.  Returns the result of the
      call on success, or NULL on failure.  This is the equivalent
      of the Python expression: o(*args).
        */
 
-     PyAPI_FUNC(PyObject *) PyObject_CallFunction(PyObject *callable_object,
+     PyAPI_FUNC(PyObject *) PyObject_CallFunction(PyObject *func,
                                                   const char *format, ...);
 
        /*
-     Call a callable Python object, callable_object, with a
+     Call a callable Python object, func, with a
      variable number of C arguments. The C arguments are described
      using a mkvalue-style format string. The format may be NULL,
      indicating that no arguments are provided.  Returns the
@@ -382,7 +382,7 @@
        */
 
 
-     PyAPI_FUNC(PyObject *) PyObject_CallMethod(PyObject *o,
+     PyAPI_FUNC(PyObject *) PyObject_CallMethod(PyObject *obj,
                                                 const char *method,
                                                 const char *format, ...);
 
@@ -396,7 +396,7 @@
        */
 
 #ifndef Py_LIMITED_API
-     PyAPI_FUNC(PyObject *) _PyObject_CallMethodId(PyObject *o,
+     PyAPI_FUNC(PyObject *) _PyObject_CallMethodId(PyObject *obj,
                                                    _Py_Identifier *method,
                                                    const char *format, ...);
 
@@ -406,25 +406,25 @@
        */
 #endif /* !Py_LIMITED_API */
 
-     PyAPI_FUNC(PyObject *) _PyObject_CallFunction_SizeT(PyObject *callable,
+     PyAPI_FUNC(PyObject *) _PyObject_CallFunction_SizeT(PyObject *func,
                                                          const char *format,
                                                          ...);
-     PyAPI_FUNC(PyObject *) _PyObject_CallMethod_SizeT(PyObject *o,
-                                                       const char *name,
+     PyAPI_FUNC(PyObject *) _PyObject_CallMethod_SizeT(PyObject *obj,
+                                                       const char *method,
                                                        const char *format,
                                                        ...);
 #ifndef Py_LIMITED_API
-     PyAPI_FUNC(PyObject *) _PyObject_CallMethodId_SizeT(PyObject *o,
-                                                       _Py_Identifier *name,
+     PyAPI_FUNC(PyObject *) _PyObject_CallMethodId_SizeT(PyObject *obj,
+                                                       _Py_Identifier *method,
                                                        const char *format,
                                                        ...);
 #endif /* !Py_LIMITED_API */
 
-     PyAPI_FUNC(PyObject *) PyObject_CallFunctionObjArgs(PyObject *callable,
+     PyAPI_FUNC(PyObject *) PyObject_CallFunctionObjArgs(PyObject *func,
                                                          ...);
 
        /*
-     Call a callable Python object, callable_object, with a
+     Call a callable Python object, func, with a
      variable number of C arguments.  The C arguments are provided
      as PyObject * values, terminated by a NULL.  Returns the
      result of the call on success, or NULL on failure.  This is
@@ -432,10 +432,10 @@
        */
 
 
-     PyAPI_FUNC(PyObject *) PyObject_CallMethodObjArgs(PyObject *o,
+     PyAPI_FUNC(PyObject *) PyObject_CallMethodObjArgs(PyObject *obj,
                                                        PyObject *method, ...);
 #ifndef Py_LIMITED_API
-     PyAPI_FUNC(PyObject *) _PyObject_CallMethodIdObjArgs(PyObject *o,
+     PyAPI_FUNC(PyObject *) _PyObject_CallMethodIdObjArgs(PyObject *obj,
                                                struct _Py_Identifier *method,
                                                ...);
 #endif /* !Py_LIMITED_API */
diff --git a/Include/ceval.h b/Include/ceval.h
--- a/Include/ceval.h
+++ b/Include/ceval.h
@@ -14,10 +14,10 @@
 #define PyEval_CallObject(func,arg) \
     PyEval_CallObjectWithKeywords(func, arg, (PyObject *)NULL)
 
-PyAPI_FUNC(PyObject *) PyEval_CallFunction(PyObject *obj,
+PyAPI_FUNC(PyObject *) PyEval_CallFunction(PyObject *func,
                                            const char *format, ...);
 PyAPI_FUNC(PyObject *) PyEval_CallMethod(PyObject *obj,
-                                         const char *methodname,
+                                         const char *method,
                                          const char *format, ...);
 
 #ifndef Py_LIMITED_API
diff --git a/Objects/abstract.c b/Objects/abstract.c
--- a/Objects/abstract.c
+++ b/Objects/abstract.c
@@ -2173,9 +2173,9 @@
 /* XXX PyCallable_Check() is in object.c */
 
 PyObject *
-PyObject_CallObject(PyObject *o, PyObject *a)
+PyObject_CallObject(PyObject *func, PyObject *args)
 {
-    return PyEval_CallObjectWithKeywords(o, a, NULL);
+    return PyEval_CallObjectWithKeywords(func, args, NULL);
 }
 
 PyObject*
@@ -2331,10 +2331,10 @@
     return result;
 }
 
-/* Positional arguments are obj followed args. */
+/* Positional arguments are arg0 followed args: [arg0, *args]. */
 PyObject *
 _PyObject_Call_Prepend(PyObject *func,
-                       PyObject *obj, PyObject *args, PyObject *kwargs)
+                       PyObject *arg0, PyObject *args, PyObject *kwargs)
 {
     PyObject *small_stack[8];
     PyObject **stack;
@@ -2356,7 +2356,7 @@
     }
 
     /* use borrowed references */
-    stack[0] = obj;
+    stack[0] = arg0;
     memcpy(&stack[1],
               &PyTuple_GET_ITEM(args, 0),
               argcount * sizeof(PyObject *));
@@ -2489,34 +2489,34 @@
 }
 
 static PyObject*
-call_function_tail(PyObject *callable, PyObject *args)
+call_function_tail(PyObject *func, PyObject *args)
 {
     PyObject *result;
 
     assert(args != NULL);
 
     if (!PyTuple_Check(args)) {
-        result = _PyObject_CallArg1(callable, args);
+        result = _PyObject_CallArg1(func, args);
     }
     else {
-        result = PyObject_Call(callable, args, NULL);
+        result = PyObject_Call(func, args, NULL);
     }
 
     return result;
 }
 
 PyObject *
-PyObject_CallFunction(PyObject *callable, const char *format, ...)
+PyObject_CallFunction(PyObject *func, const char *format, ...)
 {
     va_list va;
     PyObject *args, *result;
 
-    if (callable == NULL) {
+    if (func == NULL) {
         return null_error();
     }
 
     if (!format || !*format) {
-        return _PyObject_CallNoArg(callable);
+        return _PyObject_CallNoArg(func);
     }
 
     va_start(va, format);
@@ -2526,23 +2526,23 @@
         return NULL;
     }
 
-    result = call_function_tail(callable, args);
+    result = call_function_tail(func, args);
     Py_DECREF(args);
     return result;
 }
 
 PyObject *
-_PyObject_CallFunction_SizeT(PyObject *callable, const char *format, ...)
+_PyObject_CallFunction_SizeT(PyObject *func, const char *format, ...)
 {
     va_list va;
     PyObject *args, *result;
 
-    if (callable == NULL) {
+    if (func == NULL) {
         return null_error();
     }
 
     if (!format || !*format) {
-        return _PyObject_CallNoArg(callable);
+        return _PyObject_CallNoArg(func);
     }
 
     va_start(va, format);
@@ -2552,7 +2552,7 @@
         return NULL;
     }
 
-    result = call_function_tail(callable, args);
+    result = call_function_tail(func, args);
     Py_DECREF(args);
     return result;
 }
@@ -2589,19 +2589,20 @@
 }
 
 PyObject *
-PyObject_CallMethod(PyObject *o, const char *name, const char *format, ...)
+PyObject_CallMethod(PyObject *obj, const char *method, const char *format, ...)
 {
     va_list va;
     PyObject *func = NULL;
     PyObject *retval = NULL;
 
-    if (o == NULL || name == NULL) {
+    if (obj == NULL || method == NULL) {
         return null_error();
     }
 
-    func = PyObject_GetAttrString(o, name);
-    if (func == NULL)
+    func = PyObject_GetAttrString(obj, method);
+    if (func == NULL) {
         return NULL;
+    }
 
     va_start(va, format);
     retval = callmethod(func, format, va, 0);
@@ -2611,20 +2612,20 @@
 }
 
 PyObject *
-_PyObject_CallMethodId(PyObject *o, _Py_Identifier *name,
+_PyObject_CallMethodId(PyObject *obj, _Py_Identifier *method,
                        const char *format, ...)
 {
     va_list va;
-    PyObject *func = NULL;
-    PyObject *retval = NULL;
-
-    if (o == NULL || name == NULL) {
+    PyObject *func, *retval;
+
+    if (obj == NULL || method == NULL) {
         return null_error();
     }
 
-    func = _PyObject_GetAttrId(o, name);
-    if (func == NULL)
+    func = _PyObject_GetAttrId(obj, method);
+    if (func == NULL) {
         return NULL;
+    }
 
     va_start(va, format);
     retval = callmethod(func, format, va, 0);
@@ -2634,20 +2635,21 @@
 }
 
 PyObject *
-_PyObject_CallMethod_SizeT(PyObject *o, const char *name,
+_PyObject_CallMethod_SizeT(PyObject *obj, const char *method,
                            const char *format, ...)
 {
     va_list va;
-    PyObject *func = NULL;
-    PyObject *retval;
-
-    if (o == NULL || name == NULL) {
+    PyObject *func, *retval;
+
+    if (obj == NULL || method == NULL) {
         return null_error();
     }
 
-    func = PyObject_GetAttrString(o, name);
-    if (func == NULL)
+    func = PyObject_GetAttrString(obj, method);
+    if (func == NULL) {
         return NULL;
+    }
+
     va_start(va, format);
     retval = callmethod(func, format, va, 1);
     va_end(va);
@@ -2656,21 +2658,21 @@
 }
 
 PyObject *
-_PyObject_CallMethodId_SizeT(PyObject *o, _Py_Identifier *name,
+_PyObject_CallMethodId_SizeT(PyObject *obj, _Py_Identifier *method,
                              const char *format, ...)
 {
     va_list va;
-    PyObject *func = NULL;
-    PyObject *retval;
-
-    if (o == NULL || name == NULL) {
+    PyObject *func, *retval;
+
+    if (obj == NULL || method == NULL) {
         return null_error();
     }
 
-    func = _PyObject_GetAttrId(o, name);
+    func = _PyObject_GetAttrId(obj, method);
     if (func == NULL) {
         return NULL;
     }
+
     va_start(va, format);
     retval = callmethod(func, format, va, 1);
     va_end(va);
@@ -2720,7 +2722,80 @@
 }
 
 PyObject *
-PyObject_CallMethodObjArgs(PyObject *callable, PyObject *name, ...)
+PyObject_CallMethodObjArgs(PyObject *obj, PyObject *method, ...)
+{
+    PyObject *small_stack[5];
+    PyObject **stack;
+    Py_ssize_t nargs;
+    PyObject *func, *result;
+    va_list vargs;
+
+    if (obj == NULL || method == NULL) {
+        return null_error();
+    }
+
+    func = PyObject_GetAttr(obj, method);
+    if (func == NULL)
+        return NULL;
+
+    /* count the args */
+    va_start(vargs, method);
+    stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack),
+                            vargs, &nargs);
+    va_end(vargs);
+    if (stack == NULL) {
+        Py_DECREF(func);
+        return NULL;
+    }
+
+    result = _PyObject_FastCall(func, stack, nargs);
+    Py_DECREF(func);
+    if (stack != small_stack) {
+        PyMem_Free(stack);
+    }
+
+    return result;
+}
+
+PyObject *
+_PyObject_CallMethodIdObjArgs(PyObject *obj,
+                              struct _Py_Identifier *method, ...)
+{
+    PyObject *small_stack[5];
+    PyObject **stack;
+    Py_ssize_t nargs;
+    PyObject *func, *result;
+    va_list vargs;
+
+    if (obj == NULL || method == NULL) {
+        return null_error();
+    }
+
+    func = _PyObject_GetAttrId(obj, method);
+    if (func == NULL)
+        return NULL;
+
+    /* count the args */
+    va_start(vargs, method);
+    stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack),
+                            vargs, &nargs);
+    va_end(vargs);
+    if (stack == NULL) {
+        Py_DECREF(func);
+        return NULL;
+    }
+
+    result = _PyObject_FastCall(func, stack, nargs);
+    Py_DECREF(func);
+    if (stack != small_stack) {
+        PyMem_Free(stack);
+    }
+
+    return result;
+}
+
+PyObject *
+PyObject_CallFunctionObjArgs(PyObject *func, ...)
 {
     PyObject *small_stack[5];
     PyObject **stack;
@@ -2728,85 +2803,12 @@
     PyObject *result;
     va_list vargs;
 
-    if (callable == NULL || name == NULL) {
+    if (func == NULL) {
         return null_error();
     }
 
-    callable = PyObject_GetAttr(callable, name);
-    if (callable == NULL)
-        return NULL;
-
     /* count the args */
-    va_start(vargs, name);
-    stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack),
-                            vargs, &nargs);
-    va_end(vargs);
-    if (stack == NULL) {
-        Py_DECREF(callable);
-        return NULL;
-    }
-
-    result = _PyObject_FastCall(callable, stack, nargs);
-    Py_DECREF(callable);
-    if (stack != small_stack) {
-        PyMem_Free(stack);
-    }
-
-    return result;
-}
-
-PyObject *
-_PyObject_CallMethodIdObjArgs(PyObject *callable,
-        struct _Py_Identifier *name, ...)
-{
-    PyObject *small_stack[5];
-    PyObject **stack;
-    Py_ssize_t nargs;
-    PyObject *result;
-    va_list vargs;
-
-    if (callable == NULL || name == NULL) {
-        return null_error();
-    }
-
-    callable = _PyObject_GetAttrId(callable, name);
-    if (callable == NULL)
-        return NULL;
-
-    /* count the args */
-    va_start(vargs, name);
-    stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack),
-                            vargs, &nargs);
-    va_end(vargs);
-    if (stack == NULL) {
-        Py_DECREF(callable);
-        return NULL;
-    }
-
-    result = _PyObject_FastCall(callable, stack, nargs);
-    Py_DECREF(callable);
-    if (stack != small_stack) {
-        PyMem_Free(stack);
-    }
-
-    return result;
-}
-
-PyObject *
-PyObject_CallFunctionObjArgs(PyObject *callable, ...)
-{
-    PyObject *small_stack[5];
-    PyObject **stack;
-    Py_ssize_t nargs;
-    PyObject *result;
-    va_list vargs;
-
-    if (callable == NULL) {
-        return null_error();
-    }
-
-    /* count the args */
-    va_start(vargs, callable);
+    va_start(vargs, func);
     stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack),
                             vargs, &nargs);
     va_end(vargs);
@@ -2814,7 +2816,7 @@
         return NULL;
     }
 
-    result = _PyObject_FastCall(callable, stack, nargs);
+    result = _PyObject_FastCall(func, stack, nargs);
     if (stack != small_stack) {
         PyMem_Free(stack);
     }
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -1425,15 +1425,16 @@
    as lookup_method to cache the interned name string object. */
 
 static PyObject *
-call_method(PyObject *o, _Py_Identifier *nameid, const char *format, ...)
+call_method(PyObject *obj, _Py_Identifier *method, const char *format, ...)
 {
     va_list va;
-    PyObject *func = NULL, *retval;
-
-    func = lookup_maybe(o, nameid);
+    PyObject *func, *retval;
+
+    func = lookup_maybe(obj, method);
     if (func == NULL) {
-        if (!PyErr_Occurred())
-            PyErr_SetObject(PyExc_AttributeError, nameid->object);
+        if (!PyErr_Occurred()) {
+            PyErr_SetObject(PyExc_AttributeError, method->object);
+        }
         return NULL;
     }
 
@@ -1465,12 +1466,12 @@
 /* Clone of call_method() that returns NotImplemented when the lookup fails. */
 
 static PyObject *
-call_maybe(PyObject *o, _Py_Identifier *nameid, const char *format, ...)
+call_maybe(PyObject *obj, _Py_Identifier *method, const char *format, ...)
 {
     va_list va;
-    PyObject *func = NULL, *retval;
-
-    func = lookup_maybe(o, nameid);
+    PyObject *func, *retval;
+
+    func = lookup_maybe(obj, method);
     if (func == NULL) {
         if (!PyErr_Occurred())
             Py_RETURN_NOTIMPLEMENTED;

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Tue Nov 29 13:49:31 2016
From: python-checkins at python.org (serhiy.storchaka)
Date: Tue, 29 Nov 2016 18:49:31 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0NDY5?=
 =?utf-8?q?=3A_Fixed_memory_leak_caused_by_int_subclasses_without_overridd?=
 =?utf-8?q?en?=
Message-ID: <20161129184930.38880.73267.B07919DF@psf.io>

https://hg.python.org/cpython/rev/fd0842f34602
changeset:   105396:fd0842f34602
branch:      2.7
parent:      105371:59bd48afa1bc
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Tue Nov 29 20:49:14 2016 +0200
summary:
  Issue #24469: Fixed memory leak caused by int subclasses without overridden
tp_free (e.g. C-inherited Cython classes).

files:
  Misc/NEWS           |  3 +++
  Objects/intobject.c |  8 --------
  2 files changed, 3 insertions(+), 8 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #24469: Fixed memory leak caused by int subclasses without overridden
+  tp_free (e.g. C-inherited Cython classes).
+
 - Issue #19398: Extra slash no longer added to sys.path components in case of
   empty compile-time PYTHONPATH components.
 
diff --git a/Objects/intobject.c b/Objects/intobject.c
--- a/Objects/intobject.c
+++ b/Objects/intobject.c
@@ -139,13 +139,6 @@
         Py_TYPE(v)->tp_free((PyObject *)v);
 }
 
-static void
-int_free(PyIntObject *v)
-{
-    Py_TYPE(v) = (struct _typeobject *)free_list;
-    free_list = v;
-}
-
 long
 PyInt_AsLong(register PyObject *op)
 {
@@ -1451,7 +1444,6 @@
     0,                                          /* tp_init */
     0,                                          /* tp_alloc */
     int_new,                                    /* tp_new */
-    (freefunc)int_free,                         /* tp_free */
 };
 
 int

-- 
Repository URL: https://hg.python.org/cpython

From solipsis at pitrou.net  Wed Nov 30 04:05:40 2016
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Wed, 30 Nov 2016 09:05:40 +0000
Subject: [Python-checkins] Daily reference leaks (7efddbf1aa70): sum=4
Message-ID: <20161130090540.105639.81078.72CB956D@psf.io>

results for 7efddbf1aa70 on branch "default"
--------------------------------------------

test_collections leaked [7, 0, -7] memory blocks, sum=0
test_functools leaked [0, 3, 1] memory blocks, sum=4
test_multiprocessing_forkserver leaked [-2, 2, 0] memory blocks, sum=0


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogi_8Rur', '--timeout', '7200']

From python-checkins at python.org  Wed Nov 30 05:21:11 2016
From: python-checkins at python.org (martin.panter)
Date: Wed, 30 Nov 2016 10:21:11 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4ODIw?=
 =?utf-8?q?=3A_Fix_spelling_of_=E2=80=9Cpractice=E2=80=9D_as_a_noun?=
Message-ID: <20161130102111.12504.85351.A549D927@psf.io>

https://hg.python.org/cpython/rev/4f24641cd030
changeset:   105397:4f24641cd030
branch:      2.7
user:        Martin Panter <vadmium+py at gmail.com>
date:        Wed Nov 30 10:18:45 2016 +0000
summary:
  Issue #28820: Fix spelling of ?practice? as a noun

files:
  Doc/howto/urllib2.rst          |  2 +-
  Doc/tutorial/modules.rst       |  2 +-
  Mac/Modules/cg/CFMLateImport.c |  2 +-
  3 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst
--- a/Doc/howto/urllib2.rst
+++ b/Doc/howto/urllib2.rst
@@ -567,7 +567,7 @@
 .. [#] For an introduction to the CGI protocol see
        `Writing Web Applications in Python <http://www.pyzine.com/Issue008/Section_Articles/article_CGIOne.html>`_.
 .. [#] Google for example.
-.. [#] Browser sniffing is a very bad practise for website design - building
+.. [#] Browser sniffing is a very bad practice for website design - building
        sites using web standards is much more sensible. Unfortunately a lot of
        sites still send different versions to different browsers.
 .. [#] The user agent for MSIE 6 is
diff --git a/Doc/tutorial/modules.rst b/Doc/tutorial/modules.rst
--- a/Doc/tutorial/modules.rst
+++ b/Doc/tutorial/modules.rst
@@ -496,7 +496,7 @@
 ``__all__`` is defined.)
 
 Although certain modules are designed to export only names that follow certain
-patterns when you use ``import *``, it is still considered bad practise in
+patterns when you use ``import *``, it is still considered bad practice in
 production code.
 
 Remember, there is nothing wrong with using ``from Package import
diff --git a/Mac/Modules/cg/CFMLateImport.c b/Mac/Modules/cg/CFMLateImport.c
--- a/Mac/Modules/cg/CFMLateImport.c
+++ b/Mac/Modules/cg/CFMLateImport.c
@@ -894,7 +894,7 @@
     MoreAssertQ(fragToFix->sectionHeaders != nil);
     MoreAssertQ(fragToFix->loaderSection != nil);
     MoreAssertQ(fragToFix->section0Base != nil);        // Technically, having a nil for these two is not a problem, ...
-    MoreAssertQ(fragToFix->section1Base != nil);        // but in practise it a wildly deviant case and we should know about it.
+    MoreAssertQ(fragToFix->section1Base != nil);        // but in practice it a wildly deviant case and we should know about it.
     MoreAssertQ(importLibrary != nil);
     MoreAssertQ(lookup != nil);
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 30 06:06:30 2016
From: python-checkins at python.org (martin.panter)
Date: Wed, 30 Nov 2016 11:06:30 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1NzAx?=
 =?utf-8?q?=3A_Document_that_some_C_APIs_can_both_set_and_delete_items?=
Message-ID: <20161130110629.41710.83798.145B35B7@psf.io>

https://hg.python.org/cpython/rev/83e3c863594c
changeset:   105398:83e3c863594c
branch:      2.7
user:        Martin Panter <vadmium+py at gmail.com>
date:        Wed Nov 30 10:32:40 2016 +0000
summary:
  Issue #25701: Document that some C APIs can both set and delete items

Also document that using the dedicated functions to delete items is
preferred. Using PyObject_SetAttr/String() and PySequence_SetItem/Slice() for
deletion is deprecated.

files:
  Doc/c-api/object.rst   |  27 +++++++++++++++++++--------
  Doc/c-api/sequence.rst |  12 ++++++++++--
  Doc/c-api/typeobj.rst  |  26 ++++++++++++++++----------
  Include/abstract.h     |  24 ++++++++++++------------
  4 files changed, 57 insertions(+), 32 deletions(-)


diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst
--- a/Doc/c-api/object.rst
+++ b/Doc/c-api/object.rst
@@ -55,25 +55,35 @@
 .. c:function:: int PyObject_SetAttr(PyObject *o, PyObject *attr_name, PyObject *v)
 
    Set the value of the attribute named *attr_name*, for object *o*, to the value
-   *v*. Returns ``-1`` on failure.  This is the equivalent of the Python statement
+   *v*. Raise an exception and return ``-1`` on failure;
+   return ``0`` on success.  This is the equivalent of the Python statement
    ``o.attr_name = v``.
 
+   If *v* is *NULL*, the attribute is deleted, however this feature is
+   deprecated in favour of using :c:func:`PyObject_DelAttr`.
+
 
 .. c:function:: int PyObject_SetAttrString(PyObject *o, const char *attr_name, PyObject *v)
 
    Set the value of the attribute named *attr_name*, for object *o*, to the value
-   *v*. Returns ``-1`` on failure.  This is the equivalent of the Python statement
+   *v*. Raise an exception and return ``-1`` on failure;
+   return ``0`` on success.  This is the equivalent of the Python statement
    ``o.attr_name = v``.
 
+   If *v* is *NULL*, the attribute is deleted, however this feature is
+   deprecated in favour of using :c:func:`PyObject_DelAttrString`.
+
 
 .. c:function:: int PyObject_GenericSetAttr(PyObject *o, PyObject *name, PyObject *value)
 
-   Generic attribute setter function that is meant to be put into a type
-   object's ``tp_setattro`` slot.  It looks for a data descriptor in the
+   Generic attribute setter and deleter function that is meant
+   to be put into a type object's :c:member:`~PyTypeObject.tp_setattro`
+   slot.  It looks for a data descriptor in the
    dictionary of classes in the object's MRO, and if found it takes preference
-   over setting the attribute in the instance dictionary. Otherwise, the
-   attribute is set in the object's :attr:`~object.__dict__` (if present).
-   Otherwise, an :exc:`AttributeError` is raised and ``-1`` is returned.
+   over setting or deleting the attribute in the instance dictionary. Otherwise, the
+   attribute is set or deleted in the object's :attr:`~object.__dict__` (if present).
+   On success, ``0`` is returned, otherwise an :exc:`AttributeError`
+   is raised and ``-1`` is returned.
 
 
 .. c:function:: int PyObject_DelAttr(PyObject *o, PyObject *attr_name)
@@ -367,7 +377,8 @@
 
 .. c:function:: int PyObject_SetItem(PyObject *o, PyObject *key, PyObject *v)
 
-   Map the object *key* to the value *v*.  Returns ``-1`` on failure.  This is the
+   Map the object *key* to the value *v*.  Raise an exception and
+   return ``-1`` on failure; return ``0`` on success.  This is the
    equivalent of the Python statement ``o[key] = v``.
 
 
diff --git a/Doc/c-api/sequence.rst b/Doc/c-api/sequence.rst
--- a/Doc/c-api/sequence.rst
+++ b/Doc/c-api/sequence.rst
@@ -82,10 +82,14 @@
 
 .. c:function:: int PySequence_SetItem(PyObject *o, Py_ssize_t i, PyObject *v)
 
-   Assign object *v* to the *i*\ th element of *o*.  Returns ``-1`` on failure.  This
+   Assign object *v* to the *i*\ th element of *o*.  Raise an exception
+   and return ``-1`` on failure; return ``0`` on success.  This
    is the equivalent of the Python statement ``o[i] = v``.  This function *does
    not* steal a reference to *v*.
 
+   If *v* is *NULL*, the element is deleted, however this feature is
+   deprecated in favour of using :c:func:`PySequence_DelItem`.
+
    .. versionchanged:: 2.5
       This function used an :c:type:`int` type for *i*. This might require
       changes in your code for properly supporting 64-bit systems.
@@ -104,7 +108,11 @@
 .. c:function:: int PySequence_SetSlice(PyObject *o, Py_ssize_t i1, Py_ssize_t i2, PyObject *v)
 
    Assign the sequence object *v* to the slice in sequence object *o* from *i1* to
-   *i2*.  This is the equivalent of the Python statement ``o[i1:i2] = v``.
+   *i2*.  Raise an exception and return ``-1`` on failure; return ``0`` on success.
+   This is the equivalent of the Python statement ``o[i1:i2] = v``.
+
+   If *v* is *NULL*, the slice is deleted, however this feature is
+   deprecated in favour of using :c:func:`PySequence_DelSlice`.
 
    .. versionchanged:: 2.5
       This function used an :c:type:`int` type for *i1* and *i2*. This might
diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst
--- a/Doc/c-api/typeobj.rst
+++ b/Doc/c-api/typeobj.rst
@@ -240,12 +240,13 @@
 
 .. c:member:: setattrfunc PyTypeObject.tp_setattr
 
-   An optional pointer to the set-attribute-string function.
+   An optional pointer to the function for setting and deleting attributes.
 
    This field is deprecated.  When it is defined, it should point to a function
    that acts the same as the :c:member:`~PyTypeObject.tp_setattro` function, but taking a C string
    instead of a Python string object to give the attribute name.  The signature is
-   the same as for :c:func:`PyObject_SetAttrString`.
+   the same as for :c:func:`PyObject_SetAttrString`, but setting
+   *v* to *NULL* to delete an attribute must be supported.
 
    This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_setattro`: a subtype
    inherits both :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` from its base type when
@@ -389,9 +390,10 @@
 
 .. c:member:: setattrofunc PyTypeObject.tp_setattro
 
-   An optional pointer to the set-attribute function.
+   An optional pointer to the function for setting and deleting attributes.
 
-   The signature is the same as for :c:func:`PyObject_SetAttr`.  It is usually
+   The signature is the same as for :c:func:`PyObject_SetAttr`, but setting
+   *v* to *NULL* to delete an attribute must be supported.  It is usually
    convenient to set this field to :c:func:`PyObject_GenericSetAttr`, which
    implements the normal way of setting object attributes.
 
@@ -831,7 +833,7 @@
       typedef struct PyGetSetDef {
           char *name;    /* attribute name */
           getter get;    /* C function to get the attribute */
-          setter set;    /* C function to set the attribute */
+          setter set;    /* C function to set or delete the attribute */
           char *doc;     /* optional doc string */
           void *closure; /* optional additional data for getter and setter */
       } PyGetSetDef;
@@ -877,12 +879,14 @@
 
 .. c:member:: descrsetfunc PyTypeObject.tp_descr_set
 
-   An optional pointer to a "descriptor set" function.
+   An optional pointer to a function for setting and deleting
+   a descriptor's value.
 
    The function signature is ::
 
       int tp_descr_set(PyObject *self, PyObject *obj, PyObject *value);
 
+   The *value* argument is set to *NULL* to delete the value.
    This field is inherited by subtypes.
 
    .. XXX explain.
@@ -1263,9 +1267,11 @@
 
 .. c:member:: objobjargproc PyMappingMethods.mp_ass_subscript
 
-   This function is used by :c:func:`PyObject_SetItem` and has the same
-   signature.  If this slot is *NULL*, the object does not support item
-   assignment.
+   This function is used by :c:func:`PyObject_SetItem` and
+   :c:func:`PyObject_DelItem`.  It has the same signature as
+   :c:func:`PyObject_SetItem`, but *v* can also be set to *NULL* to delete
+   an item.  If this slot is *NULL*, the object does not support item
+   assignment and deletion.
 
 
 .. _sequence-structs:
@@ -1314,7 +1320,7 @@
 
    This function is used by :c:func:`PySequence_SetItem` and has the same
    signature.  This slot may be left to *NULL* if the object does not support
-   item assignment.
+   item assignment and deletion.
 
 .. c:member:: objobjproc PySequenceMethods.sq_contains
 
diff --git a/Include/abstract.h b/Include/abstract.h
--- a/Include/abstract.h
+++ b/Include/abstract.h
@@ -191,8 +191,8 @@
      int PyObject_SetAttrString(PyObject *o, char *attr_name, PyObject *v);
 
      Set the value of the attribute named attr_name, for object o,
-     to the value, v. Returns -1 on failure.  This is
-     the equivalent of the Python statement: o.attr_name=v.
+     to the value v. Raise an exception and return -1 on failure; return 0 on
+     success.  This is the equivalent of the Python statement o.attr_name=v.
 
        */
 
@@ -201,8 +201,8 @@
      int PyObject_SetAttr(PyObject *o, PyObject *attr_name, PyObject *v);
 
      Set the value of the attribute named attr_name, for object o,
-     to the value, v. Returns -1 on failure.  This is
-     the equivalent of the Python statement: o.attr_name=v.
+     to the value v. Raise an exception and return -1 on failure; return 0 on
+     success.  This is the equivalent of the Python statement o.attr_name=v.
 
        */
 
@@ -453,9 +453,9 @@
      PyAPI_FUNC(int) PyObject_SetItem(PyObject *o, PyObject *key, PyObject *v);
 
        /*
-     Map the object, key, to the value, v.  Returns
-     -1 on failure.  This is the equivalent of the Python
-     statement: o[key]=v.
+     Map the object key to the value v.  Raise an exception and return -1
+     on failure; return 0 on success.  This is the equivalent of the Python
+     statement o[key]=v.
        */
 
      PyAPI_FUNC(int) PyObject_DelItemString(PyObject *o, char *key);
@@ -1102,10 +1102,9 @@
      PyAPI_FUNC(int) PySequence_SetItem(PyObject *o, Py_ssize_t i, PyObject *v);
 
        /*
-     Assign object v to the ith element of o.  Returns
-     -1 on failure.  This is the equivalent of the Python
-     statement: o[i]=v.
-
+     Assign object v to the ith element of o.  Raise an exception and return
+     -1 on failure; return 0 on success.  This is the equivalent of the
+     Python statement o[i]=v.
        */
 
      PyAPI_FUNC(int) PySequence_DelItem(PyObject *o, Py_ssize_t i);
@@ -1121,7 +1120,8 @@
 
        /*
      Assign the sequence object, v, to the slice in sequence
-     object, o, from i1 to i2.  Returns -1 on failure. This is the
+     object, o, from i1 to i2.  Raise an exception and return
+     -1 on failure; return 0 on success.  This is the
      equivalent of the Python statement: o[i1:i2]=v.
        */
 

-- 
Repository URL: https://hg.python.org/cpython

From python-checkins at python.org  Wed Nov 30 06:12:40 2016
From: python-checkins at python.org (victor.stinner)
Date: Wed, 30 Nov 2016 11:12:40 +0000
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Backed_out_changeset_7efdd?=
 =?utf-8?q?bf1aa70?=
Message-ID: <20161130111240.12418.14509.423AEFB7@psf.io>

https://hg.python.org/cpython/rev/20bb8babc505
changeset:   105399:20bb8babc505
parent:      105395:7efddbf1aa70
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Wed Nov 30 12:10:54 2016 +0100
summary:
  Backed out changeset 7efddbf1aa70

files:
  Include/abstract.h   |   40 ++--
  Include/ceval.h      |    4 +-
  Objects/abstract.c   |  236 +++++++++++++++---------------
  Objects/typeobject.c |   21 +-
  4 files changed, 149 insertions(+), 152 deletions(-)


diff --git a/Include/abstract.h b/Include/abstract.h
--- a/Include/abstract.h
+++ b/Include/abstract.h
@@ -257,15 +257,15 @@
 
        /* Declared elsewhere
 
-     PyAPI_FUNC(int) PyCallable_Check(PyObject *obj);
+     PyAPI_FUNC(int) PyCallable_Check(PyObject *o);
 
-     Determine if the object, obj, is callable.  Return 1 if the
+     Determine if the object, o, is callable.  Return 1 if the
      object is callable and 0 otherwise.
 
      This function always succeeds.
        */
 
-     PyAPI_FUNC(PyObject *) PyObject_Call(PyObject *func,
+     PyAPI_FUNC(PyObject *) PyObject_Call(PyObject *callable_object,
                                           PyObject *args, PyObject *kwargs);
 
 #ifndef Py_LIMITED_API
@@ -344,7 +344,7 @@
     _PyObject_FastCall((func), &(arg), 1)
 
     PyAPI_FUNC(PyObject *) _PyObject_Call_Prepend(PyObject *func,
-                                                  PyObject *arg0, PyObject *args,
+                                                  PyObject *obj, PyObject *args,
                                                   PyObject *kwargs);
 
      PyAPI_FUNC(PyObject *) _Py_CheckFunctionResult(PyObject *func,
@@ -353,27 +353,27 @@
 #endif   /* Py_LIMITED_API */
 
        /*
-     Call a callable Python object, func, with
+     Call a callable Python object, callable_object, with
      arguments and keywords arguments.  The 'args' argument can not be
      NULL.
        */
 
-     PyAPI_FUNC(PyObject *) PyObject_CallObject(PyObject *func,
+     PyAPI_FUNC(PyObject *) PyObject_CallObject(PyObject *callable_object,
                                                 PyObject *args);
 
        /*
-     Call a callable Python object, func, with
+     Call a callable Python object, callable_object, with
      arguments given by the tuple, args.  If no arguments are
      needed, then args may be NULL.  Returns the result of the
      call on success, or NULL on failure.  This is the equivalent
      of the Python expression: o(*args).
        */
 
-     PyAPI_FUNC(PyObject *) PyObject_CallFunction(PyObject *func,
+     PyAPI_FUNC(PyObject *) PyObject_CallFunction(PyObject *callable_object,
                                                   const char *format, ...);
 
        /*
-     Call a callable Python object, func, with a
+     Call a callable Python object, callable_object, with a
      variable number of C arguments. The C arguments are described
      using a mkvalue-style format string. The format may be NULL,
      indicating that no arguments are provided.  Returns the
@@ -382,7 +382,7 @@
        */
 
 
-     PyAPI_FUNC(PyObject *) PyObject_CallMethod(PyObject *obj,
+     PyAPI_FUNC(PyObject *) PyObject_CallMethod(PyObject *o,
                                                 const char *method,
                                                 const char *format, ...);
 
@@ -396,7 +396,7 @@
        */
 
 #ifndef Py_LIMITED_API
-     PyAPI_FUNC(PyObject *) _PyObject_CallMethodId(PyObject *obj,
+     PyAPI_FUNC(PyObject *) _PyObject_CallMethodId(PyObject *o,
                                                    _Py_Identifier *method,
                                                    const char *format, ...);
 
@@ -406,25 +406,25 @@
        */
 #endif /* !Py_LIMITED_API */
 
-     PyAPI_FUNC(PyObject *) _PyObject_CallFunction_SizeT(PyObject *func,
+     PyAPI_FUNC(PyObject *) _PyObject_CallFunction_SizeT(PyObject *callable,
                                                          const char *format,
                                                          ...);
-     PyAPI_FUNC(PyObject *) _PyObject_CallMethod_SizeT(PyObject *obj,
-                                                       const char *method,
+     PyAPI_FUNC(PyObject *) _PyObject_CallMethod_SizeT(PyObject *o,
+                                                       const char *name,
                                                        const char *format,
                                                        ...);
 #ifndef Py_LIMITED_API
-     PyAPI_FUNC(PyObject *) _PyObject_CallMethodId_SizeT(PyObject *obj,
-                                                       _Py_Identifier *method,
+     PyAPI_FUNC(PyObject *) _PyObject_CallMethodId_SizeT(PyObject *o,
+                                                       _Py_Identifier *name,
                                                        const char *format,
                                                        ...);
 #endif /* !Py_LIMITED_API */
 
-     PyAPI_FUNC(PyObject *) PyObject_CallFunctionObjArgs(PyObject *func,
+     PyAPI_FUNC(PyObject *) PyObject_CallFunctionObjArgs(PyObject *callable,
                                                          ...);
 
        /*
-     Call a callable Python object, func, with a
+     Call a callable Python object, callable_object, with a
      variable number of C arguments.  The C arguments are provided
      as PyObject * values, terminated by a NULL.  Returns the
      result of the call on success, or NULL on failure.  This is
@@ -432,10 +432,10 @@
        */
 
 
-     PyAPI_FUNC(PyObject *) PyObject_CallMethodObjArgs(PyObject *obj,
+     PyAPI_FUNC(PyObject *) PyObject_CallMethodObjArgs(PyObject *o,
                                                        PyObject *method, ...);
 #ifndef Py_LIMITED_API
-     PyAPI_FUNC(PyObject *) _PyObject_CallMethodIdObjArgs(PyObject *obj,
+     PyAPI_FUNC(PyObject *) _PyObject_CallMethodIdObjArgs(PyObject *o,
                                                struct _Py_Identifier *method,
                                                ...);
 #endif /* !Py_LIMITED_API */
diff --git a/Include/ceval.h b/Include/ceval.h
--- a/Include/ceval.h
+++ b/Include/ceval.h
@@ -14,10 +14,10 @@
 #define PyEval_CallObject(func,arg) \
     PyEval_CallObjectWithKeywords(func, arg, (PyObject *)NULL)
 
-PyAPI_FUNC(PyObject *) PyEval_CallFunction(PyObject *func,
+PyAPI_FUNC(PyObject *) PyEval_CallFunction(PyObject *obj,
                                            const char *format, ...);
 PyAPI_FUNC(PyObject *) PyEval_CallMethod(PyObject *obj,
-                                         const char *method,
+                                         const char *methodname,
                                          const char *format, ...);
 
 #ifndef Py_LIMITED_API
diff --git a/Objects/abstract.c b/Objects/abstract.c
--- a/Objects/abstract.c
+++ b/Objects/abstract.c
@@ -2173,9 +2173,9 @@
 /* XXX PyCallable_Check() is in object.c */
 
 PyObject *
-PyObject_CallObject(PyObject *func, PyObject *args)
+PyObject_CallObject(PyObject *o, PyObject *a)
 {
-    return PyEval_CallObjectWithKeywords(func, args, NULL);
+    return PyEval_CallObjectWithKeywords(o, a, NULL);
 }
 
 PyObject*
@@ -2331,10 +2331,10 @@
     return result;
 }
 
-/* Positional arguments are arg0 followed args: [arg0, *args]. */
+/* Positional arguments are obj followed args. */
 PyObject *
 _PyObject_Call_Prepend(PyObject *func,
-                       PyObject *arg0, PyObject *args, PyObject *kwargs)
+                       PyObject *obj, PyObject *args, PyObject *kwargs)
 {
     PyObject *small_stack[8];
     PyObject **stack;
@@ -2356,7 +2356,7 @@
     }
 
     /* use borrowed references */
-    stack[0] = arg0;
+    stack[0] = obj;
     memcpy(&stack[1],
               &PyTuple_GET_ITEM(args, 0),
               argcount * sizeof(PyObject *));
@@ -2489,34 +2489,34 @@
 }
 
 static PyObject*
-call_function_tail(PyObject *func, PyObject *args)
+call_function_tail(PyObject *callable, PyObject *args)
 {
     PyObject *result;
 
     assert(args != NULL);
 
     if (!PyTuple_Check(args)) {
-        result = _PyObject_CallArg1(func, args);
+        result = _PyObject_CallArg1(callable, args);
     }
     else {
-        result = PyObject_Call(func, args, NULL);
+        result = PyObject_Call(callable, args, NULL);
     }
 
     return result;
 }
 
 PyObject *
-PyObject_CallFunction(PyObject *func, const char *format, ...)
+PyObject_CallFunction(PyObject *callable, const char *format, ...)
 {
     va_list va;
     PyObject *args, *result;
 
-    if (func == NULL) {
+    if (callable == NULL) {
         return null_error();
     }
 
     if (!format || !*format) {
-        return _PyObject_CallNoArg(func);
+        return _PyObject_CallNoArg(callable);
     }
 
     va_start(va, format);
@@ -2526,23 +2526,23 @@
         return NULL;
     }
 
-    result = call_function_tail(func, args);
+    result = call_function_tail(callable, args);
     Py_DECREF(args);
     return result;
 }
 
 PyObject *
-_PyObject_CallFunction_SizeT(PyObject *func, const char *format, ...)
+_PyObject_CallFunction_SizeT(PyObject *callable, const char *format, ...)
 {
     va_list va;
     PyObject *args, *result;
 
-    if (func == NULL) {
+    if (callable == NULL) {
         return null_error();
     }
 
     if (!format || !*format) {
-        return _PyObject_CallNoArg(func);
+        return _PyObject_CallNoArg(callable);
     }
 
     va_start(va, format);
@@ -2552,7 +2552,7 @@
         return NULL;
     }
 
-    result = call_function_tail(func, args);
+    result = call_function_tail(callable, args);
     Py_DECREF(args);
     return result;
 }
@@ -2589,20 +2589,19 @@
 }
 
 PyObject *
-PyObject_CallMethod(PyObject *obj, const char *method, const char *format, ...)
+PyObject_CallMethod(PyObject *o, const char *name, const char *format, ...)
 {
     va_list va;
     PyObject *func = NULL;
     PyObject *retval = NULL;
 
-    if (obj == NULL || method == NULL) {
+    if (o == NULL || name == NULL) {
         return null_error();
     }
 
-    func = PyObject_GetAttrString(obj, method);
-    if (func == NULL) {
+    func = PyObject_GetAttrString(o, name);
+    if (func == NULL)
         return NULL;
-    }
 
     va_start(va, format);
     retval = callmethod(func, format, va, 0);
@@ -2612,20 +2611,20 @@
 }
 
 PyObject *
-_PyObject_CallMethodId(PyObject *obj, _Py_Identifier *method,
+_PyObject_CallMethodId(PyObject *o, _Py_Identifier *name,
                        const char *format, ...)
 {
     va_list va;
-    PyObject *func, *retval;
-
-    if (obj == NULL || method == NULL) {
+    PyObject *func = NULL;
+    PyObject *retval = NULL;
+
+    if (o == NULL || name == NULL) {
         return null_error();
     }
 
-    func = _PyObject_GetAttrId(obj, method);
-    if (func == NULL) {
+    func = _PyObject_GetAttrId(o, name);
+    if (func == NULL)
         return NULL;
-    }
 
     va_start(va, format);
     retval = callmethod(func, format, va, 0);
@@ -2635,21 +2634,20 @@
 }
 
 PyObject *
-_PyObject_CallMethod_SizeT(PyObject *obj, const char *method,
+_PyObject_CallMethod_SizeT(PyObject *o, const char *name,
                            const char *format, ...)
 {
     va_list va;
-    PyObject *func, *retval;
-
-    if (obj == NULL || method == NULL) {
+    PyObject *func = NULL;
+    PyObject *retval;
+
+    if (o == NULL || name == NULL) {
         return null_error();
     }
 
-    func = PyObject_GetAttrString(obj, method);
-    if (func == NULL) {
+    func = PyObject_GetAttrString(o, name);
+    if (func == NULL)
         return NULL;
-    }
-
     va_start(va, format);
     retval = callmethod(func, format, va, 1);
     va_end(va);
@@ -2658,21 +2656,21 @@
 }
 
 PyObject *
-_PyObject_CallMethodId_SizeT(PyObject *obj, _Py_Identifier *method,
+_PyObject_CallMethodId_SizeT(PyObject *o, _Py_Identifier *name,
                              const char *format, ...)
 {
     va_list va;
-    PyObject *func, *retval;
-
-    if (obj == NULL || method == NULL) {
+    PyObject *func = NULL;
+    PyObject *retval;
+
+    if (o == NULL || name == NULL) {
         return null_error();
     }
 
-    func = _PyObject_GetAttrId(obj, method);
+    func = _PyObject_GetAttrId(o, name);
     if (func == NULL) {
         return NULL;
     }
-
     va_start(va, format);
     retval = callmethod(func, format, va, 1);
     va_end(va);
@@ -2722,80 +2720,7 @@
 }
 
 PyObject *
-PyObject_CallMethodObjArgs(PyObject *obj, PyObject *method, ...)
-{
-    PyObject *small_stack[5];
-    PyObject **stack;
-    Py_ssize_t nargs;
-    PyObject *func, *result;
-    va_list vargs;
-
-    if (obj == NULL || method == NULL) {
-        return null_error();
-    }
-
-    func = PyObject_GetAttr(obj, method);
-    if (func == NULL)
-        return NULL;
-
-    /* count the args */
-    va_start(vargs, method);
-    stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack),
-                            vargs, &nargs);
-    va_end(vargs);
-    if (stack == NULL) {
-        Py_DECREF(func);
-        return NULL;
-    }
-
-    result = _PyObject_FastCall(func, stack, nargs);
-    Py_DECREF(func);
-    if (stack != small_stack) {
-        PyMem_Free(stack);
-    }
-
-    return result;
-}
-
-PyObject *
-_PyObject_CallMethodIdObjArgs(PyObject *obj,
-                              struct _Py_Identifier *method, ...)
-{
-    PyObject *small_stack[5];
-    PyObject **stack;
-    Py_ssize_t nargs;
-    PyObject *func, *result;
-    va_list vargs;
-
-    if (obj == NULL || method == NULL) {
-        return null_error();
-    }
-
-    func = _PyObject_GetAttrId(obj, method);
-    if (func == NULL)
-        return NULL;
-
-    /* count the args */
-    va_start(vargs, method);
-    stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack),
-                            vargs, &nargs);
-    va_end(vargs);
-    if (stack == NULL) {
-        Py_DECREF(func);
-        return NULL;
-    }
-
-    result = _PyObject_FastCall(func, stack, nargs);
-    Py_DECREF(func);
-    if (stack != small_stack) {
-        PyMem_Free(stack);
-    }
-
-    return result;
-}
-
-PyObject *
-PyObject_CallFunctionObjArgs(PyObject *func, ...)
+PyObject_CallMethodObjArgs(PyObject *callable, PyObject *name, ...)
 {
     PyObject *small_stack[5];
     PyObject **stack;
@@ -2803,12 +2728,85 @@
     PyObject *result;
     va_list vargs;
 
-    if (func == NULL) {
+    if (callable == NULL || name == NULL) {
         return null_error();
     }
 
+    callable = PyObject_GetAttr(callable, name);
+    if (callable == NULL)
+        return NULL;
+
     /* count the args */
-    va_start(vargs, func);
+    va_start(vargs, name);
+    stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack),
+                            vargs, &nargs);
+    va_end(vargs);
+    if (stack == NULL) {
+        Py_DECREF(callable);
+        return NULL;
+    }
+
+    result = _PyObject_FastCall(callable, stack, nargs);
+    Py_DECREF(callable);
+    if (stack != small_stack) {
+        PyMem_Free(stack);
+    }
+
+    return result;
+}
+
+PyObject *
+_PyObject_CallMethodIdObjArgs(PyObject *callable,
+        struct _Py_Identifier *name, ...)
+{
+    PyObject *small_stack[5];
+    PyObject **stack;
+    Py_ssize_t nargs;
+    PyObject *result;
+    va_list vargs;
+
+    if (callable == NULL || name == NULL) {
+        return null_error();
+    }
+
+    callable = _PyObject_GetAttrId(callable, name);
+    if (callable == NULL)
+        return NULL;
+
+    /* count the args */
+    va_start(vargs, name);
+    stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack),
+                            vargs, &nargs);
+    va_end(vargs);
+    if (stack == NULL) {
+        Py_DECREF(callable);
+        return NULL;
+    }
+
+    result = _PyObject_FastCall(callable, stack, nargs);
+    Py_DECREF(callable);
+    if (stack != small_stack) {
+        PyMem_Free(stack);
+    }
+
+    return result;
+}
+
+PyObject *
+PyObject_CallFunctionObjArgs(PyObject *callable, ...)
+{
+    PyObject *small_stack[5];
+    PyObject **stack;
+    Py_ssize_t nargs;
+    PyObject *result;
+    va_list vargs;
+
+    if (callable == NULL) {
+        return null_error();
+    }
+
+    /* count the args */
+    va_start(vargs, callable);
     stack = objargs_mkstack(small_stack, Py_ARRAY_LENGTH(small_stack),
                             vargs, &nargs);
     va_end(vargs);
@@ -2816,7 +2814,7 @@
         return NULL;
     }
 
-    result = _PyObject_FastCall(func, stack, nargs);
+    result = _PyObject_FastCall(callable, stack, nargs);
     if (stack != small_stack) {
         PyMem_Free(stack);
     }
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -1425,16 +1425,15 @@
    as lookup_method to cache the interned name string object. */
 
 static PyObject *
-call_method(PyObject *obj, _Py_Identifier *method, const char *format, ...)
+call_method(PyObject *o, _Py_Identifier *nameid, const char *format, ...)
 {
     va_list va;
-    PyObject *func, *retval;
-
-    func = lookup_maybe(obj, method);
+    PyObject *func = NULL, *retval;
+
+    func = lookup_maybe(o, nameid);
     if (func == NULL) {
-        if (!PyErr_Occurred()) {
-            PyErr_SetObject(PyExc_AttributeError, method->object);
-        }
+        if (!PyErr_Occurred())
+            PyErr_SetObject(PyExc_AttributeError, nameid->object);
         return NULL;
     }
 
@@ -1466,12 +1465,12 @@
 /* Clone of call_method() that returns NotImplemented when the lookup fails. */
 
 static PyObject *
-call_maybe(PyObject *obj, _Py_Identifier *method, const char *format, ...)
+call_maybe(PyObject *o, _Py_Identifier *nameid, const char *format, ...)
 {
     va_list va;
-    PyObject *func, *retval;
-
-    func = lookup_maybe(obj, method);
+    PyObject *func = NULL, *retval;
+
+    func = lookup_maybe(o, nameid);
     if (func == NULL) {
         if (!PyErr_Occurred())
             Py_RETURN_NOTIMPLEMENTED;

-- 
Repository URL: https://hg.python.org/cpython

From tjreedy at udel.edu  Wed Nov 30 13:35:48 2016
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 30 Nov 2016 13:35:48 -0500
Subject: [Python-checkins] 
 =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4ODIw?=
 =?utf-8?q?=3A_Fix_spelling_of_=E2=80=9Cpractice=E2=80=9D_as_a_noun?=
In-Reply-To: <20161130102111.12504.85351.A549D927@psf.io>
References: <20161130102111.12504.85351.A549D927@psf.io>
Message-ID: <3730fb6c-416b-516c-8ae6-4382e143413a@udel.edu>

On 11/30/2016 5:21 AM, martin.panter wrote:
> https://hg.python.org/cpython/rev/4f24641cd030
> changeset:   105397:4f24641cd030
> branch:      2.7
> user:        Martin Panter <vadmium+py at gmail.com>
> date:        Wed Nov 30 10:18:45 2016 +0000
> summary:
>   Issue #28820: Fix spelling of ?practice? as a noun

In American English, the verb is also 'practice',
so 'practise' should never appear in our docs.
http://grammarist.com/spelling/practice-practise/


From python-checkins at python.org  Wed Nov 30 20:33:12 2016
From: python-checkins at python.org (martin.panter)
Date: Thu, 01 Dec 2016 01:33:12 +0000
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4Nzcx?=
 =?utf-8?q?=3A_Correct_documentation_of_signatures_using_const?=
Message-ID: <20161201013312.30611.97025.0BB377B0@psf.io>

https://hg.python.org/cpython/rev/2fd070fa6c15
changeset:   105400:2fd070fa6c15
branch:      2.7
parent:      105398:83e3c863594c
user:        Martin Panter <vadmium+py at gmail.com>
date:        Thu Dec 01 01:32:16 2016 +0000
summary:
  Issue #28771: Correct documentation of signatures using const

files:
  Doc/c-api/typeobj.rst |  14 ++++++++------
  1 files changed, 8 insertions(+), 6 deletions(-)


diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst
--- a/Doc/c-api/typeobj.rst
+++ b/Doc/c-api/typeobj.rst
@@ -230,8 +230,9 @@
 
    This field is deprecated.  When it is defined, it should point to a function
    that acts the same as the :c:member:`~PyTypeObject.tp_getattro` function, but taking a C string
-   instead of a Python string object to give the attribute name.  The signature is
-   the same as for :c:func:`PyObject_GetAttrString`.
+   instead of a Python string object to give the attribute name.  The signature is ::
+
+      PyObject * tp_getattr(PyObject *o, char *attr_name);
 
    This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_getattro`: a subtype
    inherits both :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` from its base type when
@@ -244,10 +245,11 @@
 
    This field is deprecated.  When it is defined, it should point to a function
    that acts the same as the :c:member:`~PyTypeObject.tp_setattro` function, but taking a C string
-   instead of a Python string object to give the attribute name.  The signature is
-   the same as for :c:func:`PyObject_SetAttrString`, but setting
-   *v* to *NULL* to delete an attribute must be supported.
+   instead of a Python string object to give the attribute name.  The signature is ::
 
+      PyObject * tp_setattr(PyObject *o, char *attr_name, PyObject *v);
+
+   The *v* argument is set to *NULL* to delete the attribute.
    This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_setattro`: a subtype
    inherits both :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` from its base type when
    the subtype's :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` are both *NULL*.
@@ -1450,7 +1452,7 @@
    all segments in ``*lenp``. The function cannot fail.
 
 
-.. c:type:: Py_ssize_t (*charbufferproc) (PyObject *self, Py_ssize_t segment, const char **ptrptr)
+.. c:type:: Py_ssize_t (*charbufferproc) (PyObject *self, Py_ssize_t segment, char **ptrptr)
 
    Return the size of the segment *segment* that *ptrptr*  is set to.  ``*ptrptr``
    is set to the memory buffer. Returns ``-1`` on error.

-- 
Repository URL: https://hg.python.org/cpython